The write side is four operations — index, get, update, delete — and two ideas that are not obvious from the operations themselves: what a document id is for, and why a document you just wrote is not immediately searchable.
Indexing a document
Two forms, and the difference is whether you choose the id.
# you choose the id — idempotent
PUT /stayhub-properties/_doc/8f1e-...
{ "title": "Cedar Cabin", "city": "Big Bear Lake" }
# Elasticsearch chooses — a new document every time
POST /stayhub-properties/_doc
{ "title": "Cedar Cabin", "city": "Big Bear Lake" }Use the first. Always, when the document corresponds to something that already has an identity in your system:
client.index(
index=settings.elasticsearch_index,
id=str(prop.public_id),
document=to_document(prop),
)Because the document id is the listing's public id, indexing the same listing twice updates rather than duplicating, deleting it is a one-line call with no lookup, and a rebuild produces byte-for-byte the same index. That is what makes indexing idempotent, and idempotence is what makes every retry strategy in lesson 8 safe rather than merely hopeful.
Let Elasticsearch generate ids and a retried write silently becomes two documents. There is no uniqueness constraint to save you.
The response, and the four numbers in it
{
"_index": "stayhub-properties-000002",
"_id": "8f1e-...",
"_version": 1,
"result": "created",
"_seq_no": 0,
"_primary_term": 1
}result is created the first time and updated after that.
That is a genuinely useful signal — it tells you whether your idea of "new" agrees with the
index's.
_version increments on every write. It is not for concurrency control any
more, despite what older documentation says; it is informational.
_seq_no and _primary_term are the pair that replaced it, and they are
covered below.
Refusing to overwrite
When creating something that must not already exist, say so:
PUT /stayhub-properties/_doc/8f1e-...?op_type=create409 version_conflict_engine_exception
[1]: version conflict, document already exists (current version [2])A clean 409 instead of a silent overwrite. This is the closest thing Elasticsearch has to a
unique constraint, and it only works on the _id.
The thing that confuses everyone
Index a document and immediately search for it:
PUT /doc-test/_doc/1 {"title": "Cedar Cabin"} # "result": "created"
GET /doc-test/_search {"query": {"match_all": {}}}
# hits: 0Zero hits. The write succeeded — it said so. Now fetch it by id:
GET /doc-test/_doc/1
# { "found": true, "_source": { "title": "Cedar Cabin" } }Found. The same document, in the same index, at the same moment: invisible to search, visible to a direct lookup.
This is not a bug and it is not a race you can wait out with a sleep. Elasticsearch is near-real-time, and the two operations read from different places.
A write goes into an in-memory buffer and the transaction log. GET by id reads the
translog, so it always sees the newest version — it is "realtime" by design. Search reads
segments, which are immutable files, and a document is only searchable once it has been
written into one. That step is called a refresh, and by default it happens once a
second on any index that has been searched recently.
refresh, and what it costs
POST /doc-test/_refresh # refresh the index now
PUT /doc-test/_doc/1?refresh=true # write, then refresh, then return
PUT /doc-test/_doc/1?refresh=wait_for # write, return when the next refresh happensThe distinction between the last two matters more than it looks.
refresh=true forces a refresh, creating a new segment immediately. Do that on
every write and you create a segment per write, and the background merging of thousands of tiny
segments will consume the cluster.
refresh=wait_for does not force anything — it holds your request open until
the scheduled refresh comes round. Same visibility guarantee, none of the segment churn. It is
almost always the right choice when a caller genuinely needs read-your-writes.
StayHub keeps forcing to one place, with the reason written next to it:
def refresh_index(*, es: Elasticsearch | None = None) -> None:
"""Force pending writes to become searchable NOW.
⚠️ Elasticsearch is near-real-time: an indexed document is normally visible about a second
later, so a test that indexes and immediately searches finds nothing and looks like a broken
query. This exists for tests and the seed script — **never call it per write in production**,
it defeats the batching that makes indexing fast.
"""Going the other way is a real tuning lever. If an index is written far more than it is searched — a bulk load, a log ingest — raising the interval reduces segment creation significantly:
PUT /stayhub-properties/_settings
{ "index": { "refresh_interval": "30s" } }
# or "-1" to disable it entirely during a bulk load, then set it backWhat you must not do is design an application around one-second visibility. Show the user the value they just submitted from your own database, not from a search that may not see it yet.
Updating
Elasticsearch documents are immutable. Every "update" reads the current
_source, applies the change, indexes the whole document again, and marks the old one
deleted. Knowing that explains most of the behaviour.
POST /doc-test/_update/1
{ "doc": { "views": 5 } }
# { "result": "updated", "_version": 3 }Send the same update again and it notices:
POST /doc-test/_update/1
{ "doc": { "views": 5 } }
# { "result": "noop", "_version": 3 }noop, and the version did not move. Elasticsearch compared the merged source to the
existing one and skipped the write. This is free deduplication and it is worth knowing about,
because it means re-sending unchanged documents is cheap — which makes "just reindex it
again" a reasonable repair strategy.
There is also a scripted form, for changes that depend on the current value:
POST /doc-test/_update/1
{ "script": { "source": "ctx._source.views += params.n", "params": { "n": 3 } } }Useful, and not a substitute for a real counter. It still rewrites the whole document, it runs a script per call, and under concurrency it needs the version checks below. Lesson 6's advice stands: counters belong in your database.
The upsert variant handles "update if present, create if not" in one call, which
saves a round trip and a race:
{
"doc": { "views": 1 },
"doc_as_upsert": true
}Why "immutable" explains the rest of the behaviour
Several things that look unrelated are the same fact seen from different angles.
Updating one field costs the same as replacing the document. There is no partial
write — the whole _source is re-indexed. So a document with a 50KB description
and a hit counter is expensive to increment, which is the mechanical reason behind lesson 6's rule
about volatile fields.
An update needs _source to exist. Disable
_source to save disk and you lose the update API, reindexing, and the ability to see
what you actually stored. Almost never worth it.
Deletes and updates both leave tombstones, so an index that is rewritten
constantly grows and then shrinks in steps as merges catch up. A docs.deleted that is
a large fraction of docs.count is a sign of a write pattern worth looking at, not
usually an emergency.
The translog is what makes a write durable before it is in a segment. By default it is fsynced on every request, which is why an acknowledged write survives a crash. There is a setting to make that asynchronous for more throughput; it trades a few seconds of acknowledged writes on a hard failure, which for a derived index is often an acceptable trade and for a source of truth never is.
Concurrency: seq_no and primary_term
Two clients read the same document, both modify it, both write. The second overwrites the first and nobody notices. The fix is optimistic concurrency control: read the document, note its position, and make the write conditional on nothing having changed since.
GET /doc-test/_doc/1
# "_seq_no": 1, "_primary_term": 1
PUT /doc-test/_doc/1?if_seq_no=0&if_primary_term=1
{ "title": "stale write" }409 version_conflict_engine_exception
[1]: version conflict, required seqNo [0], primary term [1].
current document has seqNo [1] and primary term [1]Refused, with both the expected and the actual values. The caller re-reads and retries.
Why two numbers rather than one? _seq_no is a counter per shard, incremented on
every write. _primary_term increments whenever a shard gets a new primary, which
happens after a failover. Together they are unique across the life of the shard even if a failover
resets things — a single counter could repeat after a primary change, and the whole point is
to be sure nothing was missed.
In practice you need this rarely, and StayHub does not use it at all. That is not an oversight: Postgres is the source of truth, indexing is idempotent by document id, and the outbox handler re-reads from the database rather than writing a snapshot — so a lost update in the index is repaired by the next write rather than being a permanent divergence. Optimistic concurrency matters when Elasticsearch is the store, which lesson 1 argued against.
What "acknowledged" actually promises
A successful write response has a _shards block, and it is the part everyone skips:
{ "result": "created", "_shards": { "total": 2, "successful": 1, "failed": 0 } }total counts the primary plus its replicas. successful is how many
actually took the write. On a single-node cluster with no replicas that is 1 of 2 — the
replica does not exist, so it cannot be written to, and the request still succeeded.
That is the default consistency: a write is acknowledged once the primary has it and the replicas have been asked. If a replica fails to apply it, the response still says success and the cluster repairs the replica in the background.
You can demand more with wait_for_active_shards, which refuses the write unless a
given number of copies are available before it starts. What you cannot do is get a two-phase commit,
because there is not one — which is another way of saying that if losing a write would be
unacceptable, that write's home is not Elasticsearch.
Reading failed is worth building into any indexing code that matters. It is
non-zero far more often than people expect, and nothing surfaces it for you.
Deleting
DELETE /doc-test/_doc/1
# { "result": "deleted" }
DELETE /doc-test/_doc/nope
# { "result": "not_found" }Deleting something that is not there returns not_found with a 404 status rather than
an error, and in most applications that is exactly right — removing a listing that was never
indexed is a no-op, not a failure:
client.options(ignore_status=404).delete(index=settings.elasticsearch_index, id=public_id)A delete does not free space immediately. It writes a tombstone; the document stays in its
segment until a background merge rewrites that segment without it. So a heavily updated index is
larger than the sum of its live documents, and
_cat/indices shows both counts — docs.count and
docs.deleted.
There is also _delete_by_query, which finds matching documents and deletes them one
at a time under the hood. It is not a fast bulk operation and it is not atomic; on a large index it
is a task you launch and monitor. When you want to remove most of an index, reindexing what
you want to keep into a fresh index is usually faster.
Routing: which shard a document lands on
One more thing decided at write time. By default the shard is chosen by hashing the document id,
which spreads documents evenly and means a GET by id knows exactly which shard to ask.
You can override it:
PUT /stayhub-properties/_doc/8f1e-...?routing=tenant-42
GET /stayhub-properties/_doc/8f1e-...?routing=tenant-42All documents with the same routing value land on the same shard, so a query filtered to that tenant can be told to search one shard instead of all of them. On a large multi-tenant index this is a substantial win.
It comes with a sharp edge worth stating plainly: once a document is written with a routing value, every subsequent get, update and delete must supply the same value. Forget it and the operation goes looking on the wrong shard and reports the document as missing — which looks exactly like data loss and is not. This is the main reason to leave routing alone until you have a measured reason to use it.
Reading documents back
GET /doc-test/_doc/1 # source plus metadata
GET /doc-test/_source/1 # just the source
HEAD /doc-test/_doc/1 # exists? 200 or 404
GET /doc-test/_doc/1?_source_includes=title,cityAnd _mget for several at once, which is one round trip instead of many:
{ "docs": [
{ "_id": "8f1e-..." },
{ "_id": "3a20-..." }
] }That is the operation behind the fetch-by-id pattern from lesson 6 — search for the ids,
then _mget the current records. Worth remembering it exists before writing a loop.
One more parameter worth knowing on the search side rather than the get side:
_source_excludes. A search that returns twenty listings returns twenty full
_source documents, and if one field is a long description the page does not render, you
are paying to serialise and transfer it twenty times per search. Excluding it is a one-line change
with a measurable effect on response size, and it is one of the few optimisations here that costs
nothing at all.
Absent beats flagged
One modelling decision belongs here because it is a decision about writes. When a listing is unpublished, StayHub does not update the document with a status flag — it deletes it:
"""Upsert one property into the index, or remove it if it should no longer be findable.
A DRAFT, SUSPENDED or soft-deleted property is *deleted* from the index rather than indexed
with a status flag. Filtering on `status: PUBLISHED` at query time would work too, but then
every single query pays for the filter and one forgotten `.filter()` leaks a draft into public
search results. Absent is safer than filtered.
"""Both designs work. The difference is what happens when someone writes a new query and forgets the filter — with the flag, a draft listing appears in public search results; with deletion, there is nothing to leak.
This generalises well beyond drafts. Soft-deleted records, suspended accounts, unpublished content: if it must never be found, the safest place for it is outside the index. And because indexing is idempotent and driven from the database, republishing is the ordinary write path — nothing special has to remember to un-delete it.
The one thing that makes it work
The whole design above rests on a single property, and it is worth naming: the same write, applied twice, produces the same index. Document id from the source record, whole-document writes rather than partial ones, delete-if-not-visible rather than a flag.
That is what lets the next lesson retry a failed index minutes later without checking what happened in between, lets a crashed bulk load be restarted from the beginning, and lets a rebuild run against a live index without coordination. None of those are safe if a write depends on what was there before.
What this means for your application
Four rules fall out of the above, and they are the ones StayHub is built on.
Choose your own document ids, from something that already identifies the thing. Every retry, repair and rebuild becomes safe.
Never force a refresh on the request path. Use wait_for if a caller
truly needs to see its own write, and otherwise read from your database.
Do not build read-your-writes into the UI via search. The one-second window is real, it is variable, and it will produce a bug report that cannot be reproduced.
Read the shard counts on writes that matter. A success with
"failed": 1 is a success with a caveat, and nobody will tell you.
Treat every index call as replaceable. If it fails, the next one fixes it; if it runs twice, nothing breaks. That property is what the next lesson relies on entirely, when the question becomes how to index tens of thousands of documents and how to keep them in step with a database that is still changing.