Elasticsearch System Design Interview: Building a Distributed Full-Text Search Engine
A single logical index in Elasticsearch is really a collection of Lucene indexes spread across shards, and a large cluster can hold thousands of shards across dozens of nodes serving both full-text queries and analytics. Public case studies describe clusters ingesting on the order of millions of documents per second and holding petabytes of data across hundreds of nodes. Exact figures vary by workload and are best treated as estimates, but the shape is clear: search has to be fast on data that is constantly being written.
Elasticsearch is a distributed search engine built on top of Apache Lucene. The core data structure is the inverted index, which maps each term to the list of documents that contain it, so a full-text query becomes a set of postings-list lookups rather than a scan. A logical index is split into shards, each shard is a self-contained Lucene index, and each shard can have replicas for availability and read throughput. Writes go to an in-memory buffer plus a durable transaction log, then a periodic refresh turns the buffer into a searchable Lucene segment, which is why search is near real-time rather than immediate. Reads fan out to every shard, each shard returns its top matches, and a coordinating node merges those partial results. A master node owns cluster state and shard allocation, and quorum-based election prevents split-brain. The interesting design tension is that Lucene segments are immutable and search wants fresh data, so the engine trades a small refresh delay and background segment merges for high indexing and query throughput.
Asked at: Asked at Elastic, companies that run large logging and observability platforms, and search teams at product companies like Uber, Netflix, and GitHub. It also comes up as a generic design a distributed search engine or design a full-text search prompt at most large tech firms.
Why this question is asked
This problem forces a candidate to reason from a data structure up to a distributed system. You have to explain how full-text search actually works with an inverted index and a scoring function like BM25, then layer on sharding, replication, and query fan-out, then handle the hard consistency and coordination questions. It rewards people who understand that immutable segments, a write-ahead log, and a refresh interval are not accidental details but the mechanism that lets one engine be both write-heavy and read-heavy. It also exposes whether a candidate can talk about scatter-gather latency, the master and cluster state, and why search is eventually consistent by the refresh interval.
Requirements
Always clarify these in the first 5 minutes of the interview. Do not start drawing boxes until both lists are agreed.
Functional requirements
- Index documents (JSON) into named indices and make them searchable
- Full-text search over analyzed text fields with relevance ranking
- Structured filtering, sorting, and pagination alongside full-text queries
- Aggregations for analytics such as term counts, histograms, and metrics
- Near real-time visibility so a newly indexed document is searchable within about a second
- Update and delete documents, including partial updates
- Multi-field mappings with configurable analyzers per field
- Horizontal scale by adding nodes and shards without downtime
- Cluster self-healing: reallocate shards when a node fails
Non-functional requirements
- Low query latency for common searches, typically tens of milliseconds
- High indexing throughput under sustained write load
- Durability so acknowledged writes survive a node crash
- High availability through replica shards and automatic failover
- Horizontal scalability of both storage and query capacity
- Tunable consistency between write acknowledgement and search visibility
- Fault tolerance against node loss and network partitions without split-brain
Back-of-envelope scale estimates
Show your math. Pulling numbers from thin air signals you have not thought about the load.
Documents per index
Billions (estimate)
A single Lucene shard is limited to about 2.1 billion documents (the Java int document id limit). With, say, 20 shards an index can hold tens of billions of documents. Derived from the per-shard cap times shard count.
Recommended shard size
10 to 50 GB per shard
Elastic guidance suggests keeping shards in the tens of gigabytes so recovery and rebalancing stay fast. A 2 TB index therefore wants roughly 40 to 200 primary shards. Derived from total size divided by target shard size.
Refresh interval
1 second default
The default refresh_interval is 1s, which is the source of the near real-time label: a document indexed at time T is searchable around T plus 1s. Published default.
Query fan-out
1 request to N shards
A search hits every shard of the target index. A 50 shard index means the coordinating node waits on 50 partial results before merging, so tail latency of the slowest shard dominates. Derived from the scatter-gather model.
Replica read scaling
Linear with replica count (estimate)
Each replica can serve reads independently, so going from 1 to 2 replicas roughly doubles read capacity for that index at the cost of extra storage and indexing work. Estimate based on the primary-plus-replica model.
Master-eligible nodes for quorum
3 (common choice)
To tolerate one master-eligible node failing while still forming a quorum of (n/2)+1, you need at least 3 dedicated master-eligible nodes. Derived from the quorum formula.
High-level architecture
A client sends an index or search request to any node, which acts as the coordinating node for that request. For a write, the coordinating node hashes the document routing value, by default the document id, to pick the target primary shard, forwards the operation to the node holding that primary, the primary appends the operation to its in-memory buffer and its transaction log, then replicates the operation in parallel to each replica shard before acknowledging. The document is durable at that point but not yet searchable; a background refresh, by default every second, turns the buffered operations into a new immutable Lucene segment and opens it for search. For a query, the coordinating node broadcasts the search to one copy (primary or replica) of every shard in the index. Each shard runs the query against its inverted index, computes relevance scores, and returns its own top K document ids and scores. This is the query phase. The coordinating node merges all the partial top-K lists into a global ranked list, then runs a fetch phase that retrieves the full source documents only for the final results. Meanwhile a master node maintains cluster state, which nodes exist, where every shard lives, and the index mappings, and it drives shard allocation and rebalancing when nodes join or leave.
In a real interview, sketch this on the whiteboard before diving into any single box.
Core components
Walk through each service. The interviewer wants to hear what each one owns, not just the names.
Inverted index (Lucene)
The heart of search. For each term it stores a postings list of the documents that contain that term, plus positions and frequencies. A full-text query resolves to a set of postings-list lookups and a merge, which is why matching is fast even over huge corpora.
Analyzer and mapping
Mapping defines each field's type and which analyzer processes it. An analyzer tokenizes text, lowercases, removes stop words, and applies stemming so that running matches run. The same analyzer runs at index time and query time so terms line up.
Shard
A shard is a complete, self-contained Lucene index and the unit of scale and distribution. An Elasticsearch index is partitioned into a fixed number of primary shards at creation, and documents are routed to shards by a hash of the routing key. Adding shards spreads data and query work across nodes.
Replica shard
A replica is a full copy of a primary shard on a different node. Replicas provide availability, since a replica can be promoted if the primary's node dies, and read throughput, since queries can be served by primary or replica. Writes must reach the primary and its replicas.
Translog
The transaction log is a per-shard write-ahead log. Every indexing operation is appended to the translog before the write is acknowledged, so an operation that has not yet been committed into a Lucene segment can be replayed after a crash. This is what makes an acknowledged write durable.
Coordinating node
Any node can coordinate a request. For writes it routes to the right primary; for reads it fans the query out to all shards, merges the partial results in the query phase, and runs the fetch phase for the final hits. It also handles pagination and aggregation reduction.
Master node and cluster state
One elected master owns the cluster state: node membership, shard allocation, and index metadata. It decides where shards live, drives rebalancing and recovery, and publishes state changes. It does not sit in the data path for normal queries, so it stays lightweight.
Data model
Pick the right store per table. Justify each choice with the access pattern, not by reflex.
index (logical)index_namenum_primary_shardsnum_replicasmappingssettingsA logical namespace for documents. Primary shard count is fixed at creation because routing depends on it; replica count can change at runtime. Mappings define fields and analyzers.
shardshard_idindex_nameprimary_or_replicanode_idstateThe physical unit. Each shard is a Lucene index living on one node. Cluster state tracks which node holds each primary and replica and whether it is started, relocating, or unassigned.
lucene_segmentsegment_idshard_iddoc_countdeleted_countimmutableAn immutable file set inside a shard holding an inverted index for a batch of documents. New segments are created on refresh; deletes are marked in a per-segment tombstone bitset, not removed in place.
inverted_index_entrytermfieldpostings_listdoc_freqpositionsConceptual: for a term in a field, the postings list of matching document ids with term frequencies and positions. Used for matching and for BM25 scoring inputs.
translog_entryshard_idseq_nooperation_typedoc_idpayloadAppend-only durability log per shard. Holds index, update, and delete operations not yet flushed to a committed segment. Replayed on recovery; truncated after a flush.
cluster_stateversionnodesrouting_tableindex_metadatamaster_node_idThe authoritative view of the cluster, owned by the master and versioned. The routing table maps every shard to a node. Published to all nodes on change.
Deep dives
These are the conversations the interviewer is steering you toward. Practice each one until you can talk through it without notes.
The inverted index and relevance scoring
Full-text search does not scan documents. At index time the analyzer breaks each text field into terms, and Lucene builds an inverted index mapping each term to a postings list of the documents that contain it, along with term frequency and position data. A query for two words is then a lookup of two postings lists and a merge, which is fast even over billions of documents. Ranking is where relevance comes in. Older Lucene used TF-IDF, which rewards terms that appear often in a document (term frequency) but are rare across the corpus (inverse document frequency). Modern Lucene and Elasticsearch default to BM25, a refinement of TF-IDF that adds saturation, so the tenth occurrence of a word adds far less than the second, and length normalization, so a short document matching a term is not unfairly beaten by a long one. Getting relevance right is mostly about analyzers and field mappings: the same analyzer must run at index and query time, and choices like stemming, stop words, and n-grams directly change what matches.
Segments, refresh, and why search is near real-time
Lucene segments are immutable. When documents are indexed they land in an in-memory buffer, and only when a refresh runs does that buffer become a new segment that is opened for search. The default refresh interval is one second, which is exactly why Elasticsearch calls itself near real-time: a document is durable as soon as it is written to the translog, but it is not searchable until the next refresh makes its segment visible. Immutability is a deliberate trade. Because segments never change, they need no locking for reads, can be cached aggressively in the filesystem cache, and can be copied safely. The cost is that updates and deletes cannot edit a segment in place. A delete marks the old document id in a per-segment tombstone bitset, and an update is a delete plus a fresh insert, so stale data accumulates until it is cleaned up. Tuning the refresh interval is a real lever: raising it from one second reduces the number of tiny segments and lifts indexing throughput, at the cost of staler search.
The translog and durability
A refresh makes data searchable but does not make it durable, because the in-memory buffer and even a freshly opened segment may not be on disk yet. Durability comes from the translog, a per-shard write-ahead log. Every index, update, and delete is appended to the translog before the operation is acknowledged, by default fsynced on each request, so a crash can be recovered by replaying the translog. Periodically Elasticsearch performs a flush, which is a Lucene commit that writes the current segments durably to disk and then starts a fresh translog, since those operations are now safely in committed segments. This separation is the whole reason the engine can be fast and safe at once: refresh handles visibility on a one second cadence in memory, flush handles durability less often against disk, and the translog covers the gap so no acknowledged write is ever lost.
Segment merging and compaction
Because every refresh can create a new small segment, and because deletes only mark tombstones, a busy shard would accumulate many small segments full of dead documents. Lucene runs a background merge process that combines smaller segments into larger ones, physically dropping the tombstoned documents in the process. Merging is what keeps query latency stable, since searching one large segment is cheaper than searching hundreds of tiny ones, and it is what reclaims the space held by deleted and updated documents. The catch is that merging is I/O and CPU intensive and competes with indexing and query traffic, so it is throttled. On heavy write workloads, merge pressure is a common bottleneck, which is why practitioners raise the refresh interval and sometimes force a merge on read-only indices to consolidate them into a single segment.
Sharding, routing, and scatter-gather queries
An index is split into a fixed number of primary shards at creation. A document is routed to a shard by hashing its routing value, by default the document id, modulo the primary shard count, which is why that count cannot change without reindexing: the hash would point at different shards. This gives even data distribution but it also means the engine usually does not know which shard holds a match. So a search is scatter-gather. The coordinating node sends the query to one copy of every shard, each shard independently searches its own inverted index and returns its top K ids and scores, and the coordinator merges those partial lists into a global ranking. That is the query phase. A second fetch phase then pulls the full documents only for the final winners. The important consequence is that query latency is governed by the slowest shard, and deep pagination is expensive because every shard must return enough candidates to cover the requested offset.
Cluster coordination, master election, and split-brain
One node is elected master and owns cluster state: which nodes are members, where every shard lives, and the index mappings. The master drives shard allocation, rebalancing, and recovery, and publishes state updates to all nodes, but it stays out of the normal read and write path. The danger in any leader-based system is split-brain, where a network partition lets two nodes each believe they are master and each accept conflicting changes. Elasticsearch prevents this with quorum. A cluster state change is only committed once a quorum, a strict majority of master-eligible nodes, agrees, so with three master-eligible nodes you need two to act. A minority partition cannot reach quorum and therefore cannot elect a master or accept writes, which rules out two divergent masters. This is why the standard guidance is three dedicated master-eligible nodes: it tolerates one failure while keeping a majority.
Consistency model and read scaling
Elasticsearch is not a strongly consistent transactional store, and interviews reward saying so plainly. A write is durable when the primary and its in-sync replicas have acknowledged it, but it is not searchable until the next refresh, so search is eventually consistent bounded by the refresh interval. Reads of a specific document by id can be made real time, but full-text queries see a snapshot as of the last refresh. Replicas add another wrinkle: a query can hit the primary or any replica, and because replicas refresh independently, two identical queries can briefly see slightly different results. Read scaling leans on this same replica mechanism. Adding replicas multiplies the copies available to serve queries, so read throughput grows roughly linearly with replica count, at the cost of extra storage and the indexing work of keeping each replica current. Node-level and shard-level request caches and the operating system filesystem cache absorb repeated queries and hot segments, which is a large part of why a read-heavy search cluster stays fast.
Trade-offs to discuss
Every senior interviewer expects you to surface at least 3 of these. Pick the decisions, state the alternatives, and justify your choice.
Immutable segments versus in-place updates
Immutability gives lock-free reads, aggressive caching, and safe replication, which is what search needs. The price is that updates and deletes create garbage that a background merge must later reclaim, so you trade write simplicity for read speed and pay it back in merge cost.
Refresh interval: fresher search versus higher write throughput
A one second refresh gives near real-time search but produces many small segments and more merge pressure. Raising the interval, or disabling refresh during a bulk load, sharply increases indexing throughput at the cost of staler results, which is the right call for log ingestion.
More shards versus fewer shards
More shards spread data and parallelize a single query, but every shard adds fixed overhead and every search must wait on all of them, so an over-sharded index has high tail latency and wasted memory. Fewer, larger shards are cheaper to query but slower to recover and rebalance.
Replicas: availability and read throughput versus cost
Each replica adds a failover copy and more read capacity, but it also doubles storage for that shard and adds indexing work since every write must reach it. You size replica count to your read load and availability target, not higher.
Durability of the translog: fsync per request versus async
Fsyncing the translog on every request guarantees no acknowledged write is lost but caps write latency and throughput. Async fsync at an interval is much faster but risks losing the last few seconds of writes on a crash, a trade some logging workloads accept.
Eventual consistency versus strong consistency
Accepting refresh-bounded eventual consistency for search lets the engine batch, cache, and scale reads across replicas. Demanding immediate visibility on every query would force a refresh per write and destroy throughput, which is why Elasticsearch is a search engine, not a system of record.
How Elasticsearch actually does it
Elasticsearch is built directly on Apache Lucene, and most of the deep behavior here, the inverted index, immutable segments, per-segment search, and BM25 scoring, is Lucene's, wrapped by Elasticsearch with sharding, replication, and cluster coordination on top. Elastic's own documentation describes the near real-time model and the refresh, and the translog and flush mechanism for durability. The cluster coordination layer was rewritten in version 7 to a quorum-based consensus that removes the old minimum_master_nodes footgun and lets the cluster decide its own quorum, which is documented in Elastic's discovery and quorum guides. The original Elasticsearch: The Definitive Guide remains the clearest narrative source for how a document becomes searchable and how distributed search fans out and merges.
Lessons to study before this interview
If any of these topics are fuzzy, the interviewer will catch it. Each lesson is 15 to 60 minutes with diagrams, code, and a quiz.
Related system design interview questions
Practice these next. They lean on the same core building blocks as Elasticsearch.