Search Engines (Elasticsearch, Solr)

You're running an e-commerce platform. Customers expect to type "red running shoes size 10" and get instant, relevant results. They expect faceted filters on the left, brand, price range, rating, color. They expect typo correction ("runnign shoes" still works). They expect auto-complete suggestions as they type.

PostgreSQL's tsvector can handle basic text search. But faceted filtering across dozens of dimensions? Sub-50ms response times across 50 million products? Typo-tolerant auto-complete? Real-time aggregations for analytics dashboards? You've outgrown what a general-purpose database can do.

Elasticsearch was built for exactly this. It's not a general-purpose database, it's a distributed search and analytics engine built on Apache Lucene, optimized for full-text search, structured search, and real-time aggregations. Wikipedia uses it for site search. Netflix uses it for monitoring. Uber uses it for trip data analysis. GitHub uses it to search across hundreds of millions of repositories.

Solr, the older alternative, is also built on Lucene. Both share the same core engine, but they've diverged significantly in architecture and developer experience.

Elasticsearch organizes data into indices (like database tables), which contain documents (JSON objects). Each index is split into shards distributed across a cluster of nodes.

Indexing a document:

When you index a product:

PUT /products/_doc/1
{
 "name": "Nike Air Zoom Pegasus 40",
 "category": "running shoes",
 "brand": "Nike",
 "price": 129.99,
 "color": "red",
 "sizes": [8, 9, 10, 11, 12],
 "rating": 4.5,
 "description": "Lightweight cushioning for everyday running..."
}

Elasticsearch:

1. Routes the document to a shard (based on document ID hash)
2. Passes text fields through an analysis pipeline (tokenize, lowercase, stem)
3. Builds an inverted index, a map from each token to the documents containing it
4. Stores structured fields (price, rating) in doc values, columnar storage for fast sorting and aggregations
5. Replicates the document to replica shards for fault tolerance

Querying:

A search request hits a coordinating node, which fans out to all relevant shards, each shard searches its local inverted index, results are merged and ranked by relevance, and the top N results are returned.

This scatter-gather architecture is why Elasticsearch scales horizontally, add more shards and more nodes, and search capacity grows linearly.