We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningAdvanced · Distributed Systems Core
A database that stores embeddings, lists of numbers that capture the meaning of text or images, and finds the ones closest to a question's embedding. It powers semantic search and RAG.
In short
A vector database stores embeddings, lists of numbers that capture the meaning of a piece of text, an image or a product, and finds the stored items whose numbers are closest to a question's numbers. It answers "what is most like this?" instead of "what matches these exact words?", which is why it sits behind semantic search, recommendations and RAG chatbots.
In short
A vector database stores embeddings, lists of numbers that capture the meaning of a piece of text, an image or a product, and finds the stored items whose numbers are closest to a question's numbers. It answers "what is most like this?" instead of "what matches these exact words?", which is why it sits behind semantic search, recommendations and RAG chatbots.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Vector Database across a run of lessons, not one page. These 3 alone are about 110 minutes of step-by-step reading, 1 of them with code you run in the browser, all with a quiz.
Databases built for AI, storing and searching high-dimensional embeddings that power semantic search, RAG, and recommendation engines.
System design of a production RAG system. Covers chunking, embeddings, vector search, hybrid retrieval, reranking, caching, evaluation and cost.
How to efficiently store, manage, and serve high-dimensional embedding vectors at scale.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Vector Databasepay once,
yours for life
A vector database is a database built to store vectors and to find, quickly, the stored vectors that are nearest to a new one. In AI apps those vectors are almost always embeddings.
A few words first. A vector is just a list of numbers, such as [0.12, -0.03, 0.88, ...]. An embedding is a vector made by an embedding model, a neural network trained so that texts with similar meaning get similar numbers. The model I use below, nomic-embed-text, turns any sentence into 768 numbers. Each position in the list is a dimension, so that is a 768-dimension vector. Nobody can say what one dimension means on its own; only the distances between whole vectors mean something.
Store a vector for every document, turn the user's question into a vector with the same model, and the documents whose vectors are nearest are the ones closest in meaning. Finding them is called nearest neighbour search, and doing that well at scale is the job of a vector database. It usually stores the original text and some labels, called metadata, next to each vector, so it can return readable results and filter them.
A normal search index finds documents that contain your words. Ask "stop one customer from flooding my API with too many calls" and a keyword search looks for flooding, customer and calls. The definition you want says "how many requests a client can make in a given time window": client, not customer; requests, not calls. In my test below, the vector search put that definition, Rate Limiting, first out of 200.
That is the whole point: match on meaning, not spelling. It is also the weakness. A vector search always returns its nearest items, even when none of them is a good answer, and it is weak at exact strings such as an error code or an order number. Real systems often run keyword search next to it and merge the two, which is called hybrid search.
Every vector database, from a library on your laptop to a managed cloud service, does the same four things.
Nearness is measured with a distance. Cosine distance compares the direction of two vectors and ignores their length: 0 means the same direction, larger means less alike. The other common choices are the dot product and Euclidean (L2) distance. When every vector has length 1, which nomic-embed-text's vectors from Ollama do (I checked: 1.0000 for all 200), all three put results in the same order. If you are not sure, use cosine. My cosine, dot or distance lesson shows what changes when vectors are not length 1.
One trap: Chroma's default distance is L2, not cosine. Most text embedding models are meant to be compared with cosine, so set it when you create the collection, as I do below.
Here is a real one. I took 200 definitions from this site's glossary, embedded them with nomic-embed-text in Ollama, stored them in Chroma, an open-source vector database that runs inside a Python program, and asked four questions in plain English. I ran it on a Mac on 2026-10-04 and copied the output as it came out.
nomic-embed-text expects a short task prefix in front of the text: search_document: for what you store and search_query: for questions. Leave it out and the vectors change and the results get worse, without any error.
Install
uv venv --python 3.12 .venv
uv pip install --python .venv/bin/python chromadb numpy
ollama pull nomic-embed-text:v1.5vdb.py: one call to Ollama embeds up to 64 texts
EMBED_MODEL = "nomic-embed-text:v1.5"
def embed(texts: list[str], prefix: str) -> np.ndarray:
"""nomic-embed-text wants a task prefix: 'search_document: ' for stored text, 'search_query: ' for questions."""
out = []
for i in range(0, len(texts), 64):
body = {"model": EMBED_MODEL, "input": [prefix + t for t in texts[i:i + 64]]}
req = urllib.request.Request("http://localhost:11434/api/embed", data=json.dumps(body).encode(),
headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=600) as r:
out.extend(json.loads(r.read())["embeddings"])
return np.asarray(out, dtype=np.float32)vdb.py, excerpt (... is the print loop): create the collection with cosine distance, add 200 vectors, query, then query with a filter
rows = [json.loads(l) for l in open(HERE / "demo.jsonl")]
vecs = embed([r["text"] for r in rows], "search_document: ")
client = chromadb.PersistentClient(path=str(HERE / "chroma-demo"))
col = client.create_collection("glossary", configuration={"hnsw": {"space": "cosine"}})
col.add(ids=[r["id"] for r in rows], documents=[r["text"] for r in rows],
embeddings=vecs.tolist(), metadatas=[{"level": r["level"]} for r in rows])
for q in questions:
res = col.query(query_embeddings=embed([q], "search_query: ").tolist(), n_results=3)
...
res = col.query(query_embeddings=embed([q], "search_query: ").tolist(), n_results=3,
where={"level": "foundation"})Output of .venv/bin/python vdb.py demo (distance, id, start of the text)
200 sentences -> 200 vectors of 768 numbers; vector length (norm) min 1.0000 max 1.0000
collection 'glossary': 200 items, space=cosine
Q: stop one customer from flooding my API with too many calls
0.337 rate-limiting Rate Limiting: Controlling how many requests a client can make in a gi
0.374 thundering-herd Thundering Herd: When many clients simultaneously retry or reconnect a
0.382 long-polling Long Polling: The client sends a request and the server holds it open
Q: split a huge table across many machines
0.379 data-federation Data Federation: Splitting databases by function (users in one DB, ord
0.380 horizontal-scaling Horizontal Scaling: Adding more machines to handle increased load (sca
0.405 sharding Sharding: Splitting a database into smaller pieces (shards) distribute
Q: the chatbot confidently states something that is not true
0.354 hallucination Hallucination: When a language model produces fluent, confident text t
0.404 hinted-handoff Hinted Handoff: When a replica is temporarily down, another node store
0.410 train-serve-skew Train-Serve Skew: When a feature is computed one way in training and a
Q: keep the site up when a whole data centre goes dark
0.393 long-polling Long Polling: The client sends a request and the server holds it open
0.407 hinted-handoff Hinted Handoff: When a replica is temporarily down, another node store
0.412 infrastructure-as-code Infrastructure as Code: Managing servers, networks, and cloud resource
Q: split a huge table across many machines where level == "foundation"
0.380 horizontal-scaling Horizontal Scaling: Adding more machines to handle increased load (sca
0.405 sharding Sharding: Splitting a database into smaller pieces (shards) distribute
0.416 normalization Normalization: Organizing database tables to reduce redundancy by spli
Two questions worked well: Rate Limiting and Hallucination came first. Sharding, the obvious answer to the second question, came third, behind two close relatives, and the gaps are tiny: 0.379, 0.380, 0.405. The fourth question failed. Nothing about disaster recovery made the top 3; Disaster Recovery itself ranked 81st of 200. Its definition talks about restoring systems, RPO and RTO and never mentions a data centre or a site going dark, so its vector sits far from my question. A vector database finds text that is near your question. It cannot find meaning that the stored text does not express. My searching by meaning lesson shows one rewritten description jumping from rank 34 or lower to rank 1.
Look at the distances, too. The farthest of all 200 definitions from the first question was only 0.640 away, against 0.337 for the best. Unrelated text does not score near the maximum, so a fixed cut-off such as "ignore anything above 0.5" is fragile and changes from one model to another.
The last query shows the database part. The where filter kept only definitions labelled foundation, so Data Federation dropped out and Normalization came in. Filtering on metadata, together with adding, updating and deleting items, is what separates a vector database from a bare search index.
The simplest way to find the nearest vectors is to compare the question with every stored vector and keep the best. That is exact search, also called brute force or flat search, and it never misses. Its cost grows with the number of vectors: ten times the data, about ten times the work.
Approximate nearest neighbour search, ANN, builds an index that looks at a small part of the data and usually finds the true nearest items, but not always. Recall measures how often: recall@10 is the share of the true top 10 that the index actually returned. Production systems commonly run at around 95 to 99% recall.
The most widely used ANN index is HNSW, Hierarchical Navigable Small World, published by Malkov and Yashunin in 2016. It links every vector to a few of its near neighbours, forming a graph, and stacks a few thinner copies of that graph on top. A search starts on the sparse top layer, takes long hops toward the question, drops a layer, takes shorter hops, and finishes on the bottom layer, which holds every vector.
To see the trade-off with real numbers, I embedded 26,785 paragraphs from this site's lessons (834 seconds on the laptop, 82 MB of vectors), then searched them with 300 lesson titles as questions. The right answer for each question is the exact top 10 from comparing all 26,785 vectors in NumPy. I then built two HNSW indexes in Chroma, one with the defaults and one deliberately cheap, and measured recall@10 and time per query at five ef_search values.
One trap cost me a run, so it goes first. My first sweep changed ef_search with collection.modify() inside the same program, and recall did not move at all: 0.988 at every value. The new setting is saved, but a client that already has the collection open keeps searching with the old one. Measured one fresh process per value, ef_search does what it should:
Exact search: every vector compared (the vectors have length 1, so the dot product is the cosine)
exact (NumPy, all 26,785 compared): median 3.45 ms per query, recall@10 = 1.000 by definitionHNSW in Chroma with its defaults (M=16, ef_construction=100). Each ef_search value set with modify(), then measured in a fresh process
ef_search=10 recall@10 = 0.872 median 1.04 ms
ef_search=20 recall@10 = 0.933 median 0.78 ms
ef_search=40 recall@10 = 0.966 median 1.83 ms
ef_search=100 recall@10 = 0.988 median 2.78 ms
ef_search=200 recall@10 = 0.995 median 5.06 msThe same, on a deliberately cheap index (M=4, ef_construction=10)
ef_search=10 recall@10 = 0.128 median 0.75 ms
ef_search=20 recall@10 = 0.182 median 0.84 ms
ef_search=40 recall@10 = 0.250 median 1.13 ms
ef_search=100 recall@10 = 0.372 median 1.53 ms
ef_search=200 recall@10 = 0.465 median 1.60 ms
Read it from the left. Raising ef_search bought recall at the price of time, exactly as the theory says: 0.872 at 10, 0.995 at 200. The cheap graph shows that M and ef_construction matter more than ef_search: with only 4 links per vector, no amount of searching found even half of the true top 10. The times below a few milliseconds are noisy, so read them as rough, not as a ranking.
The surprise is the left-hand column. At 26,785 vectors, checking every one took 3.45 ms and never missed, while the default index took 2.78 ms and missed 1.2% of the answers. An index pays off when exact search becomes too slow for you, which at this rate means far more data than this. Exact search's time grows in step with the data, so measure at your own size before adding an index.
An index also costs memory and disk. The default collection took 111 MB on disk for 82 MB of raw vectors, and HNSW needs its graph in memory to be fast. My vector databases and ANN lesson goes further: the other index families (IVF, product quantization), how filters break approximate search, and when to shard. It is the next step after this page.
These three words get mixed up, and the difference decides what you have to build yourself.
A dedicated vector database is one more system to run, secure, back up and keep in sync with your main data. Check these first.
pgvector (Postgres)
An open-source Postgres extension: vector columns, exact search, HNSW (m 16, ef_construction 64 by default) and IVFFlat indexes, filtered with normal SQL.
Chroma
An open-source vector database that runs inside a Python program or as a server. Used for every run on this page. Its default distance is L2.
Pinecone
A managed vector database: you send vectors and queries over an API and it runs the indexes, scaling and backups.
MongoDB Atlas Vector Search
Vector search inside MongoDB, so vectors sit next to the documents they describe, with HNSW for approximate search and an exact option.
Qdrant, Weaviate and Milvus
Open-source vector databases you can run yourself or buy as a cloud service, built for metadata filtering and large collections.
Faiss
Meta's library of vector indexes. Fast and widely used inside other systems, but by its authors' own description not a database.
A database that stores embeddings and returns the ones closest in meaning to a question. Example from this page: 200 glossary definitions stored in Chroma; the question "stop one customer from flooding my API with too many calls" returned Rate Limiting first, at a cosine distance of 0.337, though the two share few words.
Commonly used options include pgvector for Postgres, Pinecone (managed), Qdrant, Weaviate, Milvus and Chroma (open source), plus vector search built into MongoDB Atlas, Elasticsearch, Oracle and SQL Server. The best one is usually the one closest to data you already run; the embedding model matters more than the engine.
SQL is a query language, not a database, but major SQL databases now store and search vectors. Postgres does it with the pgvector extension, Oracle has a VECTOR type in its AI Database releases, and SQL Server 2025 and Azure SQL have a vector data type. So you can often add vector search without a new system.
MongoDB is a document database that can also act as one. Atlas Vector Search indexes vector fields with HNSW for approximate search and also supports exact search, so vectors can live in the same documents as the data they describe.
Yes, but more and more as a feature of databases people already run rather than as a separate product. Any app that searches by meaning, such as RAG chatbots, semantic search and recommendations, still needs nearest neighbour search. What changed is that Postgres, MongoDB, Oracle and SQL Server now offer it built in.
If you already run Postgres, start with pgvector: one system, normal SQL filters and joins, HNSW indexes. Move to a dedicated vector database when your own measurements of recall, latency or cost at your real size say pgvector is not enough. For a few thousand vectors, plain exact search in memory may be all you need.
Faiss is a library: it builds and searches vector indexes inside your program and leaves storage, updates, filtering, backups and access control to you. A vector database provides those database features around an index. Many vector databases use an index library like Faiss or hnswlib inside.
Checked against these on .
Vector Databases and Approximate Nearest Neighbor Search
How embeddings become searchable points in space, why exact nearest-neighbor search breaks at scale, and how ANN indexes like HNSW, IVF, and PQ make retrieval fast enough to serve
Design a RAG System at Scale
System design of a production RAG system. Covers chunking, embeddings, vector search, hybrid retrieval, reranking, caching, evaluation and cost.
Embedding Storage
How to efficiently store, manage, and serve high-dimensional embedding vectors at scale
RAG Caching and Cost: Semantic Cache, Embedding Reuse, and the Bill
RAG triples a chat bill through retrieved context. Cut it with a semantic cache, embedding reuse, and a reranker, without serving confidently wrong answers.
Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
HNSW
Hierarchical Navigable Small World, a graph-based nearest neighbor index that gives high recall at low latency by navigating layers of links, at the cost of more memory than IVF.
ANN (Approximate Nearest Neighbor)
Search that finds vectors close to a query without scanning every item, trading a small chance of missing a true neighbor for a large gain in speed. HNSW and IVF are the common indexes.
Cosine Similarity
A measure of how close two embeddings are, based on the angle between them rather than their length. The standard way to rank how related two pieces of text are in vector search.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Hybrid Retrieval
Running dense vector search and sparse keyword search together and fusing the results, so exact terms like an error code and paraphrased meaning both count.
This page explains the idea. The "Vector Database" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Vector Database lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Vector Database in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (180 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
