A computer cannot compare meanings. It can only compare numbers. So how does a search engine know that "reset my password" and "I forgot my password" ask the same thing, when they share only one word?

The answer is an embedding.

Think of a huge map. Every sentence gets a pin. Sentences that mean the same thing get pins close together. Sentences about different things get pins far apart. To find what someone means, you look for the pins nearest to theirs.

An embedding is the position of that pin, written as numbers. In this lesson I measure two real embedding models on my laptop, and test the thing that matters most for a global audience: do they understand other languages?

If a word below is new, read its line.

Embedding. A list of numbers that stands for the meaning of a piece of text. Also called a vector, which is why the code below names its lists vectors.

Embedding model. A program that turns text into an embedding. It is a kind of language model, trained to put similar meanings close together.

Dimensions. How many numbers are in one embedding. Each model has a fixed number.

Cosine similarity. A score for how close two embeddings point. Near 1 means very close in meaning. Lower means further apart.

Multilingual. Trained on many languages, so the same meaning in two languages lands in the same place.

Ollama. A free program that runs AI models on your own computer. I used it for every measurement here.

Here is a real one. I gave nomic-embed-text, a popular free embedding model, one sentence: "The server is slow today because many users are online."

It returned 768 numbers. The first eight are shown above. No single number means anything on its own. The meaning is in the whole list: where it points.

A different model gives a different list. bge-m3, the second model in this lesson, returns 1,024 numbers for the same sentence. Each model draws its own map, so numbers from two models do not match up. Never compare them.

To compare two embeddings, we measure the angle between them. The score is called cosine similarity.

Imagine each embedding as an arrow from the centre of the map. If two arrows point the same way, the score is 1. The further apart they point, the lower the score.

The maths is simple. Multiply the two lists number by number and add it all up. Then divide by the length of each arrow. An arrow's length is the square root of the sum of each of its numbers squared. The code in this lesson does exactly that in two lines.

Before running anything, I wrote nine sentence pairs in three groups:

• Same meaning: "The server is slow today." and "Today the server is responding slowly."
• Same topic: "The server is slow today." and "We added more memory to the database server."
• Unrelated: "The server is slow today." and "My grandmother makes excellent bread."

Here is what nomic-embed-text scored.

Same meaning scored 0.89 to 0.96. Same topic scored 0.71 to 0.76. Unrelated scored 0.52 to 0.60.

The three groups do not overlap. That is what makes search work: the right answer scores higher than a sentence that only shares a topic.

But notice that unrelated sentences did not score near 0. They scored above 0.5. Scores from one model are only meaningful compared with each other, not as a fixed scale.

Single words get embeddings too. I compared "database" with five other words.

With nomic-embed-text, "storage" and "table" came closest, then "server". "Banana" and "happiness" came last. The model has learned which words live in the same part of the map.

Readers of this course write in many languages. So I took one English sentence, "The server is slow today because many users are online.", and my own translations of it into French, Spanish, German, Japanese, Hindi and Bengali. The meaning is the same in all seven.

A good embedding should put all seven pins in the same place.

With nomic-embed-text, the translations scored 0.47 to 0.70 against the English sentence. An unrelated English sentence about bread scored 0.58. French, Spanish, Japanese, Hindi and Bengali all scored below the sentence about bread. Only German, at 0.70, came near the same-topic pairs.

In other words, this model does not see that a Hindi sentence means the same as the English one.

Then I ran exactly the same test with bge-m3, a model trained on many languages. Its own page says it supports more than 100. Ignore the other words on that page; only the size and the language line matter here.

Here is its real run.

Every translation scored 0.92 to 0.98. The unrelated English sentence scored 0.43. Now the same meaning lands in the same place, whatever the language.

bge-m3 also kept the English results in order: same meaning 0.93 to 0.96, same topic 0.63 to 0.70, unrelated 0.43 to 0.48.

The cost: bge-m3 is a 1.2 GB download against 274 MB for nomic-embed-text, and each embedding holds 1,024 numbers instead of 768, so it takes more space to store.

When embeddings go wrong, you see no error message. The search just returns worse results. These three mistakes cause most of it.

1. Mixing two models. A document is any piece of text you want to search, and your store is where you keep their embeddings. Every document must be embedded by the same model as the question. I tried mixing them on purpose: nomic's 768 numbers against bge-m3's 1,024, for the same sentence. The code stopped with no error. Python's zip silently dropped the extra 256 numbers, and the score came out as -0.007, which means nothing. Check that both lists have the same length. If you change models, embed every document again.

2. Forgetting a model's instructions. Some models expect a short label in front of the text. nomic-embed-text expects "search_query: " before a question and "search_document: " before a document. (When comparing two sentences with each other, as this lesson does, the same label goes on both.) I embedded "How do I reset my password?" with and without its label. The two embeddings scored only 0.96 against each other, not 1.0. So the label changes the embedding, and leaving it out changes your results. Read the model's page before you use it.

3. Trusting the English score for other languages. This lesson's main finding. A model that ranks English sentences well can still fail completely on Hindi or Japanese. Test the languages your users actually write in.

Here is a short script that runs both models on French, Hindi and an unrelated sentence. This is the real file, open in my VS Code.

And this is what it printed. The numbers match the lab exactly, because the script uses the same label for nomic.

To run it yourself:

1. Install Ollama from ollama.com and start it.
2. In a terminal, run ollama pull nomic-embed-text and ollama pull bge-m3.
3. Save the file below as compare_languages.py, as UTF-8 text (the default in VS Code).
4. Run python3 compare_languages.py (on Windows: python compare_languages.py).

It needs no extra Python library. I typed python because of my own setup, shown as "(venv)" in the screenshot; python3 does the same. If you see "Connection refused", Ollama is not running: start it first. The first run is slower while each model loads.

# Does an embedding model understand other languages? Test it yourself.
# Needs Ollama (ollama.com) running, and:  ollama pull nomic-embed-text  and  ollama pull bge-m3
import json, math, urllib.request

# nomic-embed-text expects a short label in front of each text; bge-m3 does not
LABEL = {"nomic-embed-text": "search_query: ", "bge-m3": ""}

def embed(model, texts):
    texts = [LABEL[model] + t for t in texts]
    body = json.dumps({"model": model, "input": texts}).encode()
    req = urllib.request.Request("http://localhost:11434/api/embed", data=body,
                                 headers={"Content-Type": "application/json"})
    return json.loads(urllib.request.urlopen(req).read())["embeddings"]

def cosine(a, b):
    dot = sum(x * y for x, y in zip(a, b))
    return dot / math.sqrt(sum(x * x for x in a) * sum(y * y for y in b))

english = "The server is slow today because many users are online."
others = {
    "French": "Le serveur est lent aujourd'hui parce que beaucoup d'utilisateurs sont en ligne.",
    "Hindi": "आज सर्वर धीमा है क्योंकि बहुत से उपयोगकर्ता ऑनलाइन हैं।",
    "unrelated": "My grandmother makes excellent bread.",
}

for model in ["nomic-embed-text", "bge-m3"]:
    vectors = embed(model, [english] + list(others.values()))
    print(model)
    for name, v in zip(others, vectors[1:]):
        print(f"  {name:<10} {cosine(vectors[0], v):.3f}")

This box runs in your browser. It holds three tiny made-up embeddings of 3 numbers each, instead of real ones with 768, so you can see the sum work. Press Run.

Then change the numbers in pizza to be closer to cat, and run it again.

• Your users write in more than one language. Test your embedding model on their real text, the way this lesson does. An English-only model can silently fail.
• Questions and documents are in different languages. A user asks in Spanish, your documents are in English. A multilingual model can match them directly. The other route is to translate the question first.
• Everything is English. A smaller English model may be enough, and cheaper to store.

Measured: two models, nine English sentence pairs, one sentence in seven languages, and one word against five others.

Not measured: search quality on real documents, other models, or other sentences. The translations are mine, one sentence per language, so treat them as an example, not a benchmark.

1. Run the script on your own sentences.
2. Compare scores within one model only. A 0.6 from one model is not a 0.6 from another.
3. Test your users' languages before you choose a model.
4. Pick a multilingual model if more than one language is involved.