Maths for ML interview question · Linear algebra: vectors and similarity · junior level
Compute the cosine similarity of a = (3, 4) and b = (4, 3) by hand. What does the result mean?
Free to read in full. This is 1 of the 120 questions in the Maths for ML and Deep Learning interview pack, with 147 diagrams. Get the pack, ₹999 · $20
Why it is asked
Cosine similarity is the default score in semantic search (search by meaning, not exact words) and RAG (retrieval-augmented generation, where an LLM first looks up relevant text), and computing it by hand shows you know what it ignores.
The answer
The cosine similarity is 0.96, which means the two vectors point in almost the same direction: the angle between them is about 16.26°.
Cosine similarity is the cosine of the angle between two vectors. It keeps direction and ignores length:
cos(a, b) = (a·b) / (‖a‖ ‖b‖)
where a·b is the dot product (multiply position by position and add) and ‖a‖ is the length of a, the square root of the sum of its squared entries.
Step by step for a = (3, 4), b = (4, 3): 1. Dot product: 3·4 + 4·3 = 24. 2. Lengths: ‖a‖ = √(9 + 16) = 5 and ‖b‖ = √(16 + 9) = 5. 3. Cosine: 24 / (5 · 5) = 24 / 25 = 0.96. 4. Angle: arccos(0.96) ≈ 16.26° (2 decimals).
How to read the value. It always lies between -1 and 1:
- 1: same direction.
- 0: at a right angle, no shared direction.
- -1: opposite directions.
Length really is ignored. For p = (1, 2, 3) and q = (2, 4, 6), q is just 2 × p, and cos(p, q) = 28 / (√14 · √56) = 28 / 28 = 1, a perfect match even though q is twice as long.
This is why cosine is popular for text embeddings: we usually care about what a text is about (the direction), not about the overall size of the numbers (the length).
What the interviewer listens for
- cos = a·b / (‖a‖‖b‖), range -1 to 1.
- Worked: 24 / 25 = 0.96, angle ≈ 16.26°.
- Scaling a vector does not change its cosine with anything.
- 1 same, 0 unrelated, -1 opposite.
Follow-ups the interviewer may ask
- What is cosine distance? Hint: 1 - cosine similarity, so 0.04 here.
- Can cosine similarity be negative for embeddings? Hint: yes in principle, though many text models rarely produce negative scores for real texts.
- How do you compute it fast for millions of vectors? Hint: normalize once, then it is a dot product, a single matrix multiply.
Common mistakes
- Reporting the dot product (24) as the similarity.
- Thinking 0.96 means '96% of the meaning is shared'; it is the cosine of an angle, not a percentage.
Written by Roni Das, who builds distributed systems and AI systems for a living.
Learn it properly
This is taught step by step in the course lesson Cosine, Dot Product or Distance: Which One Compares Embeddings?.
One question is not enough for the interview.
This is 1 of the 120 questions in the Maths for ML and Deep Learning pack
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (147 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: PayPal. Download at once from My packs.
More free questions from this pack
- A batch of 32 MNIST images goes into a dense layer with 128 units. What are the shapes, and what is the rule for when two matrices can be multiplied? (mid)
- A disease affects 1% of people. A test catches 90% of sick people (sensitivity) and wrongly flags 9% of healthy people (false positive rate). You test positive. What is the chance you are sick? (mid)
- Why do we not use mean squared error for classification with a sigmoid output? Compare the gradients numerically. (staff)
- Compute scaled dot-product attention by hand for two tokens with d_k = 2. (senior)
Free questions from the other packs
- What do temperature and top-p control, and what values would you use for data extraction versus brainstorming? (AI Engineering)
- What is hybrid search, and how do you merge keyword and vector results with Reciprocal Rank Fusion? (AI Engineering)
- Users ask "list all X" questions and RAG returns only five items. Why, and what do you do? (AI Engineering)
- How do you check that an LLM judge agrees with human reviewers? (AI Engineering)
- Cut the cost of an LLM feature serving 10 million requests a day by 60% without hurting quality. Walk through your plan. (AI Engineering)
- Why are LLMs bad at counting letters or reversing a word? (GenAI and LLM)
- Write out scaled dot-product attention with the shape of every tensor. (GenAI and LLM)
- How does speculative sampling keep exactly the target model's distribution? (GenAI and LLM)
- What is GRPO, and how do verifiable rewards train reasoning models? (GenAI and LLM)
- Design a system that answers questions from 300-page contracts accurately. (GenAI and LLM)
- What happens, step by step, when an LLM server handles one request? (Inference Engineering)
- Derive the KV cache size formula and work it out for Llama 3 70B. (Inference Engineering)
- What is PagedAttention? (Inference Engineering)
- When does speculative decoding stop helping or make things worse? (Inference Engineering)
- Design a service to serve Llama 3 70B at 10 QPS with latency targets. (Inference Engineering)
- A customer says: "We want AI for our support team." What do you do in the first meeting? (Forward Deployed Engineer)
- The customer's admin offers you a global admin account "to save time". What do you ask for instead, and why? (Forward Deployed Engineer)
- Your connector writes tickets into the customer's system. A request timed out, the connector retried, and now there are duplicate tickets. How do you make the writes safe to retry? (Forward Deployed Engineer)
- The customer wants RAG over 10 million documents, all behind their firewall. Size it for me on the whiteboard. (Forward Deployed Engineer)
- One deployment of your AI platform must serve 50 business units of the same enterprise. How do you design multi-tenancy? (Forward Deployed Engineer)