Which Model Should You Use?
So far this chapter has used one model, qwen2.5:3b, for almost everything. Sooner or later every project asks which model to use instead. There are many free models that run on a laptop, and each one's web page says it is good.
The honest way to choose is to measure them yourself, on the same tests, the same way. That sounds simple. This lesson shows that it is easy to get wrong, even when you are careful, and what to check before you trust a score.
Think of comparing runners. If every runner runs the same distance, on the same track, with the same clock, the times are comparable. But if one runner always stops to check the map before running, and the race ends after ten seconds, that runner will "lose" every race without ever showing how fast they are. Equal rules can still make an unfair race.
This lesson puts four small models through the same tests on my laptop: size, token counts, speed, arithmetic and instruction following. One of them scores 0 out of 40, and the reason is the most important part of the lesson.
Words You Need First
Benchmark. A fixed set of tests, run the same way on each model, so the results can be compared.
Token budget. The most tokens a reply may use: num_predict in Ollama, from lesson 10.
Thinking model. A model trained to reason step by step before it gives an answer. qwen3 is one. Ollama has a think setting that asks for the thinking to be switched off, or kept in a separate field; as this lesson finds, for the qwen3 build used here it could not be switched off.
Tokens per word. Each model has its own tokenizer (the Tokens and chapter), so the same text becomes a different number of tokens in each.
Sign test. From lesson 9: count the questions only model A got right and the questions only model B got right, and ask whether that split is bigger than luck. The result is a number p: the chance of a split at least that lopsided if neither model were really better.