Thinking Step by Step: When It Helps, and What It Costs

At school, a maths teacher often says: "Show your working." There are two good reasons. When a student writes each step on the page, they are less likely to lose track of a long calculation, because they do not have to hold every number in their head at once. And when the answer is wrong, the teacher can see exactly where it went wrong.

But the teacher does not ask for working on every question. If the question is "Which of these four boxes does this letter belong in?", a paragraph of working before the one-word answer makes the page longer and the marking slower, and it may not make the answer any better.

You can ask a language model to show its working too. This lesson measures when that helps and what it costs. I asked the same questions in two ways, once for the answer alone and once for the working first and then the answer, on two kinds of task: sums and word problems, which need several steps, and lesson 1's sorting task, which needs one word. Then I counted the right answers and the length of every reply.

Step by step. Asking the model to write out its working before it gives the answer. In this lesson the request is one sentence: "Think step by step, then give the final answer on a last line starting with 'Answer:'." You will often see this called chain of thought: the chain is the list of steps the model writes.

Direct. Asking for the answer alone: "Answer with the number only", or, for the sorting task, lesson 1's format rule, "Answer with the category word only, in lower case, and nothing else."

Answer line. The last line of a step-by-step reply, starting with "Answer:". A person can read the working; a program needs to find the one answer inside a long reply, and this line is where it looks.

Tokens written. How many tokens the model wrote in its reply. A token is a piece of a word, as the previous chapter showed. Ollama reports this count for every call as eval_count. More tokens written means a longer wait and, on a paid service, a larger bill.

Cut. A reply that the token limit stopped before the model had finished. Ollama marks it with done_reason set to "length". A cut step-by-step reply may never reach its answer line.