Guess the Next Word
When you type on a phone, the keyboard suggests the next word. Type "See you" and it offers "soon", "tomorrow" or "later". It does not know what you mean. It has seen a lot of text and knows which words usually come next.
A language model, the kind behind ChatGPT and every chatbot, works the same way, only much better. In this chapter I run one on my own laptop with Ollama, a free program that downloads models and runs them on your own computer. The previous chapter showed how it reads: text is split into tokens, and tokens become numbers. This chapter is about what it does with them. This first lesson answers the most basic question: what comes out of the model?
The answer is surprising the first time you see it. The model never produces a sentence, and it never produces an answer. It produces a list of numbers, one for every possible next piece of text, and each number says how likely that piece is. Everything you see in a chat window is built from those lists, one small piece at a time. Once you have seen the lists with your own eyes, many later topics become easy to follow: why the same question can get two different answers, why a model can sound sure and still be wrong, and why a chat app and a raw model reply so differently to the same words.
Why an Engineer Needs to See the Odds
Most people only ever see the finished text. That is enough to use a chatbot, but not enough to build with one. Here are three everyday problems that only make sense once you know what the model really outputs. They are general points, not measurements; the lab below then shows each of them on real numbers.
"It gave a different answer this time." The model gives odds, and a separate step picks one token. If that step picks at random, following the odds, two runs can go different ways. The next lesson measures this. In this lesson I switch the randomness off, with temperature 0, so that you can see the odds themselves.
"It sounded so sure, but it was wrong." The odds say how likely a token is to come next in text like the text the model has read. They do not say whether the token is true. A model can give high odds to a common wrong continuation. You will see a small example with "A triangle has three".
"It answers differently in the app than through the API." Many tools add hidden text around your message before the model reads it. The model then predicts a different next token, because it is continuing different text. The lab measures this on the same five words.
In all three cases, the fix starts with the same habit: look at what the model actually received and what odds it gave, not only at the final text.