Why Read the Same Rules Again?
Most real apps send the same long opening with every request. A support bot sends its rules. A document assistant sends the same contract every time someone asks about it. A coding helper sends the same files. Only the question at the end changes.
Lesson 5 showed that reading is what makes a person wait: a prompt of about 7,900 tokens kept them waiting over half a minute before the first word. If every request starts with the same 4,000 tokens, that is a lot of reading repeated for nothing.
Think of a librarian. On the first day in a new library, answering "where is the book on ?" means walking the shelves and learning the layout. After that, the layout is remembered, and each new question only needs the question itself to be read. A language model server can do the same with the start of a prompt, and in lesson 5 we saw it do so by accident.
This lesson uses that on purpose. It measures how much reading is saved, finds out exactly which part of a prompt can be reused, and turns the finding into one rule you can apply to any app: put what changes last.
Words You Need First
Prefix. The start of a prompt, up to some point. "Prefix" is the word engineers use; this lesson mostly says "start".
Prompt cache. In lesson 5, the model kept the keys and values of every token it had read, so it could write without reading again. A prompt cache keeps those keys and values after the request is finished, so a later request can use them. Hosted providers call the feature "prompt ".
Cache hit. A new prompt starts the same way as one whose keys and values are still stored, so that part does not need to be read again.
First difference. Line two prompts up token by token. The first difference is the first place where they stop matching. It decides how much can be reused, as this lesson shows.
Time to live. How long a stored start is kept before it is thrown away. Engineers shorten it to .