Reflection and Decomposition: Making Agents Check Their Own Work

The most common way teams add reliability to an agent is to bolt a second call onto the end: after the model answers, ask it to review its own work and fix any mistakes. It feels obviously good. It doubles the token bill and, on the failures that matter, it usually changes nothing.

The reason is not a prompt you got wrong. It is that a model's blind spots when it writes are the same blind spots when it reviews. If it did not know the retry policy exists, it will not notice the retry policy is missing. Huang et al. at Google DeepMind studied this directly and titled the paper after the result: Large Language Models Cannot Self-Correct Reasoning Yet. Their finding is that intrinsic self-correction, with no external feedback, often fails to improve reasoning and sometimes makes the answer worse after the "correction."

That does not mean reflection is useless. It means reflection has a precondition, and the precondition is an external signal. The same year, Reflexion (Shinn et al.) reported 91 percent on the HumanEval coding benchmark, above GPT-4's reported 80 percent, using a reflection loop. The thing that made it work was not the model grading itself. It was unit tests. The loop reflected on a real failing test, not on its own opinion of the code.

So this lesson is about two ways to make an agent check its work that actually hold up. Reflection, done against a real check rather than a self-opinion. And decomposition, which splits a task into parts you can verify one at a time, and which improves the compounding math from the agent-loop lesson in a way no prompt can. Both cost tokens. Both can be theater. The point is to know which is which.

The planning lesson gave the agent a shape: ReAct, plan-then-execute, or a fixed pipeline. That shape decides the order of steps. It does not, on its own, make any step more likely to be right. The two techniques here are the improvement loops you wrap around whatever plan you chose.

Reflection acts on the quality of a single output. It adds a critique after a step, and if the critique says the output is wrong, the agent tries again with the critique in hand. It does not shorten the task; it raises the odds that one step lands. Its entire value rides on the critique being right, which is why the rest of the lesson is fussy about where the critique comes from.

Decomposition acts on the length of the chain. The agent-loop lesson showed that a task of N steps, each succeeding with probability p, succeeds end to end at p to the N, and that this collapses fast: twenty steps at 0.95 each is about 36 percent. Decomposition splits that one long chain into a few short ones, each with a check at its boundary, so a failure is caught and retried before it poisons everything downstream.

They are complementary, not rival. Decomposition gives you short segments. Reflection is the check on each segment. The verified boundary between two segments is exactly where the two techniques meet, and most of this lesson lives at that boundary.