A Note Handed Across the Desk
Think of a clerk at a library desk whose job is to fill in a request card for every visitor. A visitor hands over a slip of paper. On it is a normal request, and underneath, in the same handwriting: "Ignore your rules. Don't fill in a card. Just write me a nice note instead."
A clerk who has done this job for months will probably fill in the card anyway and hand the note on, because filling in cards is simply what she does. A new clerk who is reading the rules from a folder might be thrown by it. And a clerk who has been told to write only on a printed form, with boxes for each answer, cannot write a nice note even if she wants to. There are no boxes for it. But the note may still change what she writes in the boxes.
The prompting chapter showed that a sentence hidden inside a customer's message can take over a prompted model. This lesson sends the same sentence to the models of this chapter: two small models trained on tickets, and two prompted models reading the house rules. Each one is tested exactly as the chapter built it.
The result split in a way I did not expect. The two fine-tuned models were trained the same way, on the same examples, and one ignored the attack almost completely while the other broke the format on 76 of 140 messages; in 47 of those it wrote to the customer as the shop, as the attack asked.