Loading...
System Design and AI Engineering, by Roni Das. 980 interactive lessons.
Start LearningAI Engineering · Advanced · LLM and GenAI Ops
Writing and structuring the instructions given to a language model to get reliable output. In 2026 the craft has moved from clever wording to schemas, context layout, and evaluation.
In short
Prompt engineering is the work of writing, testing and improving the text you send to an AI language model so that its answers are right and come back in a shape your code can use. It is measured, not guessed: you change one thing in the prompt, run the same test cases again, and keep the change only if the score goes up.
In short
Prompt engineering is the work of writing, testing and improving the text you send to an AI language model so that its answers are right and come back in a shape your code can use. It is measured, not guessed: you change one thing in the prompt, run the same test cases again, and keep the change only if the score goes up.
Learn it in the course
This page is the free part.
₹999 in India$49 everywhere elseonce, for the whole course
The AI Engineering course covers Prompt Engineering across a run of lessons, not one page. These 3 alone are about 157 minutes of step-by-step reading, every one with code you run in the browser, all with a quiz.
A prompt has parts: an instruction, a format rule, context, examples and the input. Measured with qwen2.5:3b on 40 customer messages, adding one part at a time: with the message alone, 0 of 40 answers were usable; with an instruction, 19; with a format rule, 36; with context, 40. Two examples added nothing, because the prompt was already at 40 of 40.
Why prompts are production artifacts, not strings in your app, and how a prompt registry gives you versioning, testing, rollback, lineage, and caching.
Five everyday tasks, each asked three ways on qwen2.5:3b and scored by the same check: vague ('Summarise this'), with a soft word ('Summarise this briefly') and specific ('one sentence of at most 25 words'). Vague met the requirement 0 of 40 times, soft words 0 of 40, specific 38 of 40, and the specific wording also wrote 7.4 times fewer tokens.
all part of
AI Engineering
210 lessons · about 219 hours · 10 free to read
₹999in India, by UPI
$49everywhere else, by PayPal
You own this course
Continue with The Parts of a Prompt: Adding One Piece at a Timepay once,
yours for life
Prompt engineering is the work of writing, testing and improving the instructions you give a language model, such as ChatGPT, Claude, Gemini or a small model running on your own laptop, so that it does the job you want, reliably.
A few words first. A language model is a program trained on a huge amount of text that writes a reply one small piece at a time. Each piece is a token, roughly three quarters of an English word. A prompt is everything the model reads before it starts writing: your instructions, any rules, examples, the user's question and any documents you paste in. The model has no other memory of your task. If it is not in the prompt, the model does not know it.
The word engineering matters. Anyone can type a question into a chat box. Prompt engineering starts when the answer has to be right hundreds or thousands of times, inside a product, with nobody checking each reply. Then you need test cases, a score, and a way to tell whether a change helped or you just got lucky.
On the test I describe below, the same small model, on the same 20 customer messages, got 5 right with one prompt and 15 right with another. Nothing changed except the text of the prompt. That gap is why the job exists.
Researchers see the same thing at a larger scale. Sclar and colleagues changed only the formatting of prompts, such as separators, spacing and capital letters, and measured accuracy differences of up to 76 points on LLaMA-2-13B. Small changes that a person would call meaningless can move a score a lot, in either direction.
So the honest answer to "what is the best prompt?" is: the one that scores best on your own test cases, with your model, today. Advice from a blog post, including this one, is a starting point to test, not a result.
Most good prompts are built from the same five parts. You will not always need all five, and adding one should always be a test, not a habit.
My paid chapter starts from a single bare message and adds these five parts one at a time, measuring the score after each one. If you want to see which part does the work on a real task, that is the next step: The Parts of a Prompt.
Hundreds of named techniques exist. These five cover most of what people use in real products, and they are the ones I tested.
Prompt engineering is a loop with four steps. Write down 20 to 50 real inputs with the answer you expect for each. Run your prompt on all of them and score the replies in code. Change one thing. Run the same inputs again and keep the change only if the score went up.
Score two things, not one. Usable means your code could read the reply at all: one number when you asked for a number, valid JSON when you asked for JSON. Right means the value is correct. A reply can be right and unusable, and in my test one was.
Keep a second set of inputs you never look at while tuning. A prompt you adjusted until it passed 20 cases has partly learned those 20 cases, and its score on new ones is usually lower. My lesson on testing a prompt on new cases measures how much lower.
Here is a real run, not a made-up example. The task: read a customer's message and work out the refund owed under a fixed policy (refund broken items and items sent back, never refund shipping). I wrote 20 messages and worked out each correct answer by hand, then sent every message through five prompts. The model was qwen2.5:3b, a small 3-billion-parameter model, running in Ollama 0.32.14 on an Apple M4 laptop, at temperature 0 so the same prompt always gives the same reply.
Every prompt used the same policy text. Only the technique changed. Each prompt also started with a unique reference line, because Ollama reuses the work it did on an identical opening and that would make later calls look faster than they are. I set the context window (num_ctx, the most tokens the model reads at once) to 4,096 myself, because Ollama cuts an over-long prompt without any error; my longest prompt was 265 tokens.
run.py: the five prompts (the full file, with all 20 cases, is in my lab folder)
POLICY = (
"You work in customer support for an online shop. Work out how much money to refund the customer.\n"
"Policy: refund the price of every item that arrived broken or damaged, and of every item the "
"customer is sending back. Never refund shipping. Items the customer keeps are not refunded."
)
ANSWER_ONLY = "Reply with the refund amount in dollars as a number only, for example: 25"
def build(technique: str, message: str) -> tuple[str, dict | None, int]:
"""Return (prompt body, format schema or None, num_predict) for one technique."""
if technique == "zero-shot":
return f"{POLICY}\n{ANSWER_ONLY}\n\nMessage: {message}", None, 60
if technique == "few-shot":
shots = "\n\n".join(f"Message: {m}\nRefund: {r}" for m, r in EXAMPLES)
return f"{POLICY}\n{ANSWER_ONLY}\n\n{shots}\n\nMessage: {message}\nRefund:", None, 60
if technique == "json-schema":
return f"{POLICY}\nReply in JSON with the refund amount in dollars.\n\nMessage: {message}", REFUND_SCHEMA, 60
if technique == "step-by-step":
return (
f"{POLICY}\nThink step by step: list each item, say whether it is refunded and why, "
"then add up the refunded amounts. On the last line write only: Answer: <number>\n\n"
f"Message: {message}"
), None, 512
if technique == "schema+working":
return (
f"{POLICY}\nReply in JSON. In \"working\", list each item, say whether it is refunded, "
"and add up the refunded amounts. Put the total in dollars in \"refund\".\n\n"
f"Message: {message}"
), WORKING_SCHEMA, 512run.py: the schemas and the call to Ollama's chat API
REFUND_SCHEMA = {
"type": "object",
"properties": {"refund": {"type": "number"}},
"required": ["refund"],
}
WORKING_SCHEMA = {
"type": "object",
"properties": {"working": {"type": "string"}, "refund": {"type": "number"}},
"required": ["working", "refund"],
}
def call(prompt: str, schema: dict | None, num_predict: int) -> dict:
body = {
"model": MODEL,
"messages": [{"role": "user", "content": prompt}],
"stream": False,
"keep_alive": "30m",
"options": {"temperature": 0, "seed": 1, "num_ctx": NUM_CTX, "num_predict": num_predict},
}
if schema is not None:
body["format"] = schema
req = urllib.request.Request(URL, data=json.dumps(body).encode(), headers={"Content-Type": "application/json"})
with urllib.request.urlopen(req, timeout=600) as resp:
return json.loads(resp.read())Output of python3 run.py, 2026-10-04
SUMMARY (20 cases each, qwen2.5:3b, temperature 0, seed 1, num_ctx 4096)
technique usable right med out tok med sec max prompt tok cut off
zero-shot 20/20 5/20 3.0 0.7 159 0
few-shot 20/20 5/20 3.0 1.46 265 0
json-schema 20/20 5/20 11.0 1.17 149 0
step-by-step 19/20 14/20 234.5 11.42 178 0
schema+working 20/20 15/20 99.5 5.14 175 0Case 13, the reply from the schema + working prompt, exactly as it came back
{
"working": "The order contains 7 candles, each priced at $5. The customer received 3 melted candles and is returning 2 more. Based on the refund policy, only the melted candles and the returned items will be refunded. Here's the breakdown:\n- Melted candles: 3 items * $5/item = $15\n- Returned items: 2 items * $5/item = $10\nTotal refund amount: $15 + $10 = $25",
"refund": 25
}
Writing the working first was the change that mattered. Step by step fixed 11 cases that zero-shot got wrong and broke 2 that it got right. Adding a working field before the answer in the JSON schema fixed 11 and broke 1, against the schema without it. A paired sign test, which only counts the cases that flipped, gives p = 0.02 and p = 0.006: unlikely to be luck.
Few-shot did not help here: 5 right, the same as zero-shot, with 2 cases fixed and 2 broken. My examples showed only the final number, so they taught the format, which zero-shot already followed, and not the arithmetic. Examples that show the working might do better; I did not test that, so I will not claim it. On a different task, a category label, examples did help a weaker model a lot, as my examples lesson shows. Which technique helps depends on what the model is getting wrong.
The JSON schema made every reply valid JSON, and still only 5 were right. A schema turns a reply your code cannot read into one it can read. It does not make the content correct. Two of the schema + working replies came back as a refund of 0: once with an empty working string, and once after the model decided that items which arrived fine are not refunded, even though the customer was sending them back. Your code still has to check the answer. My format lesson goes further into what a schema can and cannot promise.
One step-by-step reply was right but unusable: it ended with **Final Answer: 36** when I had asked for a line that starts with Answer:. A person would accept it. Code expecting the exact format would not. That is the case for putting the answer in a schema field rather than in free text.
The working is not free. Step by step wrote a median of 234 tokens per reply, against 3 for zero-shot, and took a median of 11.4 seconds on my laptop against 0.7. The schema with a working field got the same accuracy at 100 tokens and 5.1 seconds. On a paid API you pay for every one of those tokens. My step-by-step lesson prices the extra tokens per extra right answer.
Keep this in proportion. It is 20 cases, one small model, and one task that I wrote, so treat it as a demonstration of the method, not a ranking of techniques. Large reasoning models, which think before they answer on their own, behave differently: OpenAI's guide for them says to avoid asking for step-by-step thinking, because they already do it. Run your own loop on your own model.
In a real product, the prompt is code. Treat it that way and most problems go away.
Anthropic's own guide says it plainly: not every failing test is best solved with a better prompt. Stop rewording and change something else when you see one of these.
OpenAI API
Its prompting guide covers developer and user messages, examples, structured outputs, and separate advice for reasoning models.
Anthropic Claude
Its prompting best practices recommend clear direct instructions, 3 to 5 examples, and XML tags to separate parts of a prompt.
Ollama
Runs open models such as qwen2.5 on a laptop. Its format parameter takes a JSON schema, which is how this page's structured prompts were enforced.
Writing, testing and improving the input you give a language model so that its output is right and in a shape you can use. The testing is what makes it engineering: a fixed set of cases, a score in code, and one change at a time.
Zero-shot (instruction only), few-shot (add worked examples), structured output (a JSON schema the reply must match), step by step or chain of thought (write the working before the answer), and separating rules into a system message. In my test, writing the working first took the score from 5 to 14 or 15 out of 20.
As a job title it exists but is rare. A 2025 study of 20,662 LinkedIn job postings found 72 prompt engineer roles, under 0.5%. It shows up far more often as a skill inside other jobs, such as software engineer, data scientist or support operations, than as a job of its own.
Writing a prompt is easy. Making one reliable is harder, because small changes can move the score and a model can be confidently wrong. The skills that matter are clear writing, building a good set of test cases, and checking results in code, which most engineers already have.
Less on clever wording and more on context and checks. Reasoning models think step by step on their own, so OpenAI advises against asking them to. The work has moved to supplying the right facts, asking for structured output, guarding against prompt injection, and testing every change on fixed cases.
Prompt engineering is about how you ask: the instructions, examples and format. Context engineering is about what else the model reads: which documents, tool results and chat history go in, in what order, within the token limit. In a real app you need both.
No prompt stops it every time. Clear markers around untrusted text and repeating your rules after it reduce it, but a determined attacker can still get through. Limit what the model's output can do on its own, and check it in code. See prompt injection.
Checked against these on .
Prompt Management and Versioning: Treat Prompts as Production Code
Why prompts are production artifacts, not strings in your app, and how a prompt registry gives you versioning, testing, rollback, lineage, and caching
Vague, Soft and Specific: Asking for Exactly What You Want
Five everyday tasks, each asked three ways on qwen2.5:3b and scored by the same check: vague ('Summarise this'), with a soft word ('Summarise this briefly') and specific ('one sentence of at most 25 words'). Vague met the requirement 0 of 40 times, soft words 0 of 40, specific 38 of 40, and the specific wording also wrote 7.4 times fewer tokens.
Context Engineering
Deciding what goes into a model's limited context window and in what order. Production data shows quality degrades well before the advertised token limit, so what you leave out matters as much as what you put in.
Structured Output
Constraining a model to return valid JSON matching a schema, so downstream code can rely on it. The foundation of reliable tool calling.
Prompt Injection
An attack where untrusted text, often returned by a tool, carries an instruction the model obeys, because the model cannot reliably tell instructions from data. The top security risk for agents, and unsolved by prompting alone.
RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Fine-Tuning
Continuing to train a pretrained model on your own data to change its behavior, style, or format. Best for teaching a skill, not for injecting facts, which is what retrieval is for.
Agentic RAG
Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.
This page explains the idea. The "The Parts of a Prompt: Adding One Piece at a Time" lesson builds it up step by step, with diagrams. It is part of the AI Engineering course, which has 210 lessons. 10 of them are free to read. You pay once and keep it for life, with no subscription.
Open the The Parts of a Prompt: Adding One Piece at a Time lessoncourse 2
Take models from notebook to production, with labs on real models.