systemdesign.academyAI EngineeringFree lessonsSign inGet the course
  1. Home
  2. AI Engineering: Evaluation, LLM Ops and Security
Previous lessonAgent Identity and Authorization: Whose Authority Is It?Next lessonThe Lethal Trifecta: Why Prompt Injection Plus Tools Plus Data Is Unsolved

System Design

  • Foundation
  • Intermediate
  • Advanced
  • Capstone

AI Engineering

  • Foundation
  • Data, RAG and Agents
  • Evaluation, LLM Ops and Security

systemdesign.academy

  • Home
  • Glossary
  • Interview prep
  • Reviews
  • About
  • Pricing
  • Privacy
  • Terms
  • Refunds
  • Delivery
  • Contact
  • AI Security and Agent Safety
  • OWASP Agentic Threat Model and Red-Teaming Your Agent
  • AI Engineering · Security

    OWASP Agentic Threat Model and Red-Teaming Your Agent

    Free preview: the first part of this 19-part lesson.

    Already bought the course? Sign in to read the whole lesson.

    this lesson is part of

    AI Engineering

    Unlock this lesson and the 199 other paid lessons

    • 210 lessons in 16 chapters
    • 118 lessons built on a lab that really ran
    • An AI tutor beside every lesson that answers and draws
    • Lifetime access, every new lesson included

    $49 once, by PayPal

    See the AI Engineering course

    The rest of this lesson

    1. Why Are There Two OWASP Lists, and Which One Do You Need?
    2. What Is Actually On Both Lists?
    3. How Do You Hold Ten Or Fifteen Threats In Your Head?
    4. Which Threats Get Your Attention First?
    5. Where Does Each Threat Meet A Defense You Already Have?
    6. Where Does Each Threat Physically Attach To A Real Agent?
    7. What Do The Model-Layer Threats Actually Mean?
    8. What Does Agency Add That The Top 10 Misses?
    9. How Do You Prove A Defense Works? Attack It Yourself.
    10. Who Runs This Loop? Red Attacks, Blue Hardens.
    11. What Attacks Go In The Corpus?
    12. Do You Automate This, Or Hire A Human?
    13. How Does An Exfiltration Canary Actually Catch A Leak?
    14. Turn The Suite Into One Number: Attack Success Rate
    15. One Number Hides Too Much. Lay It Against The Catalog.
    16. Where Does This Run, And Should Every Attack Block The Merge?
    17. If Guardrails Can Be Bypassed, Why Run Them?
    18. Knowledge Check

    read the rest of this lesson in

    AI Engineering

    Unlock this lesson and the 199 other paid lessons

    • 210 lessons in 16 chapters
    • 118 lessons built on a lab that really ran
    • An AI tutor beside every lesson that answers and draws
    • Lifetime access, every new lesson included

    $49 once, by PayPal

    See the AI Engineering course

    Continue this lesson

    in Security

    This lesson is part of the AI Engineering course. Unlock the full course, a separate one-time payment, for lifetime access.

    One-time payment. Lifetime access. No subscription.
    Global
    $49/lifetime

    One-time payment via PayPal

    Get Lifetime Access
    Foundation lessons free, try before you buy
    India Special
    India
    ₹999/lifetime

    One-time payment: UPI, card or netbanking

    Get Lifetime Access
    Foundation lessons free, try before you buy
    Every ML Platform lesson, Foundation to Advanced
    Real architecture diagrams with real tool logos
    MLOps, LLMOps, and data engineering in depth
    Quizzes with instant feedback
    Built and reviewed by senior engineers
    Lifetime access, all future content included

    Foundation lessons of this course are free, no payment needed. Try them now

    You Have Four Defenses. How Do You Know They Hold?

    By now this chapter has given you four separate defenses. You can break the so injection cannot both read a secret and send it out. You can treat model output as untrusted before it touches a shell or a query. You can sandbox tools and scope each one to least privilege. You can give the agent a real, delegated identity instead of a shared superuser key.

    Four defenses, and one question that ties them together: how do you know they hold, and how do you know you did not miss a threat none of them covers?

    That is the job of this last lesson, and it has two halves. The first half is a map. The security research community, through OWASP, has already cataloged the ways LLM and agent systems get attacked. If you organize that catalog well, every defense you built lands on it in an obvious place, and any gap shows up as a branch with nothing defending it. The second half is a practice. A map tells you what could go wrong. Red-teaming tells you whether it actually does, by attacking your own agent on purpose and turning those attacks into a test that runs on every change.

    Indirect is not theoretical. Researchers have repeatedly shown production assistants reading a hidden instruction from a web page, a document, or an email and then leaking data or calling a tool the user never asked for. Simon Willison has written the clearest running account of this class of attack. The point of a threat model is to stop meeting these one surprise at a time and start meeting them as a list you already checked. By the end of the lesson you will have both halves in hand: a way to place any threat you ever read about, and a suite that tells you, on every commit, whether your specific agent is still safe.