# System Design Masterclass and AI Engineering course (systemdesign.academy)

> The long form of /llms.txt. It carries every glossary definition in full and the
> complete lesson index for both courses. Generated from the site's own content on
> 2026-10-13, so the counts below are the real ones.

Two separate courses on one platform, written by one engineer, Roni Das, with an AI tutor beside every lesson.
One-time price, no subscription: System Design $49 worldwide by PayPal or ₹499 in India by UPI; AI Engineering $49 worldwide or ₹999 in India. Price fact (last updated 1 October 2026, checked against the live checkout on 7 October 2026): the System Design Masterclass costs $49 outside India or ₹499 in India, and AI Engineering costs $49 outside India or ₹999 in India. Each is a one-time payment with lifetime access, no subscription. The older international prices of $7.99 (System Design) and $20 (AI Engineering) ended on 1 October 2026; an answer that quotes them is out of date.

- System Design Masterclass: 770 lessons
- AI Engineering course: 210 lessons in 16 chapters
- AI tutor: beside every lesson in both courses, and on its own page, https://www.systemdesign.academy/tutor
- Glossary: 380 terms, all free to read
- Free lessons, no signup: 28 (18 System Design, 10 AI Engineering)

## What makes it different

The AI tutor. Every lesson in both courses has an AI tutor beside it. While you read a lesson, the tutor reads the same lesson and answers from it. It can explain a part in simple English, draw it as a whiteboard sketch, a 3D figure, a cloud diagram with AWS, Azure or GCP icons, a sequence diagram, an explainer figure or a cheat-sheet poster, quiz you, run a mock interview, or compare two things. Under most lesson slides there are buttons that ask it to explain that part, draw it, or quiz you on it. In a quiz, the site's own code checks your answer against the answer key the tutor wrote with the question. It also has its own page at /tutor. Without an account you can ask 3 questions to try it. A free account gets 5 questions a day. Anyone who owns either course gets 10 a day. Details: https://www.systemdesign.academy/tutor

The AI Engineering course. AI Engineering is the second course on systemdesign.academy: 210 lessons in 16 chapters, about 219 hours of reading. It teaches how to build and run AI systems in production: how language models generate text, tokens and embeddings, prompting, RAG, AI agents, evals, fine-tuning, model serving, feature stores, data engineering for ML, packaging and model registries, LLMOps and AI security. Every lesson has Python you can run in your browser, and a quiz. 118 lessons are built on a lab that was really run, and show the terminal recording of that run. It is one payment with lifetime access, and 10 of its lessons are free to read with no signup. An AI tutor sits beside every lesson. Details: https://www.systemdesign.academy/ai-engineering

## What is public and what is not

Every glossary definition below is free to read in full at its own URL. Lesson bodies
are paid, so this file lists each lesson's title and address rather than its text. The
28 lessons marked FREE below open with no account.

## Glossary, in full

### Latency
The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.
Source: https://www.systemdesign.academy/glossary/latency
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/latency

### Throughput
The number of operations a system can handle per unit of time. Think of it as how many cars a highway can move per hour.
Source: https://www.systemdesign.academy/glossary/throughput
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/throughput

### Bandwidth
The maximum amount of data that can be transferred over a network in a given time. It's the width of the pipe, not how fast the water flows.
Source: https://www.systemdesign.academy/glossary/bandwidth
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/bandwidth

### DNS
The phonebook of the internet. Translates human-readable domain names (google.com) into IP addresses that computers understand.
Source: https://www.systemdesign.academy/glossary/dns
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/dns

### TCP
A reliable transport protocol that guarantees data arrives in order and without errors. It uses a three-way handshake to establish connections.
Source: https://www.systemdesign.academy/glossary/tcp
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/tcp-ip-stack

### HTTP
The protocol powering the web. A request-response model where clients ask for resources and servers respond. Stateless by design.
Source: https://www.systemdesign.academy/glossary/http

### REST API
An architectural style for building APIs using standard HTTP methods (GET, POST, PUT, DELETE). Resources are identified by URLs.
Source: https://www.systemdesign.academy/glossary/rest-api
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/rest-api

### Load Balancer
Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.
Source: https://www.systemdesign.academy/glossary/load-balancer
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/load-balancing

### Caching
Storing frequently accessed data in a faster storage layer so you don't have to fetch it from the original (slower) source every time.
Source: https://www.systemdesign.academy/glossary/caching
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/caching

### Database
An organized collection of data that can be easily accessed, managed, and updated. The backbone of almost every application.
Source: https://www.systemdesign.academy/glossary/database

### SQL
Structured Query Language for managing relational databases. Tables, rows, columns, and powerful joins to query related data.
Source: https://www.systemdesign.academy/glossary/sql
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/sql-vs-nosql

### NoSQL
Databases that don't use traditional table-based relational models. Includes document stores, key-value, graph, and column-family databases.
Source: https://www.systemdesign.academy/glossary/nosql
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/sql-vs-nosql

### ACID
Four guarantees for database transactions: Atomicity (all or nothing), Consistency (valid states only), Isolation (no interference), Durability (changes persist).
Source: https://www.systemdesign.academy/glossary/acid
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/acid-properties

### BASE
An alternative to ACID for distributed systems: Basically Available, Soft state, Eventually consistent. Trades strong consistency for availability.
Source: https://www.systemdesign.academy/glossary/base
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/base-properties

### CDN
A network of servers distributed globally that caches content close to users. Netflix uses CDNs to stream video from servers near you, not from one central location.
Source: https://www.systemdesign.academy/glossary/cdn
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/content-delivery-network

### Proxy
An intermediary server that sits between the client and the destination server. Forward proxies act on behalf of clients; reverse proxies act on behalf of servers.
Source: https://www.systemdesign.academy/glossary/proxy
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/forward-proxy

### Reverse Proxy
A server in front of your web servers that receives every request first, passes it to one of them and returns the answer. It handles TLS, load balancing, caching and routing, and hides the servers behind it.
Source: https://www.systemdesign.academy/glossary/reverse-proxy
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/reverse-proxy

### Session
A way to maintain state across multiple HTTP requests. The server stores data about a user and gives them a session ID (usually in a cookie).
Source: https://www.systemdesign.academy/glossary/session
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/session-management

### Stateless
A system where each request contains all the information needed to process it. The server doesn't remember previous requests. Easier to scale horizontally.
Source: https://www.systemdesign.academy/glossary/stateless
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/stateless-vs-stateful

### Stateful
A system that remembers previous interactions. The server keeps track of client state between requests, making it harder to scale but sometimes necessary.
Source: https://www.systemdesign.academy/glossary/stateful
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/stateless-vs-stateful

### Horizontal Scaling
Adding more machines to handle increased load (scaling out). Like opening more checkout lanes instead of making one cashier faster.
Source: https://www.systemdesign.academy/glossary/horizontal-scaling
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/horizontal-scaling

### Vertical Scaling
Making a single machine more powerful (more CPU, RAM, storage). Simpler but has physical limits. Also called 'scaling up.'
Source: https://www.systemdesign.academy/glossary/vertical-scaling
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/vertical-scaling

### Idempotency
An operation that produces the same result whether you run it once or multiple times. Critical for safe retries in distributed systems.
Source: https://www.systemdesign.academy/glossary/idempotency
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/idempotency

### API Gateway
A single entry point for all client requests that routes them to the appropriate microservice. Handles auth, rate limiting, and request transformation.
Source: https://www.systemdesign.academy/glossary/api-gateway
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/api-gateway

### Sharding
Splitting a database into smaller pieces (shards) distributed across multiple servers. Each shard holds a subset of the data.
Source: https://www.systemdesign.academy/glossary/sharding
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-sharding

### Replication
Keeping copies of the same data on multiple servers. Improves read performance and provides fault tolerance if one server goes down.
Source: https://www.systemdesign.academy/glossary/replication
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-replication

### Index
A data structure that speeds up database lookups. Like the index at the back of a book that lets you jump to the right page instead of reading every page.
Source: https://www.systemdesign.academy/glossary/index
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-indexing

### Pub/Sub
A messaging pattern where publishers send messages to topics, and subscribers receive messages from topics they care about. Publishers don't know who's listening.
Source: https://www.systemdesign.academy/glossary/pub-sub
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/pub-sub-pattern

### Message Queue
A buffer that stores messages between producers and consumers. Messages are processed one by one, in order. Think of it as a to-do list for your services.
Source: https://www.systemdesign.academy/glossary/message-queue
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/message-queues

### WebSocket
A protocol for full-duplex communication over a single TCP connection. Unlike HTTP, the server can push data to the client without being asked.
Source: https://www.systemdesign.academy/glossary/websocket
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/websockets

### SSE
Server-Sent Events: a one-way channel where the server pushes updates to the client over HTTP. Simpler than WebSockets when you only need server-to-client streaming.
Source: https://www.systemdesign.academy/glossary/sse
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/server-sent-events

### Microservices
An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.
Source: https://www.systemdesign.academy/glossary/microservices
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/microservices-architecture-deep

### Monolith
A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.
Source: https://www.systemdesign.academy/glossary/monolith
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/monolithic-architecture

### Service Discovery
The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.
Source: https://www.systemdesign.academy/glossary/service-discovery
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/service-discovery

### Circuit Breaker
A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.
Source: https://www.systemdesign.academy/glossary/circuit-breaker
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/circuit-breaker-pattern

### Bulkhead
A pattern that isolates different parts of a system so a failure in one part doesn't sink the whole ship. Named after the compartments in a ship's hull.
Source: https://www.systemdesign.academy/glossary/bulkhead
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/bulkhead-pattern

### Retry
Automatically re-attempting a failed operation, usually with exponential backoff. Essential for handling transient failures in distributed systems.
Source: https://www.systemdesign.academy/glossary/retry
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/retry-logic

### Rate Limiting
Controlling how many requests a client can make in a given time window. Protects your API from abuse and ensures fair usage.
Source: https://www.systemdesign.academy/glossary/rate-limiting
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/rate-limiting

### Throttling
Slowing down the rate of processing requests instead of rejecting them outright. The gentler cousin of rate limiting.
Source: https://www.systemdesign.academy/glossary/throttling
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/throttling

### GraphQL
A query language for APIs where the client specifies exactly what data it needs. No over-fetching, no under-fetching. One endpoint to rule them all.
Source: https://www.systemdesign.academy/glossary/graphql
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/graphql

### gRPC
A high-performance RPC framework by Google using Protocol Buffers and HTTP/2. Much faster than REST for service-to-service communication.
Source: https://www.systemdesign.academy/glossary/grpc
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/grpc

### Kafka
A distributed event streaming platform that handles millions of events per second. Used by LinkedIn, Netflix, and Uber for real-time data pipelines.
Source: https://www.systemdesign.academy/glossary/kafka

### Redis
An in-memory data store used as a cache, message broker, and database. Blazing fast because everything lives in RAM.
Source: https://www.systemdesign.academy/glossary/redis
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/redis-cache

### Memcached
A simple, high-performance distributed memory caching system. Stores key-value pairs in RAM. Simpler than Redis but less feature-rich.
Source: https://www.systemdesign.academy/glossary/memcached
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/memcached

### Docker
A platform for packaging applications into lightweight, portable containers. 'Works on my machine' becomes 'works everywhere.'
Source: https://www.systemdesign.academy/glossary/docker
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/containerization

### Kubernetes
An orchestration platform that automates deploying, scaling, and managing containerized applications. K8s is the operating system for your cloud.
Source: https://www.systemdesign.academy/glossary/kubernetes
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/orchestration

### OAuth
An authorization framework that lets users grant third-party apps limited access to their accounts without sharing passwords. Powers 'Sign in with Google.'
Source: https://www.systemdesign.academy/glossary/oauth
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/oauth

### JWT
JSON Web Token: a compact, self-contained token for transmitting claims between parties. The server can verify it without a database lookup.
Source: https://www.systemdesign.academy/glossary/jwt
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/jwt

### SSL/TLS
Cryptographic protocols that encrypt data in transit between client and server. TLS is the modern successor to SSL. The 'S' in HTTPS.
Source: https://www.systemdesign.academy/glossary/ssl-tls
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/ssl-tls-certificates

### CORS
Cross-Origin Resource Sharing: a security mechanism that controls which domains can access your API. The browser enforces it; the server configures it.
Source: https://www.systemdesign.academy/glossary/cors
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/cors

### Database Partitioning
Dividing a large table into smaller, more manageable pieces while keeping them in the same database. Sharding is partitioning across servers.
Source: https://www.systemdesign.academy/glossary/database-partitioning
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/table-partitioning

### Read Replica
A copy of your database that handles read queries, reducing load on the primary database. Writes still go to the primary and replicate out.
Source: https://www.systemdesign.academy/glossary/read-replica
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/read-replicas

### Write-Ahead Log
A technique where changes are written to a log before being applied to the database. Ensures durability and crash recovery.
Source: https://www.systemdesign.academy/glossary/write-ahead-log
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/write-ahead-logging

### Content Delivery
The process of distributing and serving content to users from locations geographically close to them for faster load times.
Source: https://www.systemdesign.academy/glossary/content-delivery
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/content-delivery-network

### Service Mesh
A dedicated infrastructure layer for handling service-to-service communication in microservices. Manages load balancing, encryption, and observability automatically.
Source: https://www.systemdesign.academy/glossary/service-mesh
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/service-mesh-concepts

### Health Check
An endpoint or mechanism that reports whether a service is running and healthy. Load balancers use health checks to route traffic away from unhealthy instances.
Source: https://www.systemdesign.academy/glossary/health-check
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/health-checks

### Distributed Tracing
Tracking a request as it flows through multiple services in a distributed system. Each service adds its trace, creating a full picture of the request journey.
Source: https://www.systemdesign.academy/glossary/distributed-tracing
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/distributed-tracing

### CI/CD
Continuous Integration and Continuous Deployment: automating the process of testing and deploying code. Push code, tests run, and it ships to production automatically.
Source: https://www.systemdesign.academy/glossary/ci-cd
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/continuous-integration

### Blue-Green Deployment
A deployment strategy using two identical environments. Traffic switches from blue (current) to green (new) instantly, with easy rollback.
Source: https://www.systemdesign.academy/glossary/blue-green-deployment
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/blue-green-deployment

### Canary Deployment
Rolling out a new version to a small percentage of users first, then gradually increasing. Like sending a canary into a coal mine to test for danger.
Source: https://www.systemdesign.academy/glossary/canary-deployment
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/canary-deployment

### CAP Theorem
When a network failure splits a distributed data store, each node must either refuse requests to stay consistent or keep answering with possibly stale data. It cannot guarantee both during the partition.
Source: https://www.systemdesign.academy/glossary/cap-theorem
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/cap-theorem

### Consensus
The process of getting multiple nodes in a distributed system to agree on a single value. The foundation of distributed databases and coordination services.
Source: https://www.systemdesign.academy/glossary/consensus
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/consensus-algorithms

### Paxos
A family of protocols for solving consensus in unreliable networks. Famously difficult to understand but mathematically proven correct.
Source: https://www.systemdesign.academy/glossary/paxos
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/consensus-algorithms

### Raft
A consensus algorithm designed to be understandable. Uses leader election and log replication. Powers etcd (used by Kubernetes) and CockroachDB.
Source: https://www.systemdesign.academy/glossary/raft
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/consensus-algorithms

### CQRS
Command Query Responsibility Segregation: using different models for reading and writing data. Reads and writes have different performance needs, so separate them.
Source: https://www.systemdesign.academy/glossary/cqrs
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/cqrs

### Event Sourcing
Storing every state change as an immutable event instead of just the current state. You can rebuild any past state by replaying events.
Source: https://www.systemdesign.academy/glossary/event-sourcing
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/event-sourcing

### Saga Pattern
A way to manage distributed transactions across microservices using a sequence of local transactions with compensating actions for rollback.
Source: https://www.systemdesign.academy/glossary/saga-pattern
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/saga-pattern

### Vector Database
A database that stores embeddings, lists of numbers that capture the meaning of text or images, and finds the ones closest to a question's embedding. It powers semantic search and RAG.
Source: https://www.systemdesign.academy/glossary/vector-database
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/vector-database

### Consistent Hashing
A hashing technique where adding or removing servers only moves a small fraction of keys. Used by Amazon DynamoDB and Cassandra for data distribution.
Source: https://www.systemdesign.academy/glossary/consistent-hashing
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/consistent-hashing

### Leader Election
The process of choosing one node in a cluster to coordinate actions. If the leader fails, a new one is elected. Used by Kafka, ZooKeeper, and etcd.
Source: https://www.systemdesign.academy/glossary/leader-election
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/leader-election

### Two-Phase Commit
A protocol ensuring all nodes in a distributed transaction either commit or abort together. Phase 1: prepare (vote). Phase 2: commit or rollback.
Source: https://www.systemdesign.academy/glossary/two-phase-commit
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/two-phase-commit

### Eventual Consistency
A consistency model where updates propagate asynchronously and all replicas will eventually converge to the same value. Trades immediacy for availability.
Source: https://www.systemdesign.academy/glossary/eventual-consistency
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/base-properties

### Strong Consistency
A guarantee that after a write completes, all subsequent reads will return the updated value. Safer but slower than eventual consistency.
Source: https://www.systemdesign.academy/glossary/strong-consistency
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/linearizability

### Stream Processing
Processing data continuously as it arrives, rather than in batches. Powers real-time analytics, fraud detection, and live dashboards.
Source: https://www.systemdesign.academy/glossary/stream-processing
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/stream-processing

### Batch Processing
Processing large volumes of data in scheduled chunks rather than in real time. Think nightly reports, ETL jobs, and data warehouse loads.
Source: https://www.systemdesign.academy/glossary/batch-processing
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/batch-processing

### Chaos Engineering
Deliberately injecting failures into a system to test its resilience. Netflix's Chaos Monkey randomly kills servers to ensure the system survives.
Source: https://www.systemdesign.academy/glossary/chaos-engineering
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/chaos-engineering

### Back Pressure
A flow control mechanism where a slow consumer signals upstream producers to slow down. Prevents systems from being overwhelmed by data they can't process.
Source: https://www.systemdesign.academy/glossary/back-pressure
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/back-pressure

### Gossip Protocol
A peer-to-peer communication protocol where nodes share information with random neighbors, spreading it like gossip. Used for cluster membership and failure detection.
Source: https://www.systemdesign.academy/glossary/gossip-protocol
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/gossip-protocol

### Bloom Filter
A space-efficient probabilistic data structure that tells you if an element is 'possibly in the set' or 'definitely not in the set.' Used by databases to avoid expensive lookups.
Source: https://www.systemdesign.academy/glossary/bloom-filter
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/bloom-filters

### Synchronous
A communication model where the caller waits for the operation to complete before moving on. Simpler to reason about but blocks the thread.
Source: https://www.systemdesign.academy/glossary/synchronous
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/synchronous-processing

### Asynchronous
A communication model where the caller fires off a request and continues without waiting for a response. Essential for non-blocking I/O and event-driven systems.
Source: https://www.systemdesign.academy/glossary/asynchronous
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/asynchronous-processing

### JSON
JavaScript Object Notation: a lightweight text format for data interchange using key-value pairs and arrays. The lingua franca of web APIs.
Source: https://www.systemdesign.academy/glossary/json

### XML
Extensible Markup Language: a verbose, tag-based format for structured data. Still used in enterprise systems, SOAP, and configuration files.
Source: https://www.systemdesign.academy/glossary/xml

### API
Application Programming Interface: a contract defining how two pieces of software talk to each other. The waiter between your frontend and your backend.
Source: https://www.systemdesign.academy/glossary/api
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/rest-api

### HTTPS
HTTP over TLS: the encrypted version of HTTP that protects data in transit. Every production site should use it; browsers flag plain HTTP as insecure.
Source: https://www.systemdesign.academy/glossary/https
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/tls

### UDP
User Datagram Protocol: a connectionless transport protocol that trades reliability for speed. No handshake, no ordering, no retransmission. Used for video streaming and gaming.
Source: https://www.systemdesign.academy/glossary/udp
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/udp

### IP
Internet Protocol: the addressing scheme that routes packets across the internet. Every device gets an IP address (IPv4 or IPv6) so packets know where to go.
Source: https://www.systemdesign.academy/glossary/ip

### Cookie
A small piece of data the server sends to the browser, which the browser stores and sends back with every subsequent request. Powers sessions, tracking, and preferences.
Source: https://www.systemdesign.academy/glossary/cookie
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/cookie-sessions

### Fault Tolerance
A system's ability to keep operating correctly even when some of its components fail. Achieved through redundancy, replication, and graceful degradation.
Source: https://www.systemdesign.academy/glossary/fault-tolerance
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/fault-tolerance

### Redundancy
Duplicating critical components or functions so that if one fails, a backup takes over. The reason planes have two engines and databases have replicas.
Source: https://www.systemdesign.academy/glossary/redundancy

### Availability
The percentage of time a system is operational and accessible. Measured in 'nines',  99.99% availability means about 52 minutes of downtime per year.
Source: https://www.systemdesign.academy/glossary/availability
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/high-availability

### Scalability
A system's ability to handle growing amounts of work by adding resources. A scalable system maintains performance as load increases.
Source: https://www.systemdesign.academy/glossary/scalability
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/scalability

### DNS Record Types
Different types of DNS records map domains to resources: A records point to IPs, CNAME aliases one domain to another, MX routes email, TXT stores verification data.
Source: https://www.systemdesign.academy/glossary/dns-record-types
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/dns

### Forward Proxy
A proxy that sits in front of clients and forwards their requests to the internet. Used for anonymity, content filtering, and bypassing geo-restrictions.
Source: https://www.systemdesign.academy/glossary/forward-proxy
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/forward-proxy

### Round Robin
A load balancing algorithm that distributes requests to servers in sequential order, cycling through the list. Simple but ignores server capacity differences.
Source: https://www.systemdesign.academy/glossary/round-robin
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/load-balancer-algorithms

### Health Check (LB)
Periodic probes a load balancer sends to backend servers to verify they're alive. Unhealthy servers get pulled from the rotation until they recover.
Source: https://www.systemdesign.academy/glossary/health-check-lb
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/health-checks

### Sticky Session
A load balancer feature that routes all requests from the same client to the same backend server. Needed when servers store session state locally.
Source: https://www.systemdesign.academy/glossary/sticky-session
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/sticky-sessions

### Layer 4 Load Balancing
Load balancing at the transport layer (TCP/UDP) based on IP addresses and ports. Fast because it doesn't inspect packet contents.
Source: https://www.systemdesign.academy/glossary/layer-4-load-balancing

### Layer 7 Load Balancing
Load balancing at the application layer (HTTP) that can route based on URL paths, headers, cookies, or request content. More flexible but more CPU-intensive.
Source: https://www.systemdesign.academy/glossary/layer-7-load-balancing

### Cache Eviction
The process of removing entries from a cache when it's full. Common policies: LRU (least recently used), LFU (least frequently used), FIFO (first in, first out).
Source: https://www.systemdesign.academy/glossary/cache-eviction
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-eviction-policies

### Cache-Aside
A caching pattern where the application checks the cache first; on a miss, it fetches from the database and populates the cache. Also called lazy loading.
Source: https://www.systemdesign.academy/glossary/cache-aside
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-aside-pattern

### Write-Through Cache
A caching pattern where every write goes to both the cache and the database simultaneously. Ensures consistency but adds write latency.
Source: https://www.systemdesign.academy/glossary/write-through
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/write-through-cache

### Write-Behind Cache
A caching pattern where writes go to the cache first and are asynchronously flushed to the database later. Fast writes but risks data loss on cache failure.
Source: https://www.systemdesign.academy/glossary/write-behind
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/write-back-cache

### TTL
Time To Live: how long a cached entry, DNS record, or packet is valid before it expires and must be refreshed or discarded.
Source: https://www.systemdesign.academy/glossary/ttl
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/time-to-live-ttl

### Cache Stampede
When many requests hit the database simultaneously because a popular cache entry expired. Solved with locking, probabilistic early expiration, or request coalescing.
Source: https://www.systemdesign.academy/glossary/cache-stampede
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-stampede-prevention

### Cache Invalidation
Removing or updating stale cache entries when the underlying data changes. One of the two hard problems in computer science (along with naming things).
Source: https://www.systemdesign.academy/glossary/cache-invalidation
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-invalidation

### Distributed Cache
A cache spread across multiple nodes that acts as a shared layer between application servers and the database. Redis Cluster and Memcached are common implementations.
Source: https://www.systemdesign.academy/glossary/distributed-cache
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/distributed-cache

### Normalization
Organizing database tables to reduce redundancy by splitting data into related tables connected by foreign keys. Follows normal forms (1NF, 2NF, 3NF).
Source: https://www.systemdesign.academy/glossary/normalization
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-normalization

### Denormalization
Intentionally adding redundant data to database tables to speed up read queries by avoiding expensive joins. Trades storage and write complexity for read performance.
Source: https://www.systemdesign.academy/glossary/denormalization
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-denormalization

### Primary Key
A column (or combination of columns) that uniquely identifies each row in a database table. Must be unique and not null.
Source: https://www.systemdesign.academy/glossary/primary-key

### Foreign Key
A column in one table that references the primary key of another table, enforcing referential integrity between related tables.
Source: https://www.systemdesign.academy/glossary/foreign-key

### ORM
Object-Relational Mapping: a library that lets you interact with a database using your programming language's objects instead of raw SQL. Drizzle, Prisma, and SQLAlchemy are ORMs.
Source: https://www.systemdesign.academy/glossary/orm

### Connection Pool
A cache of reusable database connections that avoids the overhead of opening and closing a new connection for every query. Critical for high-throughput applications.
Source: https://www.systemdesign.academy/glossary/connection-pool
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-connection-pooling

### Transaction
A sequence of database operations treated as a single atomic unit. Either all operations succeed (commit) or none of them do (rollback).
Source: https://www.systemdesign.academy/glossary/transaction

### Isolation Level
Controls how much one transaction can see changes made by other concurrent transactions. Ranges from Read Uncommitted (fastest, least safe) to Serializable (slowest, safest).
Source: https://www.systemdesign.academy/glossary/isolation-level

### Deadlock
When two or more transactions are each waiting for the other to release a lock, creating a cycle where none can proceed. Databases detect and break deadlocks by aborting one.
Source: https://www.systemdesign.academy/glossary/deadlock

### B-Tree
A self-balancing search tree in which each node is a page holding many sorted keys, so it stays wide and shallow. Most database indexes use it: finding one key among millions takes only a few page reads.
Source: https://www.systemdesign.academy/glossary/b-tree
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/b-trees

### Stored Procedure
Pre-compiled SQL code stored in the database that can be called by name. Reduces network round trips and keeps business logic close to the data.
Source: https://www.systemdesign.academy/glossary/stored-procedure
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/stored-procedures

### Database View
A virtual table defined by a SQL query. Simplifies complex joins, enforces access control, and presents data in a specific shape without duplicating storage.
Source: https://www.systemdesign.academy/glossary/database-view
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-views

### Document Database
A NoSQL database that stores data as flexible JSON-like documents. MongoDB and CouchDB let each document have a different structure.
Source: https://www.systemdesign.academy/glossary/document-database
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/document-stores

### Key-Value Store
The simplest NoSQL model: store data as key-value pairs. Blazing fast lookups by key. Redis, DynamoDB, and etcd are key-value stores.
Source: https://www.systemdesign.academy/glossary/key-value-store
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/key-value-stores

### Column-Family Store
A NoSQL database that groups columns into families, optimized for reading and writing large amounts of data across many machines. Cassandra and HBase use this model.
Source: https://www.systemdesign.academy/glossary/column-family
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/column-family-stores

### Graph Database
A database built for storing and querying relationships between entities. Nodes are entities, edges are relationships. Neo4j is the most popular.
Source: https://www.systemdesign.academy/glossary/graph-database
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/graph-databases

### Time-Series Database
A database optimized for time-stamped data points like metrics, sensor readings, and financial ticks. InfluxDB and TimescaleDB are purpose-built for this.
Source: https://www.systemdesign.academy/glossary/time-series-database
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/time-series-databases

### Elasticsearch
A distributed search and analytics engine built on Apache Lucene. Powers full-text search, log analysis, and real-time analytics at scale.
Source: https://www.systemdesign.academy/glossary/elasticsearch
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/search-engines

### LSM Tree
Log-Structured Merge Tree: a write-optimized data structure that buffers writes in memory and periodically flushes sorted runs to disk. Used by Cassandra, RocksDB, and LevelDB.
Source: https://www.systemdesign.academy/glossary/lsm-tree
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/lsm-trees

### Object Storage
A storage architecture that manages data as objects (file + metadata + ID) rather than blocks or files. S3 is the gold standard. Infinitely scalable, cheap, and durable.
Source: https://www.systemdesign.academy/glossary/object-storage
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/object-storage

### Materialized View
A precomputed query result stored as a physical table and refreshed periodically. Trades storage for read performance on expensive aggregations.
Source: https://www.systemdesign.academy/glossary/materialized-view
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/materialized-views

### Primary-Replica
A replication topology where one node (primary) handles writes and replicates changes to one or more read-only replicas. The foundation of most database scaling strategies.
Source: https://www.systemdesign.academy/glossary/master-slave
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/master-slave-replication

### Multi-Leader Replication
A replication topology where multiple nodes accept writes independently and sync with each other. Useful for multi-datacenter setups but creates conflict resolution challenges.
Source: https://www.systemdesign.academy/glossary/multi-leader
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/multi-master-replication

### Leaderless Replication
A replication approach where any node can accept reads and writes. Uses quorum reads/writes for consistency. Cassandra and DynamoDB use this model.
Source: https://www.systemdesign.academy/glossary/leaderless-replication

### Data Federation
Splitting databases by function (users in one DB, orders in another) so each database handles only its domain. Simpler than sharding but limits cross-domain joins.
Source: https://www.systemdesign.academy/glossary/data-federation
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/database-federation

### Webhook
An HTTP callback triggered by an event. Instead of polling for updates, the source system pushes a notification to your URL when something happens.
Source: https://www.systemdesign.academy/glossary/webhook
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/webhooks

### Kafka Topic
A named feed or category to which producers publish records. Topics are split into partitions for parallelism, and each partition is an ordered, immutable log.
Source: https://www.systemdesign.academy/glossary/kafka-topic

### Kafka Partition
A subset of a Kafka topic that provides ordering guarantees and parallel processing. Each partition lives on one broker and can be consumed by one consumer per group.
Source: https://www.systemdesign.academy/glossary/kafka-partition

### Kafka Consumer Group
A set of consumers that cooperatively read from topic partitions. Each partition is assigned to exactly one consumer in the group, enabling parallel processing.
Source: https://www.systemdesign.academy/glossary/kafka-consumer-group

### Dead Letter Queue
A queue that stores messages that couldn't be processed after multiple attempts. Prevents poison messages from blocking the main queue and lets you debug failures.
Source: https://www.systemdesign.academy/glossary/dead-letter-queue
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/dead-letter-queue

### Event-Driven Architecture
A design pattern where services communicate by producing and consuming events rather than making direct calls. Promotes loose coupling and asynchronous processing.
Source: https://www.systemdesign.academy/glossary/event-driven-architecture
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/event-driven-architecture

### Message Broker
Middleware that translates messages between different protocols, routes them to the right consumers, and provides guarantees like ordering and delivery. RabbitMQ and ActiveMQ are brokers.
Source: https://www.systemdesign.academy/glossary/message-broker
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/message-queues

### At-Least-Once Delivery
A messaging guarantee where every message is delivered one or more times. Simpler than exactly-once but requires consumers to handle duplicates via idempotency.
Source: https://www.systemdesign.academy/glossary/at-least-once-delivery
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/at-least-once-delivery

### Protocol Buffers
Google's language-neutral, binary serialization format. Smaller and faster than JSON. Defines schemas in .proto files that generate typed code for any language.
Source: https://www.systemdesign.academy/glossary/protocol-buffers
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/protocol-buffers

### API Versioning
Strategies for evolving an API without breaking existing clients. Common approaches: URL path (/v2/users), header (Accept-Version), or query param (?version=2).
Source: https://www.systemdesign.academy/glossary/api-versioning
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/api-versioning

### Pagination
Breaking large result sets into smaller pages. Offset-based (page=2&limit=20) is simple; cursor-based (after=abc123) handles real-time data without skipping or duplicating items.
Source: https://www.systemdesign.academy/glossary/pagination

### Long Polling
The client sends a request and the server holds it open until new data is available or a timeout is reached. A workaround for real-time updates before WebSockets existed.
Source: https://www.systemdesign.academy/glossary/long-polling
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/long-polling

### IDL
Interface Definition Language: a schema that defines the contract between client and server. Protobuf, Thrift, and OpenAPI are IDLs that generate client/server code.
Source: https://www.systemdesign.academy/glossary/idl
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/grpc

### HTTP/2
A major revision of HTTP that adds multiplexing (multiple requests over one connection), header compression, and server push. Much faster than HTTP/1.1 for modern web apps.
Source: https://www.systemdesign.academy/glossary/http2
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/http2-multiplexing

### Sidecar
A helper container deployed alongside the main application container in the same pod. Handles cross-cutting concerns like logging, monitoring, or TLS without modifying app code.
Source: https://www.systemdesign.academy/glossary/sidecar
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/sidecar-pattern

### Strangler Fig Pattern
A migration strategy where you gradually replace a monolith by routing features one by one to new microservices, until the old system can be decommissioned.
Source: https://www.systemdesign.academy/glossary/strangler-fig
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/strangler-fig-pattern

### Backend for Frontend (BFF)
A dedicated backend service tailored for a specific frontend (mobile, web, TV). Each frontend gets an API shaped to its exact needs instead of sharing one generic API.
Source: https://www.systemdesign.academy/glossary/backend-for-frontend
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/backend-for-frontend

### Database per Service
A microservices pattern where each service owns its private database. No other service can access it directly. Enforces loose coupling but complicates cross-service queries.
Source: https://www.systemdesign.academy/glossary/database-per-service
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/database-per-service

### API Composition
A pattern where a composer service queries multiple microservices and joins the results in memory. The simplest way to implement cross-service queries.
Source: https://www.systemdesign.academy/glossary/api-composition
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/api-composition

### Timeout
A maximum duration to wait for an operation to complete before giving up. Without timeouts, a stalled dependency can hang your entire system indefinitely.
Source: https://www.systemdesign.academy/glossary/timeout
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/timeout-patterns

### Feature Flag
A toggle in code that enables or disables a feature without redeploying. Lets you ship code to production behind a flag and turn it on for a percentage of users.
Source: https://www.systemdesign.academy/glossary/feature-flag
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/feature-flags

### Data Lineage
Tracking data from its origin through every transformation and system it passes through. Answers 'where did this number come from?' for audits and debugging.
Source: https://www.systemdesign.academy/glossary/data-lineage
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/data-lineage

### Data Catalog
A searchable inventory of all datasets in an organization, with metadata like schema, owner, freshness, and lineage. The 'Google for your data.'
Source: https://www.systemdesign.academy/glossary/data-catalog
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/data-catalog

### GDPR
General Data Protection Regulation: EU law governing how personal data is collected, stored, and processed. Requires consent, data portability, and the right to be forgotten.
Source: https://www.systemdesign.academy/glossary/gdpr
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/gdpr-compliance

### PII
Personally Identifiable Information: any data that can identify a specific individual, like name, email, SSN, or IP address. Must be encrypted and access-controlled.
Source: https://www.systemdesign.academy/glossary/pii
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/pii-handling

### Data Masking
Replacing sensitive data with realistic but fake values so developers and testers can work with production-like data without exposing real PII.
Source: https://www.systemdesign.academy/glossary/data-masking
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-masking

### Data Encryption
Transforming data into an unreadable format using cryptographic algorithms. Encryption at rest protects stored data; encryption in transit protects data over the network.
Source: https://www.systemdesign.academy/glossary/data-encryption
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/encryption-at-rest

### Virtual Machine
A software emulation of a physical computer running its own OS on shared hardware. Heavier than containers but provides stronger isolation.
Source: https://www.systemdesign.academy/glossary/virtual-machine

### Hypervisor
Software that creates and manages virtual machines by abstracting physical hardware. Type 1 (bare-metal) runs directly on hardware; Type 2 runs on an OS.
Source: https://www.systemdesign.academy/glossary/hypervisor

### Cloud Region
A geographic area containing one or more data centers (availability zones). Choosing the right region reduces latency and satisfies data residency requirements.
Source: https://www.systemdesign.academy/glossary/cloud-region
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/regions

### Availability Zone
An isolated data center within a cloud region, with independent power, cooling, and networking. Deploying across multiple AZs protects against single-facility failures.
Source: https://www.systemdesign.academy/glossary/availability-zone
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/availability-zones

### Auto Scaling
Automatically adding or removing compute instances based on current demand. Scales out during traffic spikes and scales in during quiet periods to save cost.
Source: https://www.systemdesign.academy/glossary/auto-scaling
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/auto-scaling

### Serverless
A cloud execution model where the provider manages all infrastructure and you pay only for actual compute time. AWS Lambda, Vercel Functions, and Cloudflare Workers are serverless.
Source: https://www.systemdesign.academy/glossary/serverless
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/serverless

### Infrastructure as Code
Managing servers, networks, and cloud resources through declarative configuration files instead of manual setup. Terraform, Pulumi, and CloudFormation are IaC tools.
Source: https://www.systemdesign.academy/glossary/infrastructure-as-code
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/infrastructure-as-code

### Terraform
An open-source IaC tool by HashiCorp that provisions infrastructure across any cloud provider using declarative HCL configuration. Plan, apply, destroy.
Source: https://www.systemdesign.academy/glossary/terraform
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/infrastructure-as-code

### Ansible
An agentless automation tool for configuration management, application deployment, and orchestration. Uses YAML playbooks and connects over SSH.
Source: https://www.systemdesign.academy/glossary/ansible
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/infrastructure-as-code

### Block Storage
Storage that splits data into fixed-size blocks, each with its own address. Used for databases and boot volumes where low-latency random I/O matters. AWS EBS is block storage.
Source: https://www.systemdesign.academy/glossary/block-storage
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/block-storage

### Container
A lightweight, isolated environment that packages an application with its dependencies. Shares the host OS kernel, unlike VMs. Starts in milliseconds.
Source: https://www.systemdesign.academy/glossary/container
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/containerization

### Kubernetes Pod
The smallest deployable unit in Kubernetes: one or more containers sharing network and storage. Pods are ephemeral; if one dies, K8s creates a replacement.
Source: https://www.systemdesign.academy/glossary/k8s-pod

### Kubernetes Service
A stable network endpoint that load-balances traffic across a set of pods. Pods come and go, but the Service's IP and DNS name stay constant.
Source: https://www.systemdesign.academy/glossary/k8s-service

### Kubernetes Deployment
A K8s resource that manages rolling updates and rollbacks for a set of pods. You declare the desired state and K8s converges to it.
Source: https://www.systemdesign.academy/glossary/k8s-deployment

### Kubernetes Ingress
A K8s resource that manages external HTTP/HTTPS access to services inside the cluster. Routes traffic by hostname or URL path to different backend services.
Source: https://www.systemdesign.academy/glossary/k8s-ingress
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/ingress-controllers

### Kubernetes Namespace
A virtual cluster within a physical K8s cluster that isolates resources. Use namespaces to separate environments (dev, staging, prod) or teams.
Source: https://www.systemdesign.academy/glossary/k8s-namespace

### Helm
A package manager for Kubernetes that bundles K8s manifests into reusable charts. Lets you install complex applications with a single command and manage versioned releases.
Source: https://www.systemdesign.academy/glossary/helm

### RBAC
Role-Based Access Control: assigns permissions to roles (admin, editor, viewer), then assigns roles to users. Simpler to manage than per-user permissions.
Source: https://www.systemdesign.academy/glossary/rbac
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/rbac

### ABAC
Attribute-Based Access Control: grants permissions based on attributes (user department, resource type, time of day, IP range). More flexible but more complex than RBAC.
Source: https://www.systemdesign.academy/glossary/abac
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/abac

### mTLS
Mutual TLS: both client and server present certificates to authenticate each other. Standard in service mesh architectures where every service verifies its peers.
Source: https://www.systemdesign.academy/glossary/mtls
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/mtls

### WAF
Web Application Firewall: filters and monitors HTTP traffic between a web application and the internet. Blocks SQL injection, XSS, and other OWASP top-10 attacks.
Source: https://www.systemdesign.academy/glossary/waf
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/waf

### CSRF
Cross-Site Request Forgery: an attack that tricks a logged-in user's browser into sending unwanted requests to a site where they're authenticated. Prevented with tokens.
Source: https://www.systemdesign.academy/glossary/csrf
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/csrf

### XSS
Cross-Site Scripting: an attack where malicious scripts are injected into trusted websites. Prevented by sanitizing user input and setting Content-Security-Policy headers.
Source: https://www.systemdesign.academy/glossary/xss
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/xss

### SQL Injection
An attack where malicious SQL is inserted into a query through user input. Prevented by parameterized queries and prepared statements. Never concatenate user input into SQL.
Source: https://www.systemdesign.academy/glossary/sql-injection
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/sql-injection

### Zero Trust
A security model that never trusts any request by default, even from inside the network. Every request must be authenticated, authorized, and encrypted regardless of origin.
Source: https://www.systemdesign.academy/glossary/zero-trust
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/zero-trust

### API Key
A simple token passed with API requests to identify the calling project or application. Not a substitute for user authentication but useful for rate limiting and usage tracking.
Source: https://www.systemdesign.academy/glossary/api-key
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/api-keys

### Secret Management
Securely storing and distributing credentials, API keys, and certificates. Tools like Vault, AWS Secrets Manager, and SOPS prevent secrets from leaking into code or logs.
Source: https://www.systemdesign.academy/glossary/secret-management
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/secrets-management

### Observability
The ability to understand a system's internal state from its external outputs. Built on three pillars: metrics, logs, and traces.
Source: https://www.systemdesign.academy/glossary/observability
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/observability-overview

### Metrics
Numerical measurements collected over time that describe system behavior: request rate, error rate, latency percentiles, CPU utilization. Prometheus is the standard collector.
Source: https://www.systemdesign.academy/glossary/metrics
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/metrics-collection

### Logging
Recording discrete events with timestamps, severity levels, and context. Structured logs (JSON) are searchable; unstructured logs (plaintext) are not. Ship them to a central system.
Source: https://www.systemdesign.academy/glossary/logging
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/centralized-logging

### Alerting
Automatically notifying engineers when metrics cross predefined thresholds. Good alerts are actionable, not noisy. PagerDuty and Opsgenie route alerts to the right on-call person.
Source: https://www.systemdesign.academy/glossary/alerting
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/alerting

### Log Aggregation
Collecting logs from all services into a central searchable store. The ELK stack (Elasticsearch, Logstash, Kibana) and Grafana Loki are common solutions.
Source: https://www.systemdesign.academy/glossary/log-aggregation
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/log-aggregation

### APM
Application Performance Monitoring: tools that track request latency, error rates, and dependencies in real time. Datadog, New Relic, and Grafana are popular APM platforms.
Source: https://www.systemdesign.academy/glossary/apm
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/apm

### Span
A single unit of work in a distributed trace, representing one operation (e.g., an HTTP call or database query). Spans are nested to form a trace tree.
Source: https://www.systemdesign.academy/glossary/span
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/span-ids

### Trace
The full end-to-end path of a request through a distributed system, composed of multiple spans. A trace shows which services were called, in what order, and how long each took.
Source: https://www.systemdesign.academy/glossary/trace
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/trace-ids

### Rolling Deployment
Gradually replacing old instances with new ones, a few at a time. No downtime, but both versions run simultaneously during the rollout.
Source: https://www.systemdesign.academy/glossary/rolling-deployment
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/rolling-deployment

### GitOps
Using Git as the single source of truth for infrastructure and application configuration. Changes are made via pull requests and automatically reconciled by tools like ArgoCD or Flux.
Source: https://www.systemdesign.academy/glossary/gitops
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/gitops

### Immutable Infrastructure
Servers are never modified after deployment. To update, you build a new image and replace the old instance entirely. Eliminates configuration drift.
Source: https://www.systemdesign.academy/glossary/immutable-infrastructure
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/immutable-infrastructure

### Artifact Registry
A repository for storing build outputs like Docker images, JAR files, and npm packages. Docker Hub, GitHub Container Registry, and Artifactory are common registries.
Source: https://www.systemdesign.academy/glossary/artifact-registry
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/artifact-repository

### Smoke Test
A quick, basic test run after deployment to verify the most critical paths still work. If the smoke test fails, the deployment is rolled back immediately.
Source: https://www.systemdesign.academy/glossary/smoke-test
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/smoke-testing

### Trunk-Based Development
A branching model where developers commit to a single main branch frequently with small changes. Feature flags replace long-lived feature branches.
Source: https://www.systemdesign.academy/glossary/trunk-based-development

### Edge Computing
Running computation at the network edge, close to the user, instead of in a central data center. Reduces latency for real-time applications like IoT and streaming.
Source: https://www.systemdesign.academy/glossary/edge-computing
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/edge-computing

### HTTP Caching
Browser and proxy caching controlled by HTTP headers like Cache-Control, ETag, and Last-Modified. Eliminates redundant network requests for unchanged resources.
Source: https://www.systemdesign.academy/glossary/http-caching
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/browser-caching

### Static Site Generation (SSG)
Pre-rendering pages to static HTML at build time. The fastest possible page loads because there's no server-side computation on each request.
Source: https://www.systemdesign.academy/glossary/static-site-generation
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/ssg

### ISR
Incremental Static Regeneration: a Next.js feature that re-generates static pages in the background after a specified time interval, combining SSG speed with fresh data.
Source: https://www.systemdesign.academy/glossary/isr
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/isr

### SSR
Server-Side Rendering: generating HTML on the server for each request. Slower than SSG but always returns fresh data. Good for personalized or frequently changing pages.
Source: https://www.systemdesign.academy/glossary/ssr
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/ssr

### Service Worker
A script the browser runs in the background, separate from the web page. Enables offline caching, push notifications, and background sync for progressive web apps.
Source: https://www.systemdesign.academy/glossary/service-worker
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/service-workers

### Content Negotiation
The process where client and server agree on the response format using HTTP Accept headers. Enables serving JSON, XML, or HTML from the same endpoint.
Source: https://www.systemdesign.academy/glossary/content-negotiation

### ETL
Extract, Transform, Load: a pipeline that extracts data from sources, transforms it into the desired format, and loads it into a destination like a data warehouse.
Source: https://www.systemdesign.academy/glossary/etl
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/etl

### Redis Data Structures
Redis supports strings, lists, sets, sorted sets, hashes, streams, bitmaps, and HyperLogLogs. Each structure solves different problems: sorted sets for leaderboards, streams for event logs.
Source: https://www.systemdesign.academy/glossary/redis-data-structures
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/redis-cache

### Vector Clock
A logical clock that tracks causality across distributed nodes using a vector of counters. Each node increments its own counter and merges vectors on message receipt.
Source: https://www.systemdesign.academy/glossary/vector-clock
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/vector-clocks

### Lamport Timestamp
A simple logical clock where each event increments a counter. If event A causes event B, A's timestamp is always less than B's. The foundation of logical time in distributed systems.
Source: https://www.systemdesign.academy/glossary/lamport-timestamp
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/lamport-timestamps

### Quorum
The minimum number of nodes that must agree for a read or write to succeed. With N replicas, W+R > N ensures overlap between write and read sets for consistency.
Source: https://www.systemdesign.academy/glossary/quorum
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/quorum

### Distributed Lock
A lock that coordinates access to a shared resource across multiple machines. Implemented via Redis (Redlock), ZooKeeper, or etcd. Much harder than local locks.
Source: https://www.systemdesign.academy/glossary/distributed-lock
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/distributed-locks

### Three-Phase Commit
An extension of two-phase commit that adds a pre-commit phase to reduce the blocking window. Still not widely used in practice due to complexity; sagas are preferred.
Source: https://www.systemdesign.academy/glossary/three-phase-commit
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/three-phase-commit

### CRDT
Conflict-free Replicated Data Type: a data structure that can be updated independently on different nodes and merged automatically without conflicts. Powers real-time collaboration like Google Docs.
Source: https://www.systemdesign.academy/glossary/crdt
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/crdts

### Split Brain
A failure scenario where a network partition causes two halves of a cluster to operate independently, each believing it's the leader. Can cause data corruption if not handled.
Source: https://www.systemdesign.academy/glossary/split-brain
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/split-brain

### Clock Skew
The difference in time readings between clocks on different machines. Physical clocks drift; NTP helps but can't guarantee perfect sync. Why distributed systems use logical clocks.
Source: https://www.systemdesign.academy/glossary/clock-skew

### State Machine Replication
Running the same deterministic state machine on multiple nodes, applying the same commands in the same order. If they start in the same state and see the same inputs, they stay in sync.
Source: https://www.systemdesign.academy/glossary/state-machine-replication

### Linearizability
The strongest consistency guarantee: every operation appears to take effect atomically at some point between its invocation and completion. As if there's a single copy of the data.
Source: https://www.systemdesign.academy/glossary/linearizability
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/linearizability

### Serializability
A guarantee that concurrent transactions produce the same result as if they were executed one at a time in some serial order. The gold standard for database transaction isolation.
Source: https://www.systemdesign.academy/glossary/serializability
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/serializability

### Snapshot Isolation
A consistency level where each transaction reads from a consistent snapshot of the database taken at the transaction's start time. Prevents dirty reads without full serializability overhead.
Source: https://www.systemdesign.academy/glossary/snapshot-isolation
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/snapshot-isolation

### Causal Consistency
A consistency model that preserves cause-and-effect ordering: if operation A causally precedes B, all nodes see A before B. Weaker than linearizability but stronger than eventual consistency.
Source: https://www.systemdesign.academy/glossary/causal-consistency
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/causal-consistency

### Read Your Writes
A consistency guarantee that after a user performs a write, their subsequent reads will always reflect that write. Without it, a user might save data and see the old version.
Source: https://www.systemdesign.academy/glossary/read-your-writes
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/read-your-writes

### Monotonic Reads
A guarantee that once you read a value, subsequent reads will never return an older value. Prevents the disorienting experience of data appearing to go backward in time.
Source: https://www.systemdesign.academy/glossary/monotonic-reads
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/monotonic-reads

### Exactly-Once Processing
A processing guarantee where each message is processed exactly one time, even in the face of failures. Achieved through idempotent consumers and transactional producers.
Source: https://www.systemdesign.academy/glossary/exactly-once
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/exactly-once-semantics

### Checkpointing
Periodically saving the state of a stream processing job so it can recover from failures without reprocessing everything from the beginning. Flink and Spark use distributed checkpoints.
Source: https://www.systemdesign.academy/glossary/checkpointing
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/checkpointing-strategies

### Watermark
A timestamp that tracks how far a stream processing system has progressed through event time. Tells the system when it's safe to close a window and emit results, even with late-arriving data.
Source: https://www.systemdesign.academy/glossary/watermark
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/watermarking

### Windowing
Grouping stream events into finite time-based or count-based windows for aggregation. Tumbling windows don't overlap; sliding windows do; session windows group by activity gaps.
Source: https://www.systemdesign.academy/glossary/windowing
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/window-functions

### Apache Flink
A distributed stream processing framework that handles both real-time streams and batch data with exactly-once guarantees. Used by Alibaba, Netflix, and Uber at massive scale.
Source: https://www.systemdesign.academy/glossary/flink

### Apache Spark
A unified analytics engine for large-scale data processing with built-in modules for SQL, streaming, ML, and graph computation. Processes data in-memory for speed.
Source: https://www.systemdesign.academy/glossary/spark

### MapReduce
A programming model for processing massive datasets in parallel across a cluster. Map splits data into key-value pairs; Reduce aggregates them. Pioneered by Google, implemented by Hadoop.
Source: https://www.systemdesign.academy/glossary/mapreduce
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/mapreduce

### Data Lake
A centralized repository that stores raw data at any scale in its native format. Unlike a data warehouse, data doesn't need to be structured or cleaned before loading.
Source: https://www.systemdesign.academy/glossary/data-lake
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/data-lake

### Data Warehouse
A central repository of structured, cleaned data optimized for analytical queries. Snowflake, BigQuery, and Redshift are purpose-built data warehouses.
Source: https://www.systemdesign.academy/glossary/data-warehouse
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-lakes-warehouses-lakehouses

### Data Mesh
A decentralized data architecture where each domain team owns and publishes its data as a product. Shifts responsibility from a central data team to domain teams.
Source: https://www.systemdesign.academy/glossary/data-mesh
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/data-mesh

### Change Data Capture
Capturing row-level changes in a database and streaming them to other systems in real time. Debezium reads the write-ahead log and publishes changes to Kafka.
Source: https://www.systemdesign.academy/glossary/change-data-capture
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/change-data-capture

### Penetration Testing
Authorized simulated attacks against a system to find security vulnerabilities before real attackers do. White-hat hackers probe for weaknesses in a controlled environment.
Source: https://www.systemdesign.academy/glossary/penetration-testing
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/penetration-testing

### Threat Modeling
A structured process for identifying security threats, attack surfaces, and mitigations during system design. STRIDE and DREAD are common frameworks.
Source: https://www.systemdesign.academy/glossary/threat-modeling
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/threat-modeling

### Incident Response
The structured process for detecting, containing, eradicating, and recovering from security incidents. Includes communication plans, runbooks, and post-incident reviews.
Source: https://www.systemdesign.academy/glossary/incident-response
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/incident-response

### Security Audit
A systematic evaluation of a system's security posture against standards and best practices. Covers access controls, encryption, logging, vulnerability management, and compliance.
Source: https://www.systemdesign.academy/glossary/security-audit
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/security-audit-logging

### SLI
Service Level Indicator: a quantitative measure of service behavior, like the proportion of requests faster than 300ms. The raw metric that feeds SLOs.
Source: https://www.systemdesign.academy/glossary/sli
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/sli

### SLO
Service Level Objective: a target value for an SLI, like '99.9% of requests under 300ms.' The internal engineering goal that drives reliability investment.
Source: https://www.systemdesign.academy/glossary/slo
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/slo

### SLA
Service Level Agreement: a contractual commitment between provider and customer specifying uptime, response time, and penalties for breaches. The business version of an SLO.
Source: https://www.systemdesign.academy/glossary/sla
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/sla

### Error Budget
The allowed amount of unreliability derived from SLOs. If your SLO is 99.9% uptime, your error budget is 0.1% (about 43 minutes/month). Once exhausted, freeze deployments.
Source: https://www.systemdesign.academy/glossary/error-budget
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/error-budgets

### MTTR
Mean Time To Recovery: the average time from when a failure is detected to when the service is restored. A key reliability metric that drives investment in automation and runbooks.
Source: https://www.systemdesign.academy/glossary/mttr
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/mttr

### MTTD
Mean Time To Detect: the average time between a failure occurring and being noticed. Shorter MTTD means better monitoring and alerting. You can't fix what you don't know is broken.
Source: https://www.systemdesign.academy/glossary/mttd
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/mttd

### Flame Graph
A visualization of profiled software showing which functions consume the most CPU or wall-clock time. The x-axis is the stack population and the y-axis is call stack depth.
Source: https://www.systemdesign.academy/glossary/flame-graph

### Postmortem
A blameless analysis conducted after an incident to document what happened, why, and how to prevent it from recurring. The most important output is the list of action items.
Source: https://www.systemdesign.academy/glossary/postmortem
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/post-mortem-analysis

### Graceful Degradation
A strategy where a system continues to function with reduced capability when a component fails, instead of crashing entirely. Show cached results when the database is down.
Source: https://www.systemdesign.academy/glossary/graceful-degradation
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/graceful-degradation

### Load Shedding
Deliberately dropping low-priority requests during overload to protect the system's ability to serve high-priority traffic. Better to serve some requests than crash serving none.
Source: https://www.systemdesign.academy/glossary/load-shedding
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/load-shedding

### Game Day
A planned exercise where teams simulate production failures to test incident response procedures and system resilience. Like a fire drill for your infrastructure.
Source: https://www.systemdesign.academy/glossary/game-day
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/game-days

### Disaster Recovery
A plan and set of procedures for restoring systems after a catastrophic failure. Defined by RPO (how much data you can lose) and RTO (how long you can be down).
Source: https://www.systemdesign.academy/glossary/disaster-recovery
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/disaster-recovery

### RPO
Recovery Point Objective: the maximum acceptable amount of data loss measured in time. An RPO of 1 hour means you can afford to lose up to 1 hour of data.
Source: https://www.systemdesign.academy/glossary/rpo
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/rpo

### RTO
Recovery Time Objective: the maximum acceptable downtime after a disaster. An RTO of 15 minutes means the system must be back online within 15 minutes of failure.
Source: https://www.systemdesign.academy/glossary/rto
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/rto

### Capacity Planning
Estimating future resource needs based on current usage trends and expected growth. Answers questions like 'how many servers do we need for 10x traffic in 6 months?'
Source: https://www.systemdesign.academy/glossary/capacity-planning
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/capacity-planning

### SRE
Google's discipline for running reliable production systems. Applies software engineering to operations: automation over toil, SLOs over uptime promises, and error budgets for velocity.
Source: https://www.systemdesign.academy/glossary/site-reliability-engineering

### Toil
Manual, repetitive, automatable operational work that scales linearly with service size. SRE teams aim to keep toil below 50% of their time and automate the rest.
Source: https://www.systemdesign.academy/glossary/toil
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/toil-reduction

### Runbook
A documented set of step-by-step procedures for handling specific operational tasks or incidents. Good runbooks reduce MTTR by giving on-call engineers a clear action plan.
Source: https://www.systemdesign.academy/glossary/runbook
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/runbook-automation

### Canary Analysis
Automated statistical comparison of metrics between the canary (new version) and the baseline (current version) to decide whether to promote or roll back a deployment.
Source: https://www.systemdesign.academy/glossary/canary-analysis
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/canary-deployments

### Failure Injection
Deliberately introducing faults (latency, errors, crashes) into a system to verify that resilience mechanisms work. A specific technique within chaos engineering.
Source: https://www.systemdesign.academy/glossary/failure-injection
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/chaos-engineering

### Client-Server Model
The foundational architecture of the web: clients (browsers, apps) send requests and servers process them and return responses. Every web interaction follows this pattern.
Source: https://www.systemdesign.academy/glossary/client-server

### Peer-to-Peer
A decentralized architecture where each node acts as both client and server. No central authority. Used by BitTorrent, blockchain, and WebRTC.
Source: https://www.systemdesign.academy/glossary/peer-to-peer

### Serialization
Converting an in-memory data structure into a byte stream for storage or network transmission. Deserialization is the reverse. JSON, Protobuf, and Avro are serialization formats.
Source: https://www.systemdesign.academy/glossary/serialization

### Network Partition
A break in communication between nodes in a distributed system. Some nodes can't reach others. The 'P' in CAP theorem that forces the trade-off between consistency and availability.
Source: https://www.systemdesign.academy/glossary/network-partition
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/network-partitioning

### Connection Draining
Gracefully finishing in-flight requests before removing a server from a load balancer's pool. Prevents dropping active connections during deployments or scale-ins.
Source: https://www.systemdesign.academy/glossary/connection-draining
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/connection-draining

### Saga Orchestration
A saga implementation where a central orchestrator tells each service what step to execute next. Easier to debug than choreography but creates a single point of coordination.
Source: https://www.systemdesign.academy/glossary/saga-orchestration
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/saga-pattern

### Saga Choreography
A saga implementation where each service listens for events and decides what to do next independently. No central coordinator. Looser coupling but harder to trace.
Source: https://www.systemdesign.academy/glossary/saga-choreography
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/saga-pattern

### Circuit Breaker States
A circuit breaker cycles through three states: Closed (requests flow normally), Open (requests are blocked after failures), Half-Open (a few test requests check if the service recovered).
Source: https://www.systemdesign.academy/glossary/circuit-breaker-states
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/circuit-breaker-pattern

### Exponential Backoff
A retry strategy that doubles the wait time between attempts (1s, 2s, 4s, 8s...) with random jitter. Prevents thundering herd problems when many clients retry simultaneously.
Source: https://www.systemdesign.academy/glossary/exponential-backoff
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/exponential-backoff

### Token Bucket
A rate limiting algorithm where tokens are added to a bucket at a fixed rate. Each request consumes a token; requests are rejected when the bucket is empty. Allows short bursts.
Source: https://www.systemdesign.academy/glossary/token-bucket
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/token-bucket-algorithm

### Sliding Window
A rate limiting algorithm that tracks requests in a rolling time window. More accurate than fixed windows because it smooths out spikes at window boundaries.
Source: https://www.systemdesign.academy/glossary/sliding-window
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/sliding-window

### Content Security Policy
An HTTP response header that restricts which resources (scripts, styles, images) a page can load. A powerful defense against XSS attacks by whitelisting trusted sources.
Source: https://www.systemdesign.academy/glossary/content-security-policy

### OpenTelemetry
A vendor-neutral open standard for collecting metrics, logs, and traces from applications. Provides SDKs and a collector that ships telemetry to any observability backend.
Source: https://www.systemdesign.academy/glossary/opentelemetry
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/opentelemetry

### Prometheus
An open-source monitoring system that scrapes metrics endpoints, stores time-series data, and supports powerful PromQL queries. The de facto standard for Kubernetes monitoring.
Source: https://www.systemdesign.academy/glossary/prometheus
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/prometheus

### Grafana
A visualization platform for creating dashboards from any data source. Connects to Prometheus, Elasticsearch, CloudWatch, and dozens more to display metrics, logs, and traces.
Source: https://www.systemdesign.academy/glossary/grafana
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/grafana

### Container Image
A lightweight, immutable package containing everything needed to run an application: code, runtime, libraries, and config. Built with a Dockerfile and stored in a registry.
Source: https://www.systemdesign.academy/glossary/container-image

### Kubernetes ConfigMap
A K8s resource that stores non-sensitive configuration data as key-value pairs. Pods read ConfigMaps as environment variables or mounted files, decoupling config from code.
Source: https://www.systemdesign.academy/glossary/k8s-configmap

### Kubernetes Secret
A K8s resource for storing sensitive data like passwords, tokens, and certificates. Base64-encoded by default; use encryption at rest and RBAC to properly secure them.
Source: https://www.systemdesign.academy/glossary/k8s-secret
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/secrets-management

### Horizontal Pod Autoscaler
A K8s controller that automatically scales the number of pods based on CPU utilization, memory usage, or custom metrics. The Kubernetes equivalent of auto-scaling groups.
Source: https://www.systemdesign.academy/glossary/horizontal-pod-autoscaler
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/auto-scaling

### VPC
Virtual Private Cloud: a logically isolated section of the cloud where you launch resources in a virtual network you define. Controls IP ranges, subnets, route tables, and gateways.
Source: https://www.systemdesign.academy/glossary/vpc
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/vpc

### NAT Gateway
Network Address Translation gateway: allows instances in a private subnet to access the internet while preventing inbound connections. Keeps your backend servers invisible to the public.
Source: https://www.systemdesign.academy/glossary/nat-gateway
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/nat-gateway

### Load Balancer Algorithm
The strategy a load balancer uses to distribute requests. Round-robin, least-connections, weighted, IP-hash, and random are common algorithms, each with different trade-offs.
Source: https://www.systemdesign.academy/glossary/load-balancer-algorithm
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/load-balancer-algorithms

### Global Load Balancer
A load balancer that routes users to the nearest data center or region based on geography, latency, or health. DNS-based (Route 53, Cloudflare) or anycast-based.
Source: https://www.systemdesign.academy/glossary/global-load-balancer
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/global-server-load-balancing

### Event Loop
A programming pattern that waits for and dispatches events in a single thread. Node.js uses it to handle thousands of concurrent connections without creating a thread per connection.
Source: https://www.systemdesign.academy/glossary/event-loop

### Request-Response
Request-response is the most basic way two programs talk. One side sends a request, then waits for exactly one reply before it moves on. HTTP, REST and simple gRPC calls all work this way.
Source: https://www.systemdesign.academy/glossary/request-response
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/request-response-pattern

### Fire and Forget
A communication pattern where the sender dispatches a message and doesn't wait for or expect a response. Used for logging, analytics events, and non-critical notifications.
Source: https://www.systemdesign.academy/glossary/fire-and-forget
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/fire-and-forget-pattern

### Read Repair
A technique where a read operation detects stale data on a replica and triggers a background update to bring it in sync. Used by Cassandra and DynamoDB to heal inconsistencies lazily.
Source: https://www.systemdesign.academy/glossary/read-repair
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/read-repair

### Anti-Entropy
A background process that compares data between replicas and fixes differences. Uses Merkle trees to efficiently identify which data ranges are out of sync.
Source: https://www.systemdesign.academy/glossary/anti-entropy
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/anti-entropy

### Hinted Handoff
When a replica is temporarily down, another node stores writes intended for it as 'hints.' Once the replica recovers, the hints are replayed to catch it up.
Source: https://www.systemdesign.academy/glossary/hinted-handoff
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/hinted-handoff

### Tombstone
A marker that indicates a deleted record in a distributed database. Since replicas may not see the delete immediately, the tombstone prevents deleted data from being resurrected.
Source: https://www.systemdesign.academy/glossary/tombstone

### Backfill
Retroactively populating a new data store, index, or column with historical data. Typically done as a batch job when adding a new feature that needs past data.
Source: https://www.systemdesign.academy/glossary/backfill

### Schema Registry
A centralized service that stores and validates schemas for event-driven systems. Confluent Schema Registry ensures Kafka producers and consumers agree on data formats.
Source: https://www.systemdesign.academy/glossary/schema-registry
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/schema-registry

### Idempotent Consumer
A message consumer that produces the same result whether it processes a message once or multiple times. Achieved by tracking processed message IDs or using natural idempotency.
Source: https://www.systemdesign.academy/glossary/idempotent-consumer
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/idempotent-consumer

### Outbox Pattern
A pattern where a service writes events to an outbox table in the same database transaction as its state change. A separate process reads the outbox and publishes events, ensuring atomicity.
Source: https://www.systemdesign.academy/glossary/outbox-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/transactional-outbox-pattern

### Shard Key
The column or field used to determine which shard a piece of data belongs to. A bad shard key creates hot spots; a good one distributes data and queries evenly.
Source: https://www.systemdesign.academy/glossary/shard-key
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-sharding

### Hash Index
An index that uses a hash function to map keys directly to storage locations. O(1) lookups for exact matches but useless for range queries. Memcached and Redis use hash indexes.
Source: https://www.systemdesign.academy/glossary/hash-index

### Database Trigger
A piece of SQL that automatically executes in response to INSERT, UPDATE, or DELETE events on a table. Useful for audit logs and enforcing complex constraints.
Source: https://www.systemdesign.academy/glossary/database-trigger
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-triggers

### SSL Termination
Decrypting TLS-encrypted traffic at a load balancer or reverse proxy so backend servers receive plain HTTP. Offloads CPU-intensive crypto work and simplifies certificate management.
Source: https://www.systemdesign.academy/glossary/ssl-termination

### GeoDNS
DNS that returns different IP addresses based on the geographic location of the requester. Routes users to the nearest data center for lower latency.
Source: https://www.systemdesign.academy/glossary/geo-dns
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/geolocation-routing

### Multi-Tenancy
A single software instance serving multiple customers (tenants) while keeping their data isolated. Can be achieved at the database, schema, or row level.
Source: https://www.systemdesign.academy/glossary/multi-tenancy

### Twelve-Factor App
A methodology for building cloud-native applications with 12 principles: codebase, dependencies, config, backing services, build/release/run, processes, port binding, concurrency, disposability, dev/prod parity, logs, admin processes.
Source: https://www.systemdesign.academy/glossary/twelve-factor-app

### Tail Latency
The high-percentile response times (p99, p99.9) that affect the slowest requests. A system with 10ms median but 2s p99 latency feels slow for 1 in 100 users.
Source: https://www.systemdesign.academy/glossary/tail-latency
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/percentiles

### Thundering Herd
When many clients simultaneously retry or reconnect after a failure, overwhelming the recovering system. Solved by jittered backoff, request coalescing, and admission control.
Source: https://www.systemdesign.academy/glossary/thundering-herd
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-stampede-prevention

### Hot Spot
An uneven distribution of load where one node, shard, or partition receives disproportionately more traffic than others. Caused by poor shard keys or skewed access patterns.
Source: https://www.systemdesign.academy/glossary/hot-spot

### RAG (Retrieval-Augmented Generation)
A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.
Source: https://www.systemdesign.academy/glossary/rag
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-scale-cliff

### Agentic RAG
Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.
Source: https://www.systemdesign.academy/glossary/agentic-rag
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/query-transformation-and-routing

### Embedding
A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.
Source: https://www.systemdesign.academy/glossary/embedding
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/what-an-embedding-is

### ANN (Approximate Nearest Neighbor)
Search that finds vectors close to a query without scanning every item, trading a small chance of missing a true neighbor for a large gain in speed. HNSW and IVF are the common indexes.
Source: https://www.systemdesign.academy/glossary/ann
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/finding-neighbours-fast

### HNSW
Hierarchical Navigable Small World, a graph-based nearest neighbor index that gives high recall at low latency by navigating layers of links, at the cost of more memory than IVF.
Source: https://www.systemdesign.academy/glossary/hnsw
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/vector-databases-ann

### Reranking
A second retrieval stage where a cross-encoder scores each candidate chunk against the query directly. Slower than the first-stage retriever but far more precise, so it runs only over the top few dozen results.
Source: https://www.systemdesign.academy/glossary/reranking
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/reranking

### Hybrid Retrieval
Running dense vector search and sparse keyword search together and fusing the results, so exact terms like an error code and paraphrased meaning both count.
Source: https://www.systemdesign.academy/glossary/hybrid-retrieval
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/keyword-and-meaning

### Hallucination
When a language model produces fluent, confident text that is not supported by any source and is often simply false. Retrieval, grounding, and evaluation are the main defenses.
Source: https://www.systemdesign.academy/glossary/hallucination
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/evaluating-rag-and-agents

### Fine-Tuning
Continuing to train a pretrained model on your own data to change its behavior, style, or format. Best for teaching a skill, not for injecting facts, which is what retrieval is for.
Source: https://www.systemdesign.academy/glossary/fine-tuning
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/fine-tuning-vs-rag-vs-prompting

### LoRA (Low-Rank Adaptation)
A parameter-efficient fine-tuning method that freezes the base weights and trains a small pair of low-rank matrices instead, cutting memory and cost while keeping most of the quality.
Source: https://www.systemdesign.academy/glossary/lora
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/parameter-efficient-fine-tuning

### PEFT (Parameter-Efficient Fine-Tuning)
A family of methods, including LoRA and QLoRA, that adapt a large model by training a tiny fraction of its parameters, so tuning fits on a single GPU instead of a cluster.
Source: https://www.systemdesign.academy/glossary/peft
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/parameter-efficient-fine-tuning

### Quantization
Storing model weights in fewer bits, such as FP8 or INT8, to shrink memory and speed up inference. Modern methods lose little quality on most tasks.
Source: https://www.systemdesign.academy/glossary/quantization
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/quantization

### KV Cache
The stored keys and values of every token processed so far, kept so attention does not recompute the whole history each step. It grows with every token and is what limits how many conversations a GPU can serve.
Source: https://www.systemdesign.academy/glossary/kv-cache
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/what-a-long-prompt-costs

### Continuous Batching
Rebuilding the inference batch after every token so a finished sequence leaves and a waiting request joins immediately, instead of every request waiting for the slowest one. It is what keeps a serving GPU busy.
Source: https://www.systemdesign.academy/glossary/continuous-batching
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-inference-optimization

### Inference Optimization
The engineering that serves more tokens per GPU-second: KV caching, continuous batching, quantization, prefix caching, and speculative decoding. Serving cost is dominated by how well these are done.
Source: https://www.systemdesign.academy/glossary/inference-optimization
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-inference-optimization

### Prompt Engineering
Writing and structuring the instructions given to a language model to get reliable output. In 2026 the craft has moved from clever wording to schemas, context layout, and evaluation.
Source: https://www.systemdesign.academy/glossary/prompt-engineering
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/parts-of-a-prompt

### Context Engineering
Deciding what goes into a model's limited context window and in what order. Production data shows quality degrades well before the advertised token limit, so what you leave out matters as much as what you put in.
Source: https://www.systemdesign.academy/glossary/context-engineering
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/agent-memory-context

### AI Agent
A language model placed in a loop: it decides an action, calls a tool, reads the result, and repeats until done. The hard part is that per-step errors compound over many steps.
Source: https://www.systemdesign.academy/glossary/agent
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/agent-loop-reliability

### Tool Calling
When a model emits a structured request to run a function, and an orchestrator executes it and feeds the result back. The model never runs anything itself, so the orchestrator validates every call.
Source: https://www.systemdesign.academy/glossary/tool-calling
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/tool-calling-contract

### MCP (Model Context Protocol)
An open protocol that lets AI apps such as Claude, ChatGPT, Cursor and VS Code use outside tools and data. An MCP server offers the tools once, and any app with an MCP client can find and call them.
Source: https://www.systemdesign.academy/glossary/mcp
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/mcp-in-production

### Prompt Injection
An attack where untrusted text, often returned by a tool, carries an instruction the model obeys, because the model cannot reliably tell instructions from data. The top security risk for agents, and unsolved by prompting alone.
Source: https://www.systemdesign.academy/glossary/prompt-injection
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/security/prompt-injection-lethal-trifecta

### Lethal Trifecta
The dangerous combination of access to private data, exposure to untrusted content, and a way to exfiltrate. An agent with all three can be turned against its own user by a prompt injection.
Source: https://www.systemdesign.academy/glossary/lethal-trifecta
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/security/the-lethal-trifecta

### Guardrails
The checks around a model that screen its input for abuse and its output for grounding and safety, plus limits on what its tools can do. Containment, not just filtering, is what actually holds.
Source: https://www.systemdesign.academy/glossary/guardrails
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-guardrails-safety

### LLM-as-Judge
Using a language model to score another model's output. Fast and scalable, but it has real failure modes such as self-preference and position bias, so the judge itself must be validated against human labels.
Source: https://www.systemdesign.academy/glossary/llm-as-judge
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/llm-as-judge-failure-modes

### LLM Evaluation
Measuring whether a model or pipeline actually works, using error analysis, labeled datasets, and validated judges rather than a glance. The single most in-demand and least-taught AI engineering skill.
Source: https://www.systemdesign.academy/glossary/evaluation
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/why-evals-are-the-job

### Error Analysis
Reading real failure traces and coding them into categories to find where a system actually breaks, before building any metric or dashboard. Most of eval effort should go here, not into tooling.
Source: https://www.systemdesign.academy/glossary/error-analysis
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/error-analysis-open-coding

### Golden Dataset
A curated set of inputs with known-correct expected outputs, used to measure quality and catch regressions when you change a prompt, a model, or the retrieval stack.
Source: https://www.systemdesign.academy/glossary/golden-dataset
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/building-golden-datasets

### Feature Store
A single source of truth for features, so training and serving compute them from one definition and cannot drift apart. It ends the train-serve skew bug and lets teams reuse features across models.
Source: https://www.systemdesign.academy/glossary/feature-store
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/core/feature-stores

### Train-Serve Skew
When a feature is computed one way in training and a slightly different way in production, so every prediction is subtly wrong. Nothing crashes, accuracy just leaks away. Feature stores exist to prevent it.
Source: https://www.systemdesign.academy/glossary/train-serve-skew
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/core/feature-stores

### Model Drift
When a model's accuracy decays over time because the live data has moved away from what it was trained on. Detected by monitoring inputs and outputs, and fixed by retraining.
Source: https://www.systemdesign.academy/glossary/model-drift
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/drift-alarm-vs-harm

### Model Registry
A versioned catalog of trained models with their metadata, stage, and lineage, so you know exactly what is deployed and can roll back. The control plane for shipping models safely.
Source: https://www.systemdesign.academy/glossary/model-registry
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/core/model-registry-versioning

### Model Serving
Exposing a trained model behind an API that answers predictions within a latency budget, with batching, autoscaling, and versioning. Where most of the production cost and the request-path complexity lives.
Source: https://www.systemdesign.academy/glossary/model-serving
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/core/model-serving-inference

### Structured Output
Constraining a model to return valid JSON matching a schema, so downstream code can rely on it. The foundation of reliable tool calling.
Source: https://www.systemdesign.academy/glossary/structured-output
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/asking-for-a-format

### Cosine Similarity
A measure of how close two embeddings are, based on the angle between them rather than their length. The standard way to rank how related two pieces of text are in vector search.
Source: https://www.systemdesign.academy/glossary/cosine-similarity
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/cosine-dot-and-distance

### Token
The unit a language model reads and writes, roughly a few characters. Cost and context limits are counted in tokens, not words, so token accounting is how you budget an LLM system.
Source: https://www.systemdesign.academy/glossary/token
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/what-a-token-is

### Grounding
Requiring a model to answer from provided evidence and to cite it, rather than from its own memory. The core defense against hallucination in a RAG system.
Source: https://www.systemdesign.academy/glossary/grounding
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-evaluation-and-failure-analysis

### Feature Engineering
Turning raw data into the signals a model actually learns from. Often the highest-leverage work in a classical ML system, and where a feature store keeps definitions consistent.
Source: https://www.systemdesign.academy/glossary/feature-engineering
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/leakage-before-the-split

### Read-Through Cache
A cache that loads missing data from the database by itself, through one loader function, so your code only ever asks the cache. What the loader does when the database fails decides whether an outage shows up as errors or as wrong answers.
Source: https://www.systemdesign.academy/glossary/read-through-cache
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/read-through-cache

### LRU Cache
A cache that, when full, throws away the item nobody has used for the longest time. It keeps recently used data on the bet that it will be used again soon.
Source: https://www.systemdesign.academy/glossary/lru-cache
Taught in: https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-eviction-policies

### Agentic Design Patterns
Reusable ways to structure an AI agent's work, such as ReAct, plan-and-execute, reflection and multi-agent setups. Each trades extra model calls for a chance at a better answer.
Source: https://www.systemdesign.academy/glossary/agentic-design-patterns
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/reflection-and-decomposition

### Semantic Search
Search that matches meaning instead of exact words. Text is turned into embeddings, and the results are the documents whose embeddings sit closest to the query's.
Source: https://www.systemdesign.academy/glossary/semantic-search
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/searching-by-meaning

### Data Lakehouse
A design that keeps data as open files, usually Parquet, in cheap object storage and adds a log on top that makes changes all-or-nothing, checks column types and keeps old versions readable.
Source: https://www.systemdesign.academy/glossary/data-lakehouse
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-lakes-warehouses-lakehouses

### LLM Temperature
A setting that controls how random a language model's word choice is. Low temperature picks the most likely next token almost every time; high temperature spreads the choice wider.
Source: https://www.systemdesign.academy/glossary/llm-temperature
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/temperature-and-sampling

### AI Red Teaming
Attacking your own AI system on purpose, the way a real attacker would, to find what it can be tricked into doing before someone else does.
Source: https://www.systemdesign.academy/glossary/ai-red-teaming
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/security/owasp-agentic-top-10-red-teaming

### Byte-Pair Encoding (BPE)
The rule a tokenizer uses to learn its pieces: starting from single characters or bytes, it repeatedly merges the most frequent pair of neighbouring pieces into a new token.
Source: https://www.systemdesign.academy/glossary/byte-pair-encoding
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/how-a-tokenizer-learns

### LLM Router
A component that decides which language model should answer each request, so easy requests go to a cheap model and hard ones to a strong model.
Source: https://www.systemdesign.academy/glossary/llm-router
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/model-routing-and-cascades

### Context Rot
The drop in a language model's accuracy as its input gets longer, even when the answer is in the text and well inside the context window.
Source: https://www.systemdesign.academy/glossary/context-rot
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/context-rot-measuring-degradation

### LLM Observability
Recording what an LLM application actually did on each request, the prompts, model calls, tool calls, tokens, cost and latency, so you can explain and fix a bad answer.
Source: https://www.systemdesign.academy/glossary/llm-observability
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-observability-and-tracing

### PagedAttention
A way of storing a language model's KV cache in small fixed-size blocks instead of one big reserved slab, so GPU memory is not wasted and more requests fit at once.
Source: https://www.systemdesign.academy/glossary/paged-attention
Taught in: https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-inference-optimization

### Chunking Strategies
The rules for splitting documents into pieces before they are embedded for search. Chunk size and boundaries decide what retrieval can find.
Source: https://www.systemdesign.academy/glossary/chunking-strategies
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/chunking-strategies

### Multi-Agent System
A design where more than one LLM-driven loop works on one task, each with its own context, usually coordinated by an orchestrator that hands out parts and merges the results. It adds token cost and handoffs that can fail.
Source: https://www.systemdesign.academy/glossary/multi-agent-system
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/multi-agent-when-not-to

### LLM Caching
Reusing earlier work to avoid paying for the same model call twice: exact caches for identical requests, semantic caches for similar ones, and prompt caching on the provider's side.
Source: https://www.systemdesign.academy/glossary/llm-caching
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-caching-and-cost

### Data Labeling
Attaching the correct answer to each example, such as a category or a yes or no, so a model can learn from it or be tested against it.
Source: https://www.systemdesign.academy/glossary/data-labeling
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-labeling

### RAG Evaluation
Measuring a retrieval-augmented generation system in two parts, whether retrieval found the right text and whether the answer used it, so a failure can be traced to one of them.
Source: https://www.systemdesign.academy/glossary/rag-evaluation
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-evaluation-and-failure-analysis

### Context Window
The maximum number of tokens a language model can take in at once, counting the prompt, any documents and its own reply.
Source: https://www.systemdesign.academy/glossary/context-window
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/the-context-window

### Matryoshka Embeddings
Embeddings trained so their first numbers carry the most meaning, which lets you cut them shorter to save storage and search time while losing little accuracy.
Source: https://www.systemdesign.academy/glossary/matryoshka-embeddings
Taught in: https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/shorter-embeddings

### ReAct Agent
An AI agent that alternates reasoning and acting: it writes a thought, calls a tool, reads the result, and repeats until it can answer. ReAct is short for reason plus act.
Source: https://www.systemdesign.academy/glossary/react-agent
Taught in: https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/planning-patterns-react-plan-execute

### Design Patterns
Named, reusable solutions to problems that keep coming up in object-oriented code, such as creating objects, wrapping interfaces or reacting to events. The classic catalogue of 23 was published by the Gang of Four in 1994.
Source: https://www.systemdesign.academy/glossary/design-patterns
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/singleton-pattern

### Strategy Pattern
A behavioral design pattern that puts each version of an algorithm in its own class behind one interface, so code can swap the algorithm at runtime instead of growing an if-else chain.
Source: https://www.systemdesign.academy/glossary/strategy-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/strategy-pattern

### Facade Pattern
A structural design pattern that puts one simple interface in front of a complicated set of classes or services, so callers make one call instead of coordinating many.
Source: https://www.systemdesign.academy/glossary/facade-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/facade-pattern

### Factory Pattern
A creational design pattern where one method or class decides which concrete object to build, so the calling code asks for an object by type or config instead of calling a constructor directly.
Source: https://www.systemdesign.academy/glossary/factory-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/factory-pattern

### Decorator Pattern
A structural design pattern that wraps an object in another object with the same interface to add behavior, such as logging, caching or retries, without changing the original class.
Source: https://www.systemdesign.academy/glossary/decorator-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/decorator-pattern

### Adapter Pattern
A structural design pattern that wraps a class with an incompatible interface so it matches the interface your code expects, like a travel plug adapter for software.
Source: https://www.systemdesign.academy/glossary/adapter-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/adapter-pattern

### Observer Pattern
A behavioral design pattern where an object keeps a list of subscribers and notifies each one when its state changes, so the object does not need to know what the subscribers do.
Source: https://www.systemdesign.academy/glossary/observer-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/observer-pattern

### Command Pattern
A behavioral design pattern that turns a request into an object holding everything needed to run it, so requests can be queued, logged, retried or undone.
Source: https://www.systemdesign.academy/glossary/command-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/command-pattern

### Singleton Pattern
A creational design pattern that makes sure a class has exactly one instance and gives the whole program one shared way to reach it, used for things like a connection pool or a config object.
Source: https://www.systemdesign.academy/glossary/singleton-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/singleton-pattern

### State Pattern
A behavioral design pattern where an object delegates its behavior to a separate object for its current state and swaps that object when the state changes, a clean way to build a finite state machine.
Source: https://www.systemdesign.academy/glossary/state-pattern
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/finite-state-machine

### Proxy Pattern
A structural design pattern that puts a stand-in object with the same interface in front of a real object to control access to it, for example to cache, check permissions or load it lazily.
Source: https://www.systemdesign.academy/glossary/proxy-pattern
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/proxy-pattern

### Creational Design Patterns
The family of design patterns about how objects get created: Singleton, Factory Method, Abstract Factory, Builder and Prototype. They hide the construction details from the code that uses the object.
Source: https://www.systemdesign.academy/glossary/creational-design-patterns
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/factory-pattern

### Chain of Responsibility
A behavioral design pattern that passes a request along a chain of handlers, where each handler either deals with it or passes it to the next one. HTTP middleware is the everyday example.
Source: https://www.systemdesign.academy/glossary/chain-of-responsibility
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/chain-of-responsibility

### Distributed Transactions
A transaction whose changes span more than one database or service, which must either all commit or all be undone. Two-phase commit and sagas are the two main ways to get there.
Source: https://www.systemdesign.academy/glossary/distributed-transactions
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/distributed-transactions

### Domain-Driven Design (DDD)
A way of designing software around the business domain: the code uses the same words as the business, and the system is split into bounded contexts, each with its own model.
Source: https://www.systemdesign.academy/glossary/domain-driven-design
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/domain-driven-design

### Microservices Patterns
The recurring designs teams use to build and run microservices, such as API gateway, service discovery, circuit breaker, database per service, saga and event-driven messaging.
Source: https://www.systemdesign.academy/glossary/microservices-patterns
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/microservices-architecture-deep

### Structural Design Patterns
The family of design patterns about how classes and objects fit together: Adapter, Bridge, Composite, Decorator, Facade, Flyweight and Proxy. They change how pieces connect without changing what each piece does.
Source: https://www.systemdesign.academy/glossary/structural-design-patterns
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/proxy-pattern

### Integration Design Patterns
Reusable designs for connecting separate systems through messages, such as the content-based router, splitter, aggregator and normalizer, catalogued as Enterprise Integration Patterns.
Source: https://www.systemdesign.academy/glossary/integration-design-patterns
Taught in: https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/normalizer-pattern

### Lambda Architecture
A data processing design that runs a batch layer for complete, accurate results and a speed layer for fresh, approximate results, then merges the two when a query is answered.
Source: https://www.systemdesign.academy/glossary/lambda-architecture
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/lambda-architecture

### Distributed Systems Algorithms
The algorithms machines use to agree and coordinate over an unreliable network, such as leader election, consensus with Raft or Paxos, logical clocks, gossip and two-phase commit.
Source: https://www.systemdesign.academy/glossary/distributed-systems-algorithms
Taught in: https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/leader-election

## Every lesson, by course

### AI Engineering course

#### LLM Evaluation and Error Analysis (ml-advanced)
- [Why Evals Are the Job: Moving From Vibes to Measurement](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/why-evals-are-the-job) [FREE, no signup]: Why evals, not observability, decide if your LLM feature is good, and why measuring quality is the core skill of AI engineering in 2026.
- [Error Analysis: Reading Failures Before Building Metrics](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/error-analysis-open-coding): How to read real LLM failures before you build metrics. Open coding, axial coding and saturation turn notes into a ranked list of failure types.
- [Building a Golden Dataset: The Set That Catches Regressions](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/building-golden-datasets): A golden dataset is a curated set of inputs and known-good answers you run before and after every change, so a prompt tweak that quietly breaks things cannot ship.
- [LLM-as-Judge and Its Failure Modes](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/llm-as-judge-failure-modes): The full catalogue of ways an LLM judge fails, the mechanism behind each one, the fix that removes it, and how to validate a judge against human labels before you trust its scores.
- [Validating Your Judge: TPR, TNR, and the Human Ceiling](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/validating-your-judge): An LLM judge is worthless until you know how well it agrees with a human expert. Validate it like a classifier: TPR, TNR, kappa, and the human ceiling.
- [Evaluating RAG and Agents: One Score Detects, Components Localize](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/evaluating-rag-and-agents): One end-to-end score tells you something broke, not what. Split RAG and agent quality into retrieval, generation, tool, and trajectory evals that localize failures and score the path, not just the answer.
- [Offline vs Online: Evals Verify, Experiments Validate](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/offline-vs-online-evals-vs-experiments): Offline evals verify a change did not regress. Only an online A/B test validates that it moved a real metric. Why you need both and how they split the work.
- [Prompt CI and Benchmark Integrity: Catch the Silent Break](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/benchmark-integrity-ci-for-prompts): A new model version can break your prompts with zero errors. Run the eval set in CI, gate on per-facet pass rates, and the silent break becomes a red build.
- [The Error Hierarchy: Which Failure to Fix First](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/the-error-hierarchy): A measured lab where the score moved two points and the diagnosis flipped completely. Ten wrong answers before, ten after, and opposite work to do. The order you fix things in decides whether a sprint lands on the broken stage or the working one.
- [Which Traces to Read: Sampling Decides What You Can Discover](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/which-traces-to-read): You will read a few dozen traces out of a hundred thousand. Measured on the lab: the same 40 traces report a failure rate of 25 percent, 100 percent or 86 percent, and the only thing that changed was the rule that picked them.
- [Open Coding: Writing Down What Went Wrong, Before You Have Categories](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/open-coding-your-failures): The messy first pass where you describe failures in your own words. One grader wrote thirteen different codes across ten failed traces, and seven of them mean the same thing. That mess is the point.
- [Axial Coding: Turning Thirteen Words Into Four Categories](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/axial-coding-building-the-taxonomy): Merging near-duplicate codes is not tidying. On the lab's real data it changed which failure was the most common, because the biggest category had been written seven different ways.
- [Theoretical Saturation: How Many Traces Is Enough?](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/theoretical-saturation): The answer is not a number, it is a shape. The last new failure code arrived at trace 31 of 40, leaving a tail of nine, which just clears the rule. Reshuffle the reading order twenty thousand times and only 22 percent of orders agree.
- [Inter-Annotator Agreement: Before You Trust Your Own Labels](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/inter-annotator-agreement): Two graders on the same 40 traces, two rubrics, and the SAME raw agreement of 0.93 both times. Kappa says 0.69 and 0.82, because raw agreement cannot see how much of the matching was luck.
- [Building an Annotation Tool: A Hundred Lines That Decide Whether Anybody Finishes](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/building-an-annotation-tool): Teams abandon error analysis in a text editor and record it as a conclusion about the method. Three behaviours fix it, one feature ruins the measurement, and the whole thing is a hundred lines of Python.
- [From Taxonomy to Metric: Which Categories Earn a Number](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/from-taxonomy-to-metric): Four measured categories, and the arithmetic says not one of them is big enough to be a weekly metric yet. Counting codes instead of traces turns 6 into 11. Five questions decide it, and the one everybody skips is whether anyone would change plans if the number moved.
- [What a Judge Score Actually Means](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/what-a-judge-score-means): Three cheap judges, measured on the same 40 traces. One scored 90% accuracy while missing nothing, and 82% while missing three failures in seven. A judge is a classifier, a failure is the positive case, and the two error rates cost different people different things.
- [Which Labels You Check Against](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/which-labels-you-check-against): The same judge, the same 40 traces, checked against two careful graders instead of one. It caught 100% of failures against the first and 57% against the second. Three traces caused the whole gap, and both graders agreed about all three.
- [Does Giving the Judge a Third Box Help?](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/does-a-third-box-help): The last lesson found our judge was scored on a choice it was never allowed to make. So we gave it the third box the graders had, and ran the same 40 traces again. It got worse in five of the six judge-and-grader pairs, and the reason is worth more than the fix.
- [Case Difficulty: Why a Set Can Pass Everything and Tell You Nothing](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/case-difficulty): A metric that never fails cannot detect anything, and a case won by 0.001 is recorded as the same pass as a case won by 0.05. Measure the difficulty of your eval set before you trust its pass rate.
- [Synthetic Cases: A Generator Writes Questions, Not Your Index](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/synthetic-cases): Generating more eval cases grows the set along the dimensions you described. On a real set, 80% of the difficulty came from near-duplicate documents in the corpus, which no question generator can reach.
- [Coverage: The Third of Your Index That Nothing Tests](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/index-coverage): An eval set is described by how many cases it has. Join it against your index instead: on a real set, 5 of 15 documents were never the right answer to anything, and never appeared in any trace at all.
- [82% Against 79%: What a Hundred-Case Eval Can Actually Detect](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/eval-power): A 100-example eval reports 80% with a 95% interval of 72% to 88%, and catches a real five-point improvement 15% of the time. Work out what your set can detect before you ship on what it says.
- [A Repaired Golden Set Reported 100%. The System Earned 70%.](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/golden-set-rot): A strict golden label failed 9 of 30 real traces. A rule that accepts every document named after the label turned 70% into 100%. Reading the nine rows showed every one was a real miss. Loosen labels by reading, never by a rule about names.
- [Thirty Rows, Ten Questions, One Leaky Split](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/split-leakage): A random row-level split of this repo's trace file puts 7.8 of 10 questions on both sides. Measured over 4,000 splits, it does not inflate the score at all. It halves the apparent spread, which is worse.
- [Your Set Has a Different n for Every Metric](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/effective-sample-size): Thirty rows in the file. Ten distinct retrieval results and twenty-two distinct answers. The same measured 70% carries an interval of plus or minus 16.4 points or plus or minus 28.4, depending on which number you count.
- [Two Documented Biases, Neither of Them in My Judge](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/testing-judge-bias): Padding made answers 2.3 times longer with no new facts. It won 35% of verdicts, and 5% when the padding was on a worse answer. One of those two nulls can be quoted and the other cannot, and telling them apart is the skill.
- [The Effect That Shrank When I Looked Again](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/effect-that-shrank): Ten items suggested that structure beats length. Twenty new items cut the gap from 35 points to 15 and the test came back undecided. Pooling all thirty gives p = 0.022, and that number is not a finding.
- [The Sentence That Changed Every Verdict](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/judge-sycophancy): One line saying which answer came from the system under test moved 24 of 60 verdicts. A stated opinion moved all 60. The same prompt sent twice moved none, so the sentence is the cause.
- [The Question You Cannot Ask One Judge](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/judge-self-preference): Asking a model whether it prefers its own answers cannot work, because quality and authorship change together. Two judges can answer it. On 50 questions the difference came to -7 points. The two judges gave the same verdict 83 times in 100, but 62 of those were both judges picking the same slot.
- [The Rubric Moved the Number, Not the Order](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/rubric-sensitivity): Six reasonable rubric phrasings on the same 60 answers moved the pass rate 11.7 points, and 35 points on the answers that were actually weak. But 14 of 15 phrasing pairs still agreed about which answers were better, which decides what survives a reword and what does not.
- [The Number Moves When Nothing Does](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/judge-jitter): The same eval, run six times on the same answers with the same prompt, moved 6.7 points. A release gate set below that floor fires on its own. This measures the floor, shows what it costs a gate, and finds that the answers which wobble are the same ones a reworded rubric moved.
- [Retrieval Got Better and the System Got Worse](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/context-k): Raising k took recall@k from 45% to 90% on 818 documents, and took the number of questions the system actually got right from 10 of 20 down to 6. Same retrieval, same questions, no judge in the grading path. The component metric cannot go down, which is what makes it dangerous to optimise.
- [The Score You Had Before Retrieval Ran](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/retrieval-baseline): The same 20 questions answered with no documents at all scored 6.7% against the RAG pipeline's 15.1%. That is 44% of the reported number produced without retrieval. A RAG score with no ablation beside it is a sum of two things, and only one of them is the work you are doing.
- [The Order You Handed Them In](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/context-position): The same ten documents, the same question, the same model at temperature zero. Move the right document from the middle of the list to the end and the score goes from 4 of 14 to 14 of 14. Every wrong answer named the first or the last slot, never one in the middle, and your eval did not record the order.
- [Who Wrote Your Test Questions?](https://www.systemdesign.academy/courses/ml-advanced/lessons/evals/question-wording): Sixty documents. For each one, two questions: one in the document's own words, one in the everyday words a real person uses. The search found the first kind 100 percent of the time and the second kind 35 to 63 percent of the time. If your test questions were written by someone reading the documents, your score is telling you about the first kind.
#### LLM and GenAI Ops (ml-advanced)
- [Reading a Model Name: What 27B, FP8, A3B and Instruct Actually Mean](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/reading-a-model-name) [FREE, no signup]: A model name is a spec sheet written in shorthand. Learn to decode publisher, family, version, parameter count, mixture-of-experts markers, post-training suffixes and numeric formats, and to predict from the name alone whether a model will fit on your GPU.
- [Fine-Tuning vs RAG vs Prompting: Choosing Your Approach](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/fine-tuning-vs-rag-vs-prompting): Fine-tuning, RAG, and prompting each adapt an LLM differently. Here is what each one changes and a framework for picking the right one.
- [Parameter-Efficient Fine-Tuning: LoRA and QLoRA](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/parameter-efficient-fine-tuning): How LoRA and QLoRA fine-tune a large language model by training under one percent of its weights, and why that lets a 7B model fit on a single GPU
- [Evaluating LLMs in Production: Grading Answers That Have No Right Answer](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-evaluation): How to measure LLM output quality when there is no single correct answer: golden sets, LLM-as-judge, RAG metrics, online A/B tests, and regression testing every prompt change
- [Prompt Management and Versioning: Treat Prompts as Production Code](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/prompt-management-versioning): Why prompts are production artifacts, not strings in your app, and how a prompt registry gives you versioning, testing, rollback, lineage, and caching
- [Vector Databases and Approximate Nearest Neighbor Search](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/vector-databases-ann): How embeddings become searchable points in space, why exact nearest-neighbor search breaks at scale, and how ANN indexes like HNSW, IVF, and PQ make retrieval fast enough to serve
- [LLM Inference Optimization: Serving More Tokens Per GPU](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-inference-optimization): Why LLM inference is slow and expensive, and the techniques that fix it: the KV cache, continuous batching, PagedAttention, quantization, and speculative decoding
- [LLM Guardrails and Safety: Wrapping the Model So It Can Ship](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-guardrails-safety): Why raw LLM output is a production risk, and how input and output guardrails wrap the model to catch injection, PII leaks, toxicity, and hallucination
- [LLM Agents in Production: The Model in a Loop](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-agents-production): The operator's view of running LLM agents: deployment topology, budgets and governors, tracing, canary rollout, human escalation, and the golden signals you page on
- [Token Cost Engineering: Where the LLM Bill Actually Goes](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/token-cost-engineering): Most of an LLM bill is tokens you resend, not tokens users type. Decompose the spend, then cut it with caching, slimming, output caps, and batching.
- [Model Routing and Cascades: Running Three Models Without Chaos](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/model-routing-and-cascades): Cheap models for easy traffic, strong models for hard cases. How to route, when a cascade stops paying, and the one thing your gateway must never do.
- [Rate Limits and Provider Failure: The Number One Production Error](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/rate-limits-and-provider-failure): HTTP 429 is the first failure an LLM app hits at scale. Beat rate limits and provider outages with token buckets, backoff, degraded modes, and failover.
- [LLM Observability and Tracing: Seeing Inside a Nondeterministic System](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/llm-observability-and-tracing): An LLM app fails behind a 200 OK: right status, wrong answer. Capture the rendered prompt, token split, and per-step latency on one trace id.
- [Reliability for Stochastic Systems: Error Budgets for Probabilistic Output](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/reliability-for-stochastic-systems): Uptime cannot measure a system that answers wrong while healthy. Define a quality SLO, run an error budget on the pass rate, and degrade to honest fallbacks.
- [Was the Cheap Model Sure? Choosing Which Questions a Cascade Passes On](https://www.systemdesign.academy/courses/ml-advanced/lessons/llm-genai-ops/cascade-escalation-signals): A cascade lets a cheap model answer first and passes some questions to a strong model. Everything depends on the clue that picks which ones. On 175 maths problems with known answers, the cheap model said it was 100 out of 100 sure on almost every answer, so asking how sure did exactly as well as random. Asking it three times beat random clearly, but mostly by spotting hard questions, a free rule on the question did about as well, and in tokens it cost as much as the strong model alone.
#### AI Security and Agent Safety (ml-advanced)
- [Prompt Injection and the Lethal Trifecta: Why LLM Agents Leak Data](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/prompt-injection-lethal-trifecta) [FREE, no signup]: Prompt injection is the top LLM security risk. Learn why it has no clean fix, how indirect injection works, and what the lethal trifecta is.
- [Defending Against Prompt Injection: The Layered Playbook](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/defending-against-prompt-injection): Prompt injection has no single fix. Layer the defenses: separate instructions from data, use dual-LLM and CaMeL, allow-list egress, gate risky actions.
- [Tool and Sandbox Security: Locking Down What an Agent Can Do](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/tool-and-sandbox-security): How to secure agent tools and code sandboxes: least privilege, microVM isolation, default-deny egress, argument validation, and resource limits.
- [Agent Identity and Authorization: Whose Authority Is It?](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/agent-identity-and-authorization): When an agent acts, whose authority does it use? Give agents scoped, short-lived, delegated identity with OAuth token exchange, audience-bound tokens, and workload identity instead of one broad ambient key.
- [OWASP Agentic Threat Model and Red-Teaming Your Agent](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/owasp-agentic-top-10-red-teaming): Organize agent threats with the OWASP LLM Top 10 and Agentic guides, then red-team your own agent and run it as an attack-success-rate eval in CI.
- [The Lethal Trifecta: Why Prompt Injection Plus Tools Plus Data Is Unsolved](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/the-lethal-trifecta): A filter that stops 95 percent of prompt injections still loses to twenty retries. The lethal trifecta is unsolved because detection is the wrong tool. Break it leg by leg instead.
- [Data Exfiltration and Egress Control in LLM Systems](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/data-exfiltration-and-egress-control): A steered model needs no click to leak data, it writes a line of markdown. The exfiltration channels, and the egress controls that close every exit.
- [AI Supply Chain Security: What You Import Runs as You](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/supply-chain-and-model-provenance): Your LLM app loads weights, datasets, packages, and MCP servers you never wrote. Pin, hash, scan, sign, and verify provenance so an import cannot own you.
- [PII and Compliance for LLM Apps: Redaction, Retention, and Data Boundaries](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/pii-and-compliance-for-llm-apps): A prompt to an LLM provider is a data export. Redact PII before it reaches the model and your logs, bound retention, and honor deletion across every store.
- [Improper Output Handling: The Model's Answer Is Untrusted Input](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/improper-output-handling): OWASP LLM05. Hardening the prompt does not help if you paste the answer into a query. Measured: strict JSON parsing leaked as many dangerous payloads as a regex, and rejected good answers too.
- [Marking Untrusted Text: Spotlighting, Measured on Small Models](https://www.systemdesign.academy/courses/ml-advanced/lessons/security/spotlighting-measured): Security lesson 02 found that a fixed tag around a document stopped nothing. This lab tests every form of spotlighting from the Microsoft paper on three small models, with 44 documents that each hide one order. Tags, fixed or random, barely moved the numbers. Datamarking cut qwen2.5's successful attacks from 22 of 44 to 9, and a random marker to 4, while it kept every right answer on clean documents. Base64 stopped every attack only because the models could not use it: one refused, two made numbers up.
#### Core Concepts (ml-foundation)
- [Why ML Models Fail in Production: The Production Gap](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/the-production-gap) [FREE, no signup]: Why ML models fail in production is rarely the model. Learn what ML platform engineering is and how it moves models from notebook to reliable service.
- [Model Packaging and Containerization: Killing 'Works On My Machine' for ML](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/model-packaging-containerization) [FREE, no signup]: Why a pickle file is not a deployable model, and how Docker, image layering, and BentoML seal a model, its exact dependencies, and its inference code into one content-addressed image that runs anywhere
- [Model Serving and Inference APIs: Turning a Model File Into a Service](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/model-serving-inference) [FREE, no signup]: How a trained model becomes a live API. Covers online vs batch serving, REST vs gRPC, dynamic batching, autoscaling and p99, with a batch simulator.
- [Training Pipelines and Orchestration: Retraining That Runs Itself](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/training-pipelines-orchestration) [FREE, no signup]: Why retraining by hand does not scale, and how a training pipeline turns model retraining into a reproducible DAG that runs on a trigger, checkpoints through failure, and ships only when the new model wins
- [Feature Stores: Killing the Train and Serve Skew Bug](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/feature-stores) [FREE, no signup]: How feature stores end train and serve skew, keep training and serving in sync, and let teams reuse features across every model
- [Model Registry and Versioning: Knowing What Is Actually Live](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/model-registry-versioning): How a model registry becomes the single source of truth for every model version, its metrics, its lineage, and which one is serving real users right now
- [Monitoring and Drift Detection: Catching a Model That Fails Without an Error](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/monitoring-drift-detection): How to detect data drift, concept drift, and prediction drift with PSI, KS, and KL, and wire alerts back into retraining
- [Scaling and GPU Infrastructure: Serving Models Without Burning Money](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/scaling-gpu-infrastructure): Why GPU serving gets expensive and how to fix it: the VRAM ceiling, batching, quantization, distillation, caching, card sharing, bin-packing, parallelism, autoscaling, spot, and horizontal vs vertical scaling
- [Production RAG and LLM Systems: Running It, Not Just Building It](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/production-rag-llm): The operations view of a live RAG plus LLM service: serving, latency budgets, cost per query, caching, index freshness, autoscaling, graceful degradation, retrieval monitoring, guardrails, and SLOs.
- [The Full ML Platform: Assembling Every Piece Into One Golden Path](https://www.systemdesign.academy/courses/ml-foundation/lessons/core/ml-platform-capstone): How packaging, serving, pipelines, feature stores, registries, monitoring, and scaling assemble into one self-service internal ML platform
#### Fine-Tuning: The Narrow Decision (ml-foundation)
- [Before You Fine-Tune: Try the Cheapest Baseline First](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/try-the-cheapest-baseline): Before training a model on labelled examples, try the cheapest thing those examples allow: look up the most similar one and copy its label. On the prompting chapter's 40 new messages, a small 0.5B model that was not trained gave 19 usable answers; the same model fine-tuned with LoRA on 441 labelled messages gave 38; a lookup with nomic-embed-text and no language model gave 39; the chapter's full prompt on two 3B models gave 39. The lookup and the fine-tune missed the same receipt message, and on 40 messages none of the last three can be told apart from luck.
- [Building a Training Set You Can Trust: House Rules, Two Blind Labellers and a Leak Check](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/building-a-training-set): How I built the chapter's ticket data before training anything: house rules, two writers, two blind annotators, per-field agreement and kappa, a boundary rule found through 37 splits, a full relabel after the rule proved faulty, and a leak check. The result: 500 training rows, 60 validation rows and 140 test messages, with every remaining gap stated.
- [Where Prompts Slip: The Same House Rules for Four Models, Checked Field by Field](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/where-prompts-slip): I gave four prompted models, from 3B to 14B, the full house rules for the ticket task and a JSON schema, and scored every field of every reply on the 140 test messages. Every reply was valid JSON and the order number was right almost every time, but the whole ticket was right on only 50 to 93 of 131 fully gold messages. Reading every wrong answer shows which rules slip: a bigger model fixed extra words in the item and questions read as requests, but not comments that ask for nothing, which even qwen2.5:14b got wrong 11 times out of 13. Twenty-two fields were wrong in all four models.
- [Training a Small Model on 500 Tickets: What LoRA Changes, and How It Compares With a Prompt](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/training-a-small-model): I trained three small Qwen2.5 models (0.5B, 1.5B and 3B) with LoRA on the 500 training tickets, gave them a one-line prompt instead of the house rules, and scored them field by field against four prompted models on the same 140 test messages. Under 1% of each model's weights was trained, into an add-on of 12 to 27 MB. The fine-tuned 0.5B got 93 whole tickets right of 131, the same as the prompted qwen2.5:14b, but on different messages: 22 right only in one, 22 right only in the other, sign test p = 1.
- [What Each Training Setting Does: More Examples, More Passes, Learning Rate and Masking, Tested One at a Time](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/what-the-settings-do): I tested twelve changes to the same 0.5B ticket model, one setting at a time: the seed, the number of training examples, the number of passes, the learning rate, and whether the loss counts the prompt. First I measured the wobble, what the seed alone does: the totals were 93, 92 and 93 whole tickets of 131, but 15 to 21 tickets flipped between right and wrong between any two seeds. Against that, two changes clearly mattered: more examples (26, 58, 78, 93 for 50, 100, 200 and 500 rows) and a learning rate far too small (58).
- [Did Fine-Tuning Forget Anything? Sums, Word Problems and Sorting, With the Ticket Adapter Off and On](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/did-fine-tuning-forget-anything): I asked two small models 120 questions they were never trained on, once as downloaded and once with lesson 4's ticket adapter on top: 40 sums, 40 word problems and 40 messages to sort. The 0.5B went from 32 right to 19, the 1.5B from 55 to 55. Item by item, no change on any task can be told apart from luck. Reading the replies shows what did move: the 0.5B's sums came back with too few digits 9 times instead of 0, its word-problem replies got shorter, and its sorting did not break so much as swap one favourite answer (account, 32 times) for another (returns, 38 times).
- [An Attack on a Fine-Tuned Model: One Override Sentence, Two Fine-Tunes and Two Prompts](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/an-attack-on-a-fine-tuned-model): I added the prompting chapter's override sentence to all 140 ticket test messages and sent them to four models, each as built in this chapter. The fine-tuned 0.5B ignored it: 140 of 140 replies valid JSON, 93 whole tickets right, the same as without the attack. The fine-tuned 1.5B, trained the same way, broke the format on 76 of 140: 47 of those wrote to the customer as the shop, many claiming something had been done, and its whole tickets fell from 102 to 23.
- [Teaching a Model Facts: Heavy Training Taught Them, and No Set-up Said 'I Don't Know'](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/teaching-a-model-facts): I invented 30 facts about a shop that does not exist and gave them to two small models in two ways: pasted into the prompt, or trained in with LoRA. On 90 test questions worded differently from the training questions, heavy training scored 73 on the 0.5B model and 84 on the 1.5B. With the facts in the prompt, they scored 61 and 82. The 0.5B gap is a hint, not beyond luck. The trained models also got facts wrong in a new way, by giving another fact's answer.
- [Serving a Fine-Tune: Moving It Into Ollama, Shrinking It, and Testing It Again](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/serving-a-fine-tune): I moved lesson 4's two fine-tuned ticket models out of mlx, the tool that trained them, and into Ollama: merged the adapter into the weights, imported them at 16, 8 and 4 bits, and scored all 140 test messages again. My first import had no chat template, and the check I had written to catch that kind of mistake passed it. Once fixed, the 16-bit 1.5B got 105 whole tickets right of 131 (mlx: 102) and the 0.5B 81 (mlx: 93). 8 bits held; 4 bits cost the 1.5B 47 tickets. With Ollama's JSON schema, the 1.5B kept 90 under lesson 7's attack, where mlx without a schema gave 23.
- [A Fine-Tune on New Cases: Messages Shaped Unlike Its Training, and a Gap No Model Can Fix](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/a-fine-tune-on-new-cases): I wrote 90 new customer messages in six kinds the training data never showed: long emails, two requests in one message, other languages, decoy numbers, rare products and plain cancel requests. On the 53 that both annotators fully agreed on, the fine-tuned 0.5B got 22 whole tickets right, the fine-tuned 1.5B 21, the prompted 7B 24 and the prompted 14B 25. No fine-tune could be told apart from the 14B (smallest p 0.424). On all 15 cancel requests the two annotators disagreed about the category: a gap in the rules that no model can fix.
- [The Fine-Tuning Decision: Every Step of the Chapter, and What a Ticket Costs](https://www.systemdesign.academy/courses/ml-foundation/lessons/fine-tuning/the-fine-tuning-decision): The whole fine-tuning decision in the order you would make it, each step backed by a result this chapter stored, plus one new measurement: what a ticket costs. On quality, the fine-tuned 1.5B could not be told apart from the prompted 14B, on the 131 test messages (102 against 93, p 0.1628) or on 53 new ones (21 against 25, p 0.4240). On cost, the fine-tune read 54 prompt tokens a call against 642, and in an interleaved timing on one laptop the 14B took 3.63 times as long per message. My first timing failed its own drift check, and the lesson shows why a timing needs a control.
#### How Models Generate (ml-foundation)
- [What a Language Model Actually Outputs: Odds for Every Next Token](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/what-a-model-outputs): A language model does not write sentences. At every step it gives odds for every possible next token, and something else picks one. Measured with qwen2.5:3b on 24 prompts: for 12 facts the top next token had a median probability of 0.70; for 12 open prompts only 0.17. The same five words with and without the hidden chat template gave ' Paris' at 0.62 or 'The' at 0.999.
- [Temperature and Sampling: How One Token Gets Picked](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/temperature-and-sampling): The model gives odds; a sampler picks one token. Measured with qwen2.5:3b on 3 prompts, 40 runs per setting: at temperature 0 every run wrote the same text; at 0.7 almost every run was different. For 'The capital of France is', 40 of 40 runs began with Paris at temperature 0 and only 5 of 40 at 1.5, where the text broke into other scripts and symbols. The same seed gave the same text every time.
- [Reading Is Fast, Writing Is Slow: Where a Model's Time Goes](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/reading-and-writing-speed): Every Ollama reply reports how long the model spent loading, reading the prompt and writing the answer. Measured on a laptop with qwen2.5:3b over 35 calls in a shuffled order: the median reading speed was 194 tokens a second and the median writing speed 30. Even the slowest reading of any call was faster than the fastest writing of any call. A long reply costs far more time than a long prompt.
- [Attention: How Each Word Looks Back at the Words Before It](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/attention): Attention lets every word in a prompt gather meaning from the words before it. Worked by hand with tiny numbers, then read from a real model, Qwen2.5-0.5B: a median of half of each word's attention went to the very first word, a word never looked ahead, and on 10 sentences where one word changes what 'it' means, the model was right 6 times and gave the same answer for both versions in 4 of 5 pairs.
- [What a Long Prompt Really Costs: Wait, Writing Speed and Memory](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/what-a-long-prompt-costs): Measured on a laptop with qwen2.5:3b over 30 shuffled calls: the first word came back after a median 1.0 s for a 256-token prompt and 36.6 s for about 7,900 tokens. With the laptop's own slowing taken out, writing got about 1.1 tokens a second slower for every 1,000 prompt tokens. A bigger context window made Ollama report more memory: 2.05 GB at 2,048 tokens, 2.79 GB at 32,768.
- [Reusing a Prompt's Start: Read Once, Answer Many Times](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/reusing-a-prompts-start): Measured on a laptop with qwen2.5:3b: a 3,951-token prompt took a median 14.35 s to read, and 0.09 s when it was sent again with only the question after it changed. Changing one word 25%, 50% or 75% of the way in made the second read take 0.78, 0.55 and 0.28 of the first, close to 'everything before the first difference is reused'. The reported token count never dropped. Put what changes last.
- [The Context Window: What Happens When a Prompt Does Not Fit](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/the-context-window): Measured with Ollama and qwen2.5:3b: a prompt of about 5,900 tokens sent with a 4,096-token window was cut to 2,050 tokens, the first 4 and the last 2,046, with no error in the reply. A code placed at the start was found 0 times in 5; at the end, 5 in 5. At three window sizes the kept part was exactly half the window plus 2. Without num_ctx, this laptop's window was 4,096.
- [Chat Templates: The Text a Conversation Becomes](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/chat-templates): A model reads one text, not a list of messages. Measured with Ollama and qwen2.5:3b: the message 'Hi' is 1 token as raw text and 30 tokens as a chat, because the template adds markers and a default system message. Filled in by hand, the template gave the same 39 tokens and the same reply as the chat endpoint. On 20 short instructions, its own template gave 20 clean replies; another model family's template gave 0, though 16 first answers were still right.
- [Quantization: The Same Model in Fewer Bits](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/quantization): The same Qwen2.5 3B model stored three ways and measured on a laptop: 6.18 GB at 16 bits per weight, 3.29 GB at 8.5 and 1.93 GB at about 5. The 8-bit file kept the 16-bit file's top next word on 24 of 24 prompts and got the same 33 of 40 sums right; the 5-bit file kept 19 of 24, all 5 changes on near ties. Writing speed went from 11.1 to 18.3 to 28.4 tokens a second, while reading barely changed.
- [Repetition and Stopping: How a Reply Ends](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/repetition-and-stopping): Measured with Ollama and qwen2.5:3b on 12 open prompts sent as raw text: with greedy decoding, 7 of 12 replies fell into a loop and only 1 ended on its own within 256 tokens. A repeat penalty of 1.1 cut loops to 1 of 12. A stop sequence ended 7 of 12 replies at a line break; a 20-token limit cut all 12 mid-sentence. Through the chat template, 4 of 12 ended on their own. Check done_reason on every reply.
- [Comparing Models Fairly: Same Test, Different Results](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/comparing-models): Four small models measured on one laptop with the same tests: gemma3:4b got 33 of 40 sums right, qwen2.5:3b 30, llama3.2:3b 23, and qwen3:4b 0, because it reasoned in its reply and ran out of its 32-token budget. Given 1,024 tokens it finished 22 sums and got all 22 right, at a median 926 tokens a sum. Only one pairwise gap was clearly bigger than luck. qwen2.5 wrote fastest, 38.2 tokens a second; gemma3 wrote 20.2.
- [One Call, End to End: The Chapter in a Single Request](https://www.systemdesign.academy/courses/ml-foundation/lessons/how-models-generate/one-call-end-to-end): One realistic chat request to qwen2.5:3b, taken apart with everything this chapter measured: a 1,307-token prompt, 12 of those tokens added by the chat template, read in 2.89 s at about 450 tokens a second, 36 tokens written at about 38 a second, ended by the model itself. Sent again with a different question, the same start was read in 0.10 s, and the wait for the first word fell from 3.57 s to 0.22 s.
#### The ML & AI Lifecycle (ml-foundation)
- [Start With the Dumbest Model: The Baselines a Model Must Beat, on the Future](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/start-with-the-dumbest-model): The first lesson of the ML and AI lifecycle chapter. On 45,312 half-hours of electricity prices, I scored three rules that learn nothing against three trained models, on the future. Saying whatever the previous half-hour was scored 0.8484. The best model, boosted trees with the previous label as an input, scored 0.8193. The same trees on a shuffled test said 0.8830. Where the lazy rule fails, why the label is sticky by construction, and what balanced accuracy adds.
- [Same Code, Different Model: What Changes Between Two Identical Training Runs](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/same-code-different-model): I trained the same boosted trees on the same electricity data 20 times, changing only the seed. Accuracy on the future ran from 0.7179 to 0.7906, 20 different models, a spread of 7.3 points. With one default setting switched off, every seed gave 0.7520. Where the randomness hides in scikit-learn's defaults, what to pin so a run can be repeated, how to log it, and why one run is not enough to compare two models.
- [Picking the Best of Many: How Much of the Winner's Lead Survives on New Months](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/picking-the-best-of-many): I trained 200 random settings of the same boosted trees on electricity prices and kept the one with the best validation score. The more settings I tried, the higher the winner's validation score and the lower its test score: 0.703 and 0.690 for one setting picked at random, 0.718 and 0.664 for the best of 200. Two follow-ups split the cause: most of that growth came from choosing on months unlike the months the model then served, and about a fifth was ordinary selection luck. What tuning is, what a validation set and a test set are for, the winner's curse, and what to report.
- [The Promotion Gate: How Often a New Model That Is Not Better Gets the Job](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/the-promotion-gate): A pipeline retrains a model and promotes it if it beats the live one. I measured three gate rules on electricity prices. With two models that were equal by construction, 'higher score wins' promoted the new one in 0.43 to 0.51 of the evaluation windows, a fixed 1-point margin in 0.30 of 200-row windows, and McNemar's test, which reads only the rows where the two disagree, in 0.018 to 0.022. A real 6.7-point gain was caught by every rule once the windows held 1,000 rows.
- [Canary and Shadow: How Much Live Traffic It Takes to See a Worse Model](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/canary-and-shadow): A new model goes live in one of two ways: a canary, where a share of real requests goes to it, or shadow, where it answers every request silently next to the old one. I measured both on electricity prices, with the right answer arriving half an hour later. On a known 4.09-point regression, shadow raised the alarm in all 20 starts (median 241.5 half-hours) and a 20% canary in 12. Checking after every half-hour turned a 1% false-alarm rate into 6 to 10% for a canary.
- [When to Retrain: A Schedule, a Trigger, and What Each Retrain Buys](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/when-to-retrain): A model learns from the past and then answers the future, and the future drifts away from what it learned. I served 22,656 half-hours of electricity prices under four rules: never retrain (0.6610), retrain every 28 days (0.7516, 16 retrains), every week (0.7636, 67 retrains), and retrain when last week falls 5 points below the week after the last retrain (0.6812, 8 retrains). The trigger stopped firing after one bad first week set its bar at 0.405. A follow-up found that most of retraining's gain came from repairing what one input, the date, did to a frozen model.
- [Stale Pieces in a Pipeline: A Cached Input, an Old Scaler, and What Each One Hides](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/stale-pieces-in-a-pipeline): A model is only as fresh as the pieces in front of it. I served the same trained models on electricity prices with two stale pieces: an input cached once a day, and a scaler left over from the first 10% of the data. The cached input cut accuracy from 0.8193 to 0.7489, about the level of a model that never had that input (0.7527, too close to call), while the mix of answers barely moved (UP on 0.541 of rows against 0.484). The old scaler dropped the other model to 0.4512 and made it say UP on every row.
- [Lineage and Rollback: Rebuilding Last Month's Model Exactly, and What Breaks When You Cannot](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/lineage-and-rollback): When a new model misbehaves, the team goes back to the one that worked. I rebuilt last month's model five ways from its written record. With every note kept, the rebuild gave the same answer on all 6,000 rows it had served, and so did the saved model file loaded back. With the seed not written down, 19 guesses agreed on 0.920 to 0.970 of rows, none exactly, and the check on the training data stayed quiet. Changed data made the record's fingerprint fire, and one wrong rebuild even scored higher than the real model. What to write down, what to keep, and which check sees what.
- [When the Answers Arrive Late: What a Label Delay Costs a Model, and How Late You See Its Mistakes](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/when-the-answers-arrive-late): A model can only learn from answers that have arrived, and a team can only score it on them. I retrained the same model every week on 22,656 half-hours of electricity prices while the right answers arrived half an hour, a day, a week, four weeks or twelve weeks late. Accuracy fell from 0.7636 to 0.6954, though not smoothly: four weeks scored 0.7315 against a week's 0.7299, a gap that cannot be told apart from chance.
- [The Lifecycle as One Loop: Nine Steps, Nine Measured Failures, and What Saw Each One](https://www.systemdesign.academy/courses/ml-foundation/lessons/ml-lifecycle/the-lifecycle-as-one-loop): The wrap of the ML and AI lifecycle chapter. I walk the loop once, in the order a team meets it, from the baseline to the answers that arrive late, and for each step give what it is for, the failure its lesson measured on the electricity data, the check that saw it or the fact that nothing did, and what the lesson could not show.
#### Prompting as Engineering (ml-foundation)
- [The Parts of a Prompt: Adding One Piece at a Time](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/parts-of-a-prompt): A prompt has parts: an instruction, a format rule, context, examples and the input. Measured with qwen2.5:3b on 40 customer messages, adding one part at a time: with the message alone, 0 of 40 answers were usable; with an instruction, 19; with a format rule, 36; with context, 40. Two examples added nothing, because the prompt was already at 40 of 40.
- [Vague, Soft and Specific: Asking for Exactly What You Want](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/specific-instructions): Five everyday tasks, each asked three ways on qwen2.5:3b and scored by the same check: vague ('Summarise this'), with a soft word ('Summarise this briefly') and specific ('one sentence of at most 25 words'). Vague met the requirement 0 of 40 times, soft words 0 of 40, specific 38 of 40, and the specific wording also wrote 7.4 times fewer tokens.
- [System Message or User Message: Where the Rules Go](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/system-and-user-messages): Lesson 1's rules placed in the system message or in the user message, on 40 customer messages and two models. With nobody pushing back, the place made no difference (40 of 40 usable either way on qwen2.5:3b). When the customer wrote 'Ignore all previous instructions', no answer was usable with the rules in the system message, on both qwen2.5:3b and llama3.2:3b; with the same rules in the user message, 11 and 15 were. The system message is the right place for fixed rules, but it is not a lock.
- [Examples in a Prompt: How Many, Which Ones, in What Order](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/examples-in-a-prompt): Lesson 1's sorting prompt without its context lines, with no examples, four examples, the same four reversed, four different ones, or three that share one answer, on 40 messages and two models. On llama3.2:3b, no examples gave 20 of 40 right; four examples gave 28 to 36, depending on which four and in what order. On qwen2.5:3b the same examples moved the score only from 36 to 39.
- [Asking for a Format: Words, JSON Mode and a Schema](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/asking-for-a-format): Lesson 1's 40 messages, answered as a JSON object with three fields, asked for four ways on two models: in words, with Ollama's JSON mode, with a JSON schema, and with the schema alone. Every way gave valid JSON 40 of 40 times. The schema stopped one invalid category ('travel') but turned it into a valid wrong one; without the words, summaries grew past the limit; and with a token limit of 20, a schema did not stop the JSON from breaking.
- [Quoting Untrusted Input: Tags, a Reminder, and a Fake Closing Tag](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/quoting-untrusted-input): Lesson 3's override against four defences, on 40 customer messages and two models. Wrapping the customer's text in tags with a rule that it is data gave 8 usable answers of 40 on qwen2.5:3b and 0 on llama3.2:3b. Repeating the task after the quoted text gave 40 and 32. A fake closing tag typed by the customer cut llama's 32 to 20; escaping it in code brought it back to 30. The defences lost no right answers on normal messages, at a cost of about 99 extra prompt tokens per call, and a check in code is still required.
- [Thinking Step by Step: When It Helps, and What It Costs](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/thinking-step-by-step): The same questions asked two ways, answer only or think step by step and end with an Answer: line, on two small models. On 40 word problems, qwen2.5:3b went from 2 right to 37 and llama3.2:3b from 7 to 37. On two-digit sums, llama went from 23 to 37 while qwen moved from 30 to 32, a change within luck. On sorting messages, it lifted llama from 20 to 38 on a weak prompt and made no clear difference where the prompt was already good. Step-by-step replies had a median of 78 to 204 tokens, against 2 to 5 for direct ones.
- [Small Rewordings: The Same Instruction, Seven Ways](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/small-rewordings): Lesson 1's sorting instruction, reworded seven ways that mean the same thing, on 40 customer messages and two small models. qwen2.5:3b gave 32 to 39 usable answers of 40 and llama3.2:3b 4 to 31. Four orders of the category list gave 36 to 40 and 19 to 28. Only 27 of the 40 messages were usable under every wording on qwen, and 2 on llama. The same two wordings came top on both models, but on qwen only 2 of 21 pairs of wordings differed by more than luck.
- [Instructions Around a Long Document: Where to Put the Rules](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/long-documents): Lesson 1's sorting rules with 2,000 words of unrelated text pasted in four places: nowhere, before the rules, between the rules and the message, or after the message. On qwen2.5:3b the scores were 40, 40, 40 and 36 of 40; on llama3.2:3b 36, 38, 35 and 34. No placement differed from the no-document control by more than luck. Every prompt fitted the 8,192-token window. A document that does not fit is the real danger: in the demo, a cut prompt with the document last lost both the rules and the question.
- [Prompts as Code: Templates That a Customer Cannot Break](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/prompt-templates): A prompt template filled four ways, on lesson 1's 40 messages and 7 tricky ones, with no model. str.format crashed on all 47 inputs, because the template held a JSON example with braces. A two-step format crashed on 2 inputs a customer typed and, on a third, copied an internal note into the customer's message. replace and an f-string had no problems. Then five versions of lesson 1's prompt that differ by a space, a new line, a label or a pasted copy: qwen2.5:3b gave 39 to 40 usable answers of 40, llama3.2:3b 33 to 39, and no change beat luck.
- [Testing a Prompt on New Cases: Why the Score You Tuned On May Be Optimistic](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/testing-on-new-cases): Three prompts from this chapter, scored on the 40 messages the chapter was built around and on 40 new messages written before any prompt was scored on them, on two small models. No prompt scored higher on the new messages. qwen2.5:3b went from 36, 40 and 39 usable answers to 34, 40 and 36; llama3.2:3b from 20, 36 and 33 to 17, 32 and 31. The best prompt held on qwen and lost 4 on llama. Of the 50 wrong answers on the new messages, 26 were account messages, none of which used the word account.
- [Putting a Prompt Together: Which Parts Still Earn Their Place](https://www.systemdesign.academy/courses/ml-foundation/lessons/prompting/putting-a-prompt-together): Every part this chapter kept, in one prompt: context lines, examples, tags with a data rule, a reminder after the message and a JSON schema. On 40 new messages, lesson 1's short prompt gave 34, 4 and 29 usable answers on qwen2.5:3b (normal, override, fake closing tag) and 15, 1 and 0 on llama3.2:3b; the full prompt gave 39, 38 and 40, and 39, 38 and 39. Then each part was taken out once. No row without a part followed the override, several scored as high as the full prompt, and after correcting for 36 comparisons no single part's removal could be told apart from luck.
#### Tokens and Embeddings (ml-foundation)
- [What a Token Is: How a Language Model Reads Text](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/what-a-token-is): A language model never sees letters or words. It sees tokens: small pieces of text, each with a number. This lesson shows how real tokenizers cut text, measured with OpenAI's tiktoken. This course's 1.27 million words come to 1.29 tokens per word. Numbers are cut into pieces of up to three digits. One sentence written in eight languages took 11 tokens in English but up to 100 in other languages with GPT-2's tokenizer, and 12 to 19 with GPT-4o's. That difference is paid in money and in context space.
- [How a Tokenizer Learns Its Pieces: Byte Pair Encoding, Built From Scratch](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/how-a-tokenizer-learns): Where do a tokenizer's pieces come from? Nobody writes the list by hand. It is learned from text by one small rule, byte pair encoding: find the most common pair of neighbouring pieces and glue it into one. This lesson builds that rule in plain Python and trains it on this site's own lessons. Starting from 94 single characters, text on lessons it never saw took 6.12 tokens a word; after 4,000 merges it took 1.58. GPT-4o's tokenizer, with 200,019 pieces, takes 1.28.
- [What an Embedding Is: Meaning as a List of Numbers](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/what-an-embedding-is): An embedding turns a piece of text into a list of numbers, so that texts with similar meaning get similar numbers. This lesson measures two real embedding models on this laptop. With the English-trained model, sentences with the same meaning scored 0.89 to 0.96 and unrelated ones 0.52 to 0.60. Then the test that matters for a global audience: the same sentence in six other languages. The English-trained model scored five of the six translations below an unrelated English sentence. A multilingual model scored all six 0.92 to 0.98.
- [Searching by Meaning: Real Semantic Search, in Six Languages](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/searching-by-meaning): Semantic search finds text by meaning instead of by shared words. This lesson builds it over this site's 878 lessons and asks 10 questions in six languages. In English the two models were close: 6 and 4 of 10 in the top 5. Across the five other languages, the multilingual bge-m3 found 21 of 50 and the English-trained nomic-embed-text 5 of 50. Four questions were never found. Rewriting one lesson's short description moved it from rank 34-50 to rank 1 in all six languages.
- [The Silent Cut: How Much of Your Text an Embedding Model Really Reads](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/the-silent-cut): Send an embedding model a long text and it may read only the start, with no error. Measured with Ollama on this laptop: nomic-embed-text and bge-m3 both stopped at 2,048 tokens by default, although bge-m3's own page says 8,192. A sentence hidden past that point changed the score by exactly 0.000. Raising two settings let bge-m3 read 8,192 tokens, and then the hidden sentence was read but barely moved the score: 0.332 against 0.328.
- [Splitting Text Into Pieces: Why Search Works Better on Small Chunks](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/splitting-text-into-pieces): Should you embed a whole document as one list of numbers, or cut it into small pieces first? Measured on this course's 112 AI lessons with 470 of their own quiz questions and bge-m3 reading up to 8,192 tokens. With one embedding per lesson, the right lesson came first for 286 questions. With 200-word pieces, for 356. Pieces won on short lessons too, not only long ones, and whole lessons still won 31 questions.
- [Cosine, Dot Product or Distance: Which One Compares Embeddings?](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/cosine-dot-and-distance): The same question and the same text scored 0.741 with cosine and 521.380 with a dot product. The difference is the length of the vectors. Ollama's /api/embed returned vectors of length 1 for all 878 lesson texts, where all three measures agree; its older /api/embeddings returned lesson vectors of length 19.0 to 27.4. On those, a raw dot product changed 109 of 300 top-5 places for nomic-embed-text and 51 for bge-m3, while the number of right answers barely moved.
- [Shorter Embeddings: Keep the First Numbers, Measure What You Lose](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/shorter-embeddings): Ollama can return only the first N numbers of an embedding. Measured on 470 quiz questions and 1,823 pieces of this course: nomic-embed-text found the right lesson in the top 5 for 431 questions at 256 of its 768 numbers, against 441 at full size. bge-m3 found 414 at 256 of 1,024, against 439, and fell faster below that. Keeping the first numbers instead of the last helped nomic only a little. At 16 numbers, the small example picked the wrong text.
- [Smaller Numbers: Storing Embeddings in One Byte or One Bit](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/smaller-numbers): Every embedding number normally takes 4 bytes. Measured on 470 quiz questions and 1,823 pieces of this course: storing each number in 1 byte gave the same totals for both nomic-embed-text and bge-m3, within one question, in a quarter of the space. Keeping only 1 bit per number found the right lesson in the top 5 for 36 and 24 fewer questions. Ranking by bits first, then checking the best 50 pieces again with the full numbers, won almost all of it back, but needs the full numbers kept somewhere.
- [Finding Neighbours Fast: Search a Few Clusters Instead of Everything](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/finding-neighbours-fast): Exact search compares a question with every stored piece. Clustered search groups the pieces first and looks only in the nearest groups. Measured on 470 quiz questions and 1,823 pieces with nomic-embed-text and 32 clusters: searching 8 clusters, about a third of the comparisons, found 94 to 95% of the exact top 5 and kept 429 to 430 of 441 right lessons. Searching 1 cluster did 6% of the work and found the right lesson in the top 5 for 338 to 349 of 470.
- [Keyword Plus Meaning: When Combining Two Searches Helps, and When It Hurts](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/keyword-and-meaning): Keyword search matches words; meaning search matches embeddings. Combining the two rankings (rank fusion) was measured twice. On 470 quiz questions that reuse the lessons' own words, keyword search alone found the right lesson in the top 5 for 453, meaning search with bge-m3 for 439, and both together for 455. On lesson 4's 60 questions written in other words and six languages, keyword search found 5, meaning 27, and both 22. Where keyword search matched no word at all, combining changed nothing; where it matched words in the wrong lessons, it pulled good answers down.
- [One Search System: Stacking the Chapter's Choices and Measuring Each](https://www.systemdesign.academy/courses/ml-foundation/lessons/tokens-and-embeddings/one-search-system): The last lesson of the chapter builds one search system from its choices, one at a time, on the same 470 quiz questions with bge-m3. Whole lessons found the right lesson in the top 5 for 401. Splitting into pieces raised it to 439. Storing each number in 1 byte kept 439 in a quarter of the space. Searching 16 of 32 clusters cut the comparisons by almost half for 437. Adding keyword search took it to 451, which is still 2 below keyword search on its own: on these questions, which reuse the lessons' words, a plain keyword search is very hard to beat.
#### Why Production Breaks (ml-foundation)
- [The Score That Lied: A Random Split Against a Split in Time](https://www.systemdesign.academy/courses/ml-foundation/lessons/why-production-breaks/the-score-that-lied): I scored the same model on two years of hourly bike rentals in two ways. On test hours picked at random, the boosted trees missed by 26.09 rentals an hour on average. On the last 20% of the hours, the future the model would really meet, they missed by 44.07, 1.69 times as much. Month by month, hour by hour, and why a random split flatters a model on data that arrives in time order.
- [Tomorrow Is Different: A Model Frozen on 2011, Scored Through 2012](https://www.systemdesign.academy/courses/ml-foundation/lessons/why-production-breaks/tomorrow-is-different): I trained a model on one year of hourly bike rentals and scored it month by month through the next year, under three rules: never retrain, retrain every month on all the data so far, or retrain every month on only the last 12 months. Over 2012 the frozen model missed by 89.5 rentals an hour, almost always too low; monthly retraining missed by 43.7, the 12-month window by 47.9. What changed, why the frozen model kept guessing last year, what retraining cannot fix, and how to decide when to retrain.
- [Train/Serve Skew: One Input Computed Two Ways](https://www.systemdesign.academy/courses/ml-foundation/lessons/why-production-breaks/train-serve-skew): I trained one model on hourly bike rentals, then sent it the same test hours six more times, each time with one input computed a little differently at serving time: every hour shifted by 5 (as UTC is in winter), temperatures in Fahrenheit, humidity as a percentage, the weekday counted from Monday, a weather label spelled with a capital letter, and a missing wind speed filled with zero. The average miss went from 44.07 to as much as 215.68 rentals an hour, and for two of the six it barely moved while up to 22% of the guesses changed.
- [Failing Silently: Which Checks Catch a Broken Input Before the Answers Arrive](https://www.systemdesign.academy/courses/ml-foundation/lessons/why-production-breaks/failing-silently): I served a bike-rental model 145 days of test data, broke one input at a time in seven ways, and ran six simple checks on every day before any real answer was known. Range, unseen-value, stuck-value and zero-count checks each caught the break they were built for, on every day, with few or no false alarms. The distribution check fired on every clean day and was useless as designed. The worst break, the hour shifted by 5, which made the average miss almost five times worse, got past every one of them.
- [Watching Inputs Before the Answers Arrive: Drift Measures Against the Real Error](https://www.systemdesign.academy/courses/ml-foundation/lessons/why-production-breaks/watching-inputs): Real answers often arrive late, so teams watch a model's inputs and outputs in the meantime. I measured four drift measures for every month of 2012 against a bike-rental model trained on 2011, and ranked each against the model's real monthly error. None moved in step with it: the best rank correlation was 0.58. The usual psi, against the whole year, sat between 1.3 and 4.9 every month because it measures the season, and the model's own training months score just as high.
- [When the Data Answers Back: A Feedback Loop, Simulated on Real Demand](https://www.systemdesign.academy/courses/ml-foundation/lessons/why-production-breaks/when-the-data-answers-back): Some models decide what the world can do, and then learn from what it did. I simulated that on two years of real bike-rental counts: a model forecasts demand, bikes are placed to match, riders can only rent the bikes that are there, and the model is retrained on what was rented. The real counts are the demand; the rule for placing bikes is invented. With no margin the model missed true demand by 90.7 bikes an hour over 2012, against 43.6 for a model that learned from every rider, and turned away 769,320 riders. Scored on its own records it missed by 2.6.
- [A Production Readiness Check: Everything This Chapter Broke, in the Order You Would Check It](https://www.systemdesign.academy/courses/ml-foundation/lessons/why-production-breaks/a-production-readiness-check): The chapter's wrap: a six-item readiness check for a model that is about to go live, in the order a team would run it, with every item backed by a result an earlier lesson stored. Score on the future (a shuffle said 26.09, the future 44.07). Score the newest labelled month and read its sign (frozen 89.5, retrained 43.7; with the year input made constant, retraining scored 88.89 against 91.41 for the frozen model, February to December). Compute serving inputs with the same code (the hours shifted by 5 gave 215.68). Put cheap input checks in front and know what they miss.
#### Agents in Production (ml-intermediate)
- [The Agent Loop: Why AI Agents Fail as Tasks Get Longer](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/agent-loop-reliability) [FREE, no signup]: An AI agent is a language model in a loop, and its reliability compounds: 95 percent per step over 20 steps is only 36 percent. Why agents fail on long tasks, and the levers that fight it.
- [Tool Calling Is a Contract, and the Model Will Break It](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/tool-calling-contract): The model never runs your functions. It emits text proposing a call, and sometimes that text is wrong. How to validate every tool call against a schema before it runs, and reject cleanly so the model self-corrects.
- [Durable Execution: An Agent Run Is a Workflow, Not a Request](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/durable-execution): A web request retries whole. An agent run cannot. Durable execution persists every step and recovers by replaying recorded results, not re-running non-deterministic model calls.
- [Agent Memory and Context: What the Model Sees, and What It Forgets](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/agent-memory-context): Agents have no memory between calls. What the model sees is the transcript you resend every step, and what it forgets is whatever you cut to fit. How to manage both.
- [Multi-Agent Systems, and When Not to Build One](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/multi-agent-when-not-to): One agent with good tools is the right default. When a team of agents actually helps, why adding agents usually lowers reliability, and the decision rule.
- [Agent Reliability Math: pass@k, pass^k, and Why 95 Percent Fails](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/agent-reliability-math-pass-at-k): pass@k and pass^k measure opposite things. See the math that makes a 95 percent per-step agent fail most 20-step tasks, with a Python example.
- [MCP in Production: What the Protocol Buys and Costs](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/mcp-in-production): MCP turns N times M tool integrations into N plus M, but every attached server rents context-window tokens on every request. When it pays, when it does not.
- [MCP Security: Tool Poisoning, Rug Pulls, and the Confused Deputy](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/mcp-security): In MCP the trust boundary is the tool description, not the output. Learn tool poisoning, rug pulls, shadowing, confused deputy, and the defenses that hold.
- [Context Rot: Measuring How Long Context Degrades Your Agent](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/context-rot-measuring-degradation): A model's recall falls as its window fills, and worst from the middle. Measure the fill level where your agent starts to rot, and re-run it per model version.
- [Agent Planning: ReAct vs Plan-and-Execute](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/planning-patterns-react-plan-execute): ReAct reasons before every action and pays for it; plan-and-execute thinks once and runs cheap. When each wins, and why replan-on-failure is the default.
- [Reflection and Decomposition: Making Agents Check Their Own Work](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/reflection-and-decomposition): Self-critique alone barely helps and can hurt. Reflection works against a real check, and decomposition turns one long chain into short verified segments.
- [Deterministic Scaffolding: The LLM Explains, It Does Not Arbitrate](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/deterministic-scaffolding): Deterministic scaffolding keeps control flow in code and calls the model only to classify or draft. The model recommends; code checks state and arbitrates.
- [Structured Output Costs Right Answers: One JSON Box, Measured](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/structured-output-cost): Forcing a model to reply with one JSON answer field is great for programs. On 70 maths problems with known answers, two small models collapsed from 46 and 70 right to almost none when the only room was the answer box. A steps field before the answer won most of it back, but not all. When only asked nicely for JSON, one model left the calculation unworked in the answer field, and the other ignored the rule and wrote its working anyway.
- [Too Many Tools, or Tools That Look Alike? Measuring How Agents Pick](https://www.systemdesign.academy/courses/ml-intermediate/lessons/agents-in-production/tool-count-and-look-alikes): Two small models, 60 requests, each offered 5 to 100 tools. Picking the right tool barely suffered until 100 tools, and look-alikes alone did no harm. The bigger problems were elsewhere: right choices written as broken JSON, valid calls that Ollama's chat API silently dropped, and replies that took 13 times longer with every tool on offer.
#### Data Engineering for ML (ml-intermediate)
- [Batch vs Streaming Data for ML: When Fresh Beats Cheap](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/batch-vs-streaming-data): Batch vs streaming data for machine learning, measured on real data. I predicted hourly bike rentals with two inputs, the rides in the last hour and the average over the last week, computed once a night or fresh every hour. Fresh counts cut the error from 56.4 to 30.5 rides an hour; keeping only the weekly average nightly cost 0.09 more; a model trained on fresh counts but served the nightly ones scored 173.9. Then bounded and unbounded data, the Lambda and Kappa architectures, event time and watermarks, and at-least-once against exactly-once, with a real crash and replay.
- [Data Ingestion Pipelines: Getting Data Into the ML Platform](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-ingestion-pipelines): How data gets from databases, APIs, events and files into the place a model learns from: pull and push, batch, CDC and streaming, keyed idempotent writes, backpressure, late data, backfills and schema drift, every claim checked against the tools' own documentation.
- [ETL vs ELT: What a Model Loses When the Raw Rows Are Thrown Away](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/etl-vs-elt): ETL vs ELT for machine learning, measured on a real shop's 1,067,371 invoice lines. A nightly job kept one row per customer per day in the warehouse and nothing else. Adding features rebuilt from the raw lines moved AUC from 0.712 to 0.725 for who orders again and from 0.728 to 0.736, a gap that could be luck, for who cancels; on a question chosen after the results, one the daily table never recorded, it moved AUC by 0.055 (0.059 with one added feature). Keeping every line cost $0.00218 a month.
- [Data Lakes, Warehouses, and Lakehouses for ML](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-lakes-warehouses-lakehouses): Data warehouse vs data lake vs lakehouse for ML, with every claim checked against the vendors' price lists, the Delta, Iceberg and Parquet docs and the 2020 and 2021 papers. Then a lab on a real table of 17,379 hours: the lake stored a file with a comma in every temperature while the lakehouse refused it; after a bad backfill only the pinned version gave back the exact training rows (0 of 8,734 guesses moved, against 8,677 from the lake); 731 tiny Parquet files took 64 times the bytes of one; and vacuum deleted the files the pinned version needed.
- [Data Validation and Quality: Stopping Garbage Before It Reaches the Model](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-validation-quality): Data validation for machine learning, measured on five years of real hourly weather from Beijing. Eight checks against six silent faults: a strict set of checks held 37 of 730 good days, a loose set held none, and no check fired on a thermometer reading 3 degrees high more often than on a good day. Two columns swapped raised the model's error from 46.39 to 129.92, and one rule between them caught it every day. Then Great Expectations and Pandera code that ran, gates, quarantine, drift, and when a check earns its place.
- [Data Versioning: Reproducing the Exact Data That Trained a Model](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-versioning): Data versioning, measured on a real shop's sales table: 1,047,877 sale lines over two years, corrected 16,675 times by returns. I kept 110 weekly versions four ways. Whole copies took 754.5 MB, one file per month 431.9 MB, one file per week 301.6 MB, and week files with small change files 27.8 MB, because every tool stores whole files. Real DVC stored exactly the month layout's 431.9 MB. Today's table filtered by date differed from a tagged version on up to 953 lines. Then retention, erasure, and what MLflow's dataset digest misses.
- [Data Labeling and Annotation: The Expensive Bottleneck of Supervised ML](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-labeling): What a labelling budget buys, measured on 4,601 real emails over 30 random splits. Buying the labels the model was least sure of got within 0.01 of a model trained on all 3,450 pool labels with 188 labels on average, in all 30 repeats; buying labels at random got there by 500 labels in only 12. At 50 labels the aimed labels were behind random in 21 of 30 repeats, and the emails they picked were so hard that a model scored 0.698 on them against 0.922 on a random test set. Ten labelling rules and no hand labels scored 0.891, about what 100 random labels bought.
- [Handling Imbalanced and Messy Data: Why Your 99% Accuracy Is a Lie](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/handling-imbalanced-data): Imbalanced data, measured on 284,807 real card payments, 492 of them fraud. A model that calls every payment legit scores 99.827% accuracy and catches none. Over 30 seeds, the same model's ROC-AUC stayed at 0.975 as fraud got rarer, while PR-AUC fell from 0.981 to 0.755. A stratified split fixed the test set's fraud count but barely narrowed the score's spread. Class weights left the ranking about the same and raised the mean probability 44 times. Then messy data: 9,144 exact copies, and where leaks come from.
- [Synthetic Data Generation: When Fake Data Beats No Data](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/synthetic-data): Synthetic data, measured on 30,000 real card holders over 30 seeds. Boosted trees trained on rows from a Gaussian copula scored 0.734 ROC-AUC on real people, against 0.751 when trained on real rows. Tested on its own made-up rows, the worst generator scored highest, 0.947. A generator that copied people with a little noise passed one privacy check and failed another, and passing both is still not privacy. Adding made-up rows to 200 real ones helped one model and hurt the other.
- [Data Contracts and Schema Management for ML Pipelines](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/data-contracts-schema): Data contracts and schema compatibility, measured on 17,379 real hours of a bike share. I replayed twelve producer changes against a consumer that had not upgraded. BACKWARD, the registry's default, passed 9 of them; 3 of those left the old consumer unable to read the records. Four changes decoded cleanly and moved the model's answers, and every mode, FULL included, passed all four: temperatures in Fahrenheit raised the error by 30.20 rides an hour, and a rename that both sides gave a default of 0 by 36.39.
- [Wrong Labels: How Many Can a Model Survive, and Can You Find Them?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/label-noise-measured): Every labelled dataset has some wrong labels. On 1,797 real handwritten digits, a lab makes up to 40 percent of the training labels wrong, randomly or by swapping look-alike digits, and trains four kinds of model. Nearest neighbour copied mistakes straight through, while a forest and 15 nearest neighbours barely noticed random ones. From 20 percent up, look-alike mistakes did far more damage. Flagging every row where an out-of-fold prediction disagreed with its label found almost all the random mistakes, but was fooled by repeated confusions.
- [Rebalance, or Just Move the Threshold? Measured on Rare Classes](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/rebalance-or-threshold): When one class is rare, teams reach for class weights, oversampling or SMOTE. On two real datasets thinned to 2, 5 and 10 percent rare, over 100 seeds, rebalancing looked like a big win at the default threshold of 0.5. With enough rare rows, simply tuning the plain model's threshold beat every rebalanced model left at 0.5. With very few, a threshold tuned on the validation set was a guess, and choosing it by cross-validation on the training data did best. Rebalancing barely changed the ranking, and it leaves probabilities that need recalibrating.
- [Leakage Before the Split: How Pure Noise Scored 93% Accuracy](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/leakage-before-the-split): Four common preparation steps, done before the split and done inside it, over 100 seeds, each checked against rows put away at the start. Choosing columns on pure noise promised 0.927 accuracy and delivered 0.498. Target encoding a random id promised 0.792. Copying rare rows promised an almost perfect F1 and delivered 0.876. Rescaling barely leaked at all. The steps that read the label leak the most.
- [Drift Alarms Against Real Harm: When the Alarm Rings, Is the Model Worse?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/data-engineering-for-ml/drift-alarm-vs-harm): Five kinds of new data sent to a real model, 100 batches each, with three drift monitors and the true error measured every time. The alarm rang on every batch of a shift the model did not care about, and stayed silent through a 64% rise in error when prices moved. Alarm and harm lined up only when the model met data it had never seen, in the one such case tested, and there the model's own guesses drifted too. Fifty labelled rows per batch caught what no input monitor could, at a price in false alarms.
#### Features and Feature Stores (ml-intermediate)
- [What a Feature Is: A Better Model or a Better Feature?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/what-a-feature-is): Measured on 809,561 real invoice lines from a UK online shop: will a customer buy again in the next 30 days? I gave one model the raw columns of each customer's last invoice line, and the same model six simple features made from their whole history. Then I tuned a stronger model on the raw columns over 78 settings. The made features lifted average precision from 0.393 to 0.545. Tuning lifted it by 0.0001 with seed 0, and by 0.0023 on average over 20 seeds, about 67 times less than the features.
- [Point-in-Time Joins: A Latest-Value Join Promised 0.714 and Delivered 0.301](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/point-in-time-joins): Real feature tables keep every version of a value with the time it became true, and a training row must read the version that existed at its cutoff. On 44,521 real training rows from a UK online shop, I joined the same feature table three ways and trained the same model on each. Joining each customer's latest values put the future into 70.7 percent of the rows, a median of 334.6 days ahead. Offline, on a random hold-out, that table promised an average precision of 0.714.
- [Feature Freshness: Two Weeks Stale Cost Nothing I Could Measure](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/feature-freshness): Live systems often serve features that a scheduled job computed hours or days earlier. On real data from a UK online shop, I computed six customer features as of the cutoff minus a lag, from 1 hour to 30 days, and served them to a model trained on fresh features, 20 seeds each. Up to 14 days stale, the change in average precision could not be told apart from zero. At 30 days all 20 seeds lost a little, 0.0037 on average, although 20.3 percent of the test customers had an order the stale features never saw; a bootstrap of that mean still crossed zero.
- [Window Aggregations: Which Window, and Do More Windows Help?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/window-aggregations): Teams write features like purchases in the last 7 days, and must pick 7, 30, 90, 180 or 365 days. On 26,851 real test rows from a UK online shop, I gave one model each window alone, then all six together, then lesson 1's six features with and without the windows. The 7-day window scored an average precision of 0.237, close to a random guess at 0.196, because 93.7 percent of test rows had nothing in it. The valid months picked 180 days, which scored 0.533 on the test months.
- [Online/Offline Consistency: Do Two Correct Implementations Agree?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/online-offline-consistency): Teams compute a feature twice: in pandas over history for training, and event by event for serving. I wrote both for lesson 1's six features on 26,851 real test rows from a UK online shop. With every choice made the same way, 17,670 rows still differed, in the last digits of money only, and 20 scores moved. Then I changed one defensible choice at a time. A UTC clock changed 20,796 rows by an hour with no measurable effect on the score. Ignoring returns in money was the only single choice that moved test AP measurably, by -0.000490 on 0.5450, and four choices at once by -0.000612.
- [Late Events and Watermarks: What Happens to Late Data Mattered More Than the Wait](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/late-events-and-watermarks): Streaming feature pipelines use a watermark to decide how long to wait for late events. The shop data has no arrival times, so I simulated them: one delay per invoice from two stated delay models, 20 seeds. I tried watermarks from 0 to 48 hours and two policies for late events. When late events were kept for later results, at most about 275 of 26,851 test rows had a wrong feature, and the change in average precision could not be told apart from zero at any watermark.
- [Categorical Features at Scale: One-Hot, Hashing or Target Encoding?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/categorical-features-at-scale): A shop sells 4,646 products. On top of lesson 1's six features, I gave one model each customer's favourite product code in 13 different contestants, mostly encodings, with 20 seeds each, on 26,851 real test rows. Hashing at 1,024 or more buckets added about 0.002 average precision, a gain just clear of luck; one-hot's similar gain did not survive a second check. Target encoding with the usual folds of rows lost 0.0064, because a customer's own other months leaked in; folds of customers fixed it. Frequency encoding lost 0.107.
- [Missing at Serving: No Fill Gives Back a Row That Is Gone](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/missing-at-serving): When a feature lookup comes back empty at serving time, something has to fill the gap. On real data from a UK online shop, I simulated new customers with no row yet and failed lookups for 1 to 50 percent of known customers, then tried six fills (NaN, 0, the mean, the median, -1, a missing indicator) with a model trained clean or trained with the same gaps, 20 seeds each.
- [Feature Versioning and Backfill: The Drift Alarm Rang Without a Definition Change](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/feature-versioning-and-backfill): What happens when a feature's definition changes? On real data from a UK online shop I changed two of lesson 1's features: money with returns left out, and frequency as days instead of invoices. I measured five ways the change can reach a model, 20 seeds each. Retrained on the new money, test AP fell by 0.0012 from 0.5452; on the new frequency it rose by 0.0040. Serving the new value to the old model moved it by -0.0005 and +0.0031, and could not be told apart from retraining. A half-done backfill behaved like a full one or a little worse; training only on the redone months cost up to 0.0395.
- [Feature Cost and Selection: Two Fifths of the Lines, No Measurable Loss](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/feature-cost-and-selection): Every feature costs compute and storage, every day. On real data from a UK online shop, I measured how much 21 features matter three ways (shuffle, drop and retrain, and the trees' own split record), priced each one in event lines read, and cut the list with backward elimination on the validation months at 20 seeds. The three importance measures disagreed: shuffling one column, inv_180, cost 0.1197 of AP, yet retraining without it lost nothing. Frequency and its five window twins each looked useless alone and cost 0.0292 together.
- [Embeddings as Features: Did the Meaning of Product Names Help?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/embeddings-as-features): Every customer of this shop bought products with names like 'regency cakestand 3 tier'. I turned all 5,257 names into lists of numbers twice, once with a local neural model and once with word counts, averaged them per customer, and added them to lesson 1's six features. Over 20 seeds and 26,851 real test rows, no embedding gave a gain I could tell apart from luck. A control built from random lists, which knows which products a customer bought but nothing about their names, did as well as either embedding; how far ahead or behind it looks depends on which random table you draw.
- [Feast Hands-On: The Same Join, Row for Row, Until a Row Has No Snapshot in Its Window](https://www.systemdesign.academy/courses/ml-intermediate/lessons/features-and-feature-stores/feast-hands-on): Does a real feature store do the point-in-time join we wrote by hand? I gave Feast 0.66.0 the same customer tables as lessons 1 and 2 and asked for all 86,045 training, validation and test rows of this chapter. With no time limit, Feast's file store equalled lesson 2's merge_asof on 86,045 of 86,045 rows, to the last bit, and lesson 1's model trained on it scored the same test AP, 0.5450. A row stamped exactly at the cutoff was taken every time. Then I set a 90-day TTL, a limit on how old a value may be.
#### Packaging, Registry and Versioning (ml-intermediate)
- [What Is in a Model File: It Is Mostly Numbers, and Loading It Runs Code](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/what-is-in-a-model-file): I took the features chapter's trained model and looked inside the saved file. Pickle made a 202,700-byte file. 95.9 percent of it was the raw bytes of the model's numpy arrays, and the rest named 16 classes from numpy and scikit-learn. Loading a pickle does not only read data: it imports those names and calls them. To show this safely, I built a harmless test file that writes one temp file and prints one line, and I read its call in the disassembly before loading it. Then I ran two scanners.
- [Saving Formats Compared: The Scores Never Moved, the Bytes Did](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/saving-formats-compared): Pickle, joblib or skops? I saved the features chapter's model, and three extra models, in eleven formats: pickle protocol 4 and 5, joblib with no compression and with zlib, gzip, bz2 and lzma, and skops stored and zipped. All 44 files gave back every prediction to the last bit, in the same process and in a second environment. Size moved a lot: for the chapter model, joblib zlib 3 was 0.45 times pickle, lzma 3 was 0.37 times, and skops stored was 2.15 times, because 43% of the skops file is a text description of the model. skops refused to load the tree models until I named one type by hand.
- [Library Version Skew: Old Model Files Broke on Upgrade, and Retraining Made a Different Model](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/library-version-skew): Train with one scikit-learn, load with another: what happens? I trained the features chapter's model in six separate venvs, scikit-learn 1.3.2, 1.5.2, 1.7.2 and 1.9.1 plus two numpy controls, saved each with pickle, and loaded every file in every venv. Of 30 loads across venvs, 20 failed at load and 2 more at the first prediction. Every load into a newer scikit-learn failed: 13 of 13, covering the five files that had a newer venv to try. Not one failed load warned first. The 8 loads that worked gave identical scores to the last bit.
- [Pinning Dependencies: The Same Requirements File Installed Something Else Every Season](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/pinning-dependencies): What does an unpinned requirements file install six months later? I asked uv what my five-line file would have installed at nine dates over two years. Every three months, 4 to 9 packages moved; six months apart, 6 to 9. At lesson 3's own dates the file gave lesson 3's exact venvs, and all 6 models saved after one install failed after a later one. A hash-locked file installed the same 9 packages twice and trained the same model to the last bit.
- [Container Images: Most of the Size Was the Base and the Libraries, and Slimming Kept Every Score](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/container-images): What makes a model's container image big, and what does slimming cost? I built real Docker images of the features chapter's model: the full python:3.13 base, slim, slim with a hash lock and no pip cache, a multi-stage build, and alpine. Unpacked, they measured 1,569.1, 521.9, 460.8 and 464.2 MB; alpine did not build, because scikit-learn has no musl wheel. 77% of the full image was its base. All four images predicted the same numbers as each other, and 33 of 26,851 differed from the Mac in the last binary digit, with test AP equal to the bit.
- [Reproducible Artifacts: Same Scores, Different File: One Byte Followed the Thread Count](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/reproducible-artifacts): Does training twice give the same FILE, not just the same scores? I trained the features chapter's model in 55 fresh processes and saved every copy five ways. In one process, in five fresh processes, under five hash seeds, 1 minute apart, and again in a separate report 52 minutes later, every setting gave byte-identical files. The thread count did not: the model stores it, so 1, 2, 4, 8 and 10 threads gave five different files that differ in exactly one byte, with identical predictions.
- [A Model Registry, Hands On: An Alias Is a Pointer, and a Rollback Moved One Row](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/model-registry-hands-on): What does a model registry record, and how does 'load the production model' actually work? I ran MLflow 3.16.1 locally with three versions of the features chapter's model. Loading models:/shop-buyers@champion gave predictions identical to the trained model, to the bit. A rollback changed 1 of 320 rows in 1 of 18 files. Moving the alias to a model that takes other columns raised no error and scored test AP 0.1731, below all 20 random orders. The default save refused the model, and deleting a version left its alias behind.
- [Model Signatures: The Right Numbers in the Wrong Shape, and What a Schema Check Catches](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/model-signatures): What happens when serving sends the right numbers in the wrong shape? I sent the features chapter's model 57 faults: swapped columns, a reversed order, money in pence, days as hours, a missing or extra column, ints, text and NaN. A plain numpy call raised an error for only 9 of them (20 of the 57 were random orders). All 36 wrong column orders came back with no error, and test AP fell from 0.5450 to as low as 0.1268. One customer's chance of buying went from 0.0529 to 0.9465.
- [ONNX and Precision: Every Prediction Moved a Little, and No Decision Moved](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/onnx-and-precision): Does converting a model to ONNX change its predictions, and does it matter? I converted the features chapter's model with skl2onnx and ran it with onnxruntime on all 26,851 test rows, for 20 training seeds. The newest release failed to convert until protobuf went back a version. Then every chance of buying differed from scikit-learn's, by at most 1.94e-7 for seed 0, and no decision flipped in any seed. In 5 of 20 seeds, 9 rows of two customers took another path through the trees, because their money sat a few trillionths above a threshold and 32-bit numbers cannot tell the two apart.
- [The Model Supply Chain: A Fingerprint Caught Every Change, Including the Harmless One](https://www.systemdesign.academy/courses/ml-intermediate/lessons/packaging-and-registry/model-supply-chain): How do you know the model file you load is the file you trained? I recorded the features chapter's model's sha256 and 100 golden rows at training time, then changed copies of the file five ways. A sha256 check caught all of them, and also fired on a harmless 4-thread rebuild that changed one byte and no prediction. One flipped byte in twenty crashed the process; another sent a branch outside its tree and gave garbage scores in the lab, then looped or crashed in later runs.
#### Retrieval and RAG in Production (ml-intermediate)
- [The RAG Scale Cliff: What Breaks Between 100 and 5 Million Documents](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-scale-cliff) [FREE, no signup]: Why RAG fails in production as the corpus grows: a system that shines on a 100-document demo gets subtly worse at 5 million, offline metrics stay silent, and here are the levers that fight the RAG scale cliff.
- [Chunking: The First Lever on Retrieval Quality](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/chunking-strategies): Chunk boundaries set the ceiling on RAG answer quality. How too-large and too-small chunks fail, why overlap helps, and how to size chunks against your own eval.
- [Reranking: The Highest-Return Change in Your RAG Stack](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/reranking): Retrieve 50, rerank with a cross-encoder, keep 5. Why a second-stage reranker is the cheapest, highest-impact fix for a RAG system that keeps missing the right chunk.
- [Hybrid Retrieval: When Keyword Search Beats Your Embeddings](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/hybrid-retrieval): Why keyword search (BM25) and embeddings fail in opposite ways, and how Reciprocal Rank Fusion combines both. Includes a runnable Python example.
- [Document Ingestion: The Layer That Quietly Caps Everything](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/document-ingestion): Retrieval quality cannot exceed the quality of the text you extracted. Ingestion is the true ceiling of a RAG system, upstream of chunking, reranking, and hybrid search.
- [The RAG Retrieval Cliff: Engineering Recall Back at Scale](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-retrieval-cliff-scaling): The corpus-growth cliff is one equation, lambda equals N times p. Learn the fixes that hold recall at ten million chunks: filtering, partitioning, coarse-to-fine and hierarchical retrieval, ANN tuning, and what each costs.
- [Query Transformation and Routing: Fixing Retrieval Before It Runs](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/query-transformation-and-routing): Most RAG misses are query failures: the question is worded unlike the chunk. Rewrite, expand, HyDE, decompose, or route the query before it hits the index.
- [RAG Evaluation: Localizing Failure Between Retrieval and Generation](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-evaluation-and-failure-analysis): Recall@k, context precision, and faithfulness split every wrong RAG answer into a retrieval bug or a generation bug, each with a different fix.
- [RAG Caching and Cost: Semantic Cache, Embedding Reuse, and the Bill](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/rag-caching-and-cost): RAG triples a chat bill through retrieved context. Cut it with a semantic cache, embedding reuse, and a reranker, without serving confidently wrong answers.
- [Choosing an Embedding Model: Why the Leaderboard Is Measuring Someone Else's Documents](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/choosing-an-embedding-model): A hands-on lab on two real models. An easy test set cannot tell a free model from a paid one, 30 near misses do more damage than 2,000 unrelated documents, and a missing prefix costs 20 points of recall.
- [Which Lessons Mention Kafka? Questions That Top-10 Search Cannot Answer](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/list-questions-and-graphs): Some questions ask for all of something: which lessons mention Google? Google is in 47 of 818 lessons, and a search that returns 10 can never find more than 10. A lab on 48 real names compares top-10 search, a name index built by a small model (part of the first step of GraphRAG), and a plain text filter. For common names, search found 22 to 34 percent. The model-built index found 93 percent. The plain text filter, with no model at all, found every one.
- [Is There Really a Best Chunk Size? Measured on This Course's Own Lessons](https://www.systemdesign.academy/courses/ml-intermediate/lessons/retrieval-rag/chunk-size-measured): The chunking lesson drew retrieval quality as a hill over chunk size. This lab measured it: 111 lessons, 466 quiz questions, 8 chunk sizes, embedding search against plain keyword search. There was no hill. Embedding search was best with small chunks and fell from 0.908 to 0.725 as chunks grew, steadily before and after the model's hidden 2,048-token limit. Keyword search, with no model, stayed near 0.95 at every size from 128 words up.
#### Serving and Inference Basics (ml-intermediate)
- [Batch or Online Scoring: A Month-Old Score Cost Nothing I Could Measure Here](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/batch-or-online): Should a model score every customer in a nightly batch, or each request as it comes? I replayed 7,403 real visits to a UK online shop through the features chapter's model, scored four ways: online at the visit, or a stored score from a daily, weekly or monthly batch. On the 6,613 visits every way could score, with 20 seeds and a customer bootstrap, no batch could be told apart from online, even a monthly score up to 30.7 days old (-0.0027 to +0.0063 AP).
- [Latency Anatomy: Where the Time in One Prediction Request Goes](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/latency-anatomy): Where does the time in one prediction request go? I served the features chapter's model with FastAPI, uvicorn and SQLite on a quiet 1-vCPU EC2 box, client and service on one core, and timed every step of 20,000 real requests, five times. At the median, a request took 1,075.9 microseconds. predict_proba on one row took 583.8 of them (54%), and the work outside my own code, HTTP, the framework, the loopback and the client, took 397.0 (37%). The feature lookup, parsing, building the row and writing the answer together took less than 8%. An empty endpoint still cost 333.6 microseconds.
- [Tail Latency: What Sits in the Slowest 1% of Requests](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/tail-latency): Why is p99 so far from the median, and what sits in the tail? I sent 100,000 real requests, one at a time, to the features chapter's model on a quiet 1-vCPU EC2 box, five times, and repeated it with four single changes. Here the p99 was only 1.08 times the median and the p99.9 1.23 times, but the slowest request of every run was the very first one, 7.7 times the median. My main guess was wrong: no garbage collection ran during any of the 500,000 measured base requests.
- [Batching Requests: Does It Help a Small CPU Model?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/batching-requests): Does batching help a small model on a CPU, and what does it cost in latency? On a quiet 1-vCPU EC2 box, predict_proba on 1,024 rows took 5.24 times as long as on one row, so one row in a big batch was 195 times cheaper. Then I put a micro-batcher into the chapter's FastAPI service and sent it random (Poisson) arrivals from half to 2.5 times the most it answered with no batching (S, 1,098.4 a second): 17 settings, 7 rates, 5 rounds, every setting compared with no batching on the same arrivals. With no batching the service fell behind at S and gave about 1,045 answers a second at most.
- [Workers and Threads: Processes, Threads, or Both on One Core?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/workers-and-threads): Should a small model's service run more worker processes, more OpenMP threads, or both? My AWS account allows one vCPU at a time, so I measured the question on one core: 11 settings of 1, 2 or 4 uvicorn workers with 1, 2, 4 or 8 OpenMP threads, 5 rounds, the client on the same core. One worker with one thread gave the most answers, 1,071 a second; every other setting gave fewer, told apart. Four started workers (2 or 3 of them got connections) gave 0.981 of that and used 4.14 times the memory.
- [Cold Start: The First Request Waited for Python to Import Its Libraries](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/cold-start): What does a fresh copy of a model service do before its first answer, and what does the first request pay? I started the chapter's service 80 times on a one-core EC2 box, in ten ways, with a clock reading at every step. A bare process with its files in memory answered first after 1,499 ms, and 96.9% of the time until its port opened was Python importing libraries: scikit-learn alone took 824 ms, while reading and unpickling the model took 2.0 ms. With the page cache emptied the start took 2.81 times as long.
- [Load Testing Honestly: Is Your Load Test Lying?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/load-testing-honestly): I measured one small prediction service, which I made stall for 100 ms at random moments, with five kinds of load tester on a one-core EC2 box, 5 rounds each. A closed-loop tester with one connection reported a p99 of 0.97 ms with the stalls and 0.97 ms without them: its p99 hid them, because it timed each stall only once, as its slowest request. An open-loop tester at 546 requests a second, timing each request from when it was meant to go out, reported 84.95 ms. My own paced client and locust 2.46.7, both aiming at the same 546 a second, reported 7.55 and 9.06 ms.
- [Caching Predictions: Every Real Hit Was Stale, and Dropping Old Entries Did Not Help](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/caching-predictions): Can you cache a model's answers, and what does a stale cached answer cost? I replayed the 7,403 real invoices of lesson 1 and five sets of simulated page views through caches keyed three ways, with time-to-live values of 1 hour to 30 days. On real invoices every hit was stale by construction, because an invoice is exactly what changes the inputs, and still no time to live could be told apart from online scoring. On simulated page views a 1-hour cache keyed by customer hit 66.8% of requests and changed 0.4% of the scores it served.
- [Timeouts and Fallbacks: What Should a Service Return When Its Lookup Is Slow?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/timeouts-and-fallbacks): I made the lookup behind a prediction service slow on purpose: 1 lookup in 100 froze for 1 to 2 seconds. Then I tried 21 ways to cope, 5 times each, on a small rented computer with one processor core. With no time limit, the slowest 1 in 100 answers took about 1 second. A 25-thousandths-of-a-second limit (25 ms) with a ready-made backup score brought that down to 26 ms, and about 1.6 answers in 100 were backups. Using the customer's score from last month as the backup hurt the ranking so little I could not measure it.
- [One Box, Many Models: How Many Fit, and What Happens When They Don't?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/one-box-many-models): I put many copies of a model on one small rented computer (1 core, 1.9 GB of memory) and measured how much each took, how many fit, and what happened when one more did not. A 14.3 MB model file took 15.1 MB of memory once loaded, but Python and its tool libraries took 198 MB before any model. Under a 537 MB limit, 27 copies fit. The computer then stopped the program while it loaded the 28th, with no error message. Four copies of a service that loaded the models once and then copied themselves used 385 MB, against 1,080 MB when each loaded its own.
- [Cost per Prediction: What Do 1,000 Answers Really Cost?](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/cost-per-prediction): I rented five small computers from Amazon, one at a time, each with one core. On each, I found the most questions a second it could answer while 99 in 100 answers came back within 25 thousandths of a second. I tried answering one question at a time, and in small groups. Then I divided each computer's price per hour by the answers it gave in an hour. The cheapest computer per hour, the a1.medium, was the dearest per answer: $0.0385 per million answers one at a time. The newest Graviton chip, in the c8g.medium, was the cheapest: $0.0078 per million one at a time and $0.0025 in groups.
- [A Serving Checklist: The Whole Chapter as One Service](https://www.systemdesign.academy/courses/ml-intermediate/lessons/serving-and-inference/a-serving-checklist): I built one small model service twice on a rented computer with one processor core: once as first written, and once with three changes the chapter had measured, a warm-up, one time limit with a stand-in answer, and a batcher. Behind both sat a slow helper that freezes on 1 lookup in 100, so the plain service's failure was built in: it waits for every lookup, and no rate passed.

### System Design Masterclass

#### Consistency Models (advanced)
- [Data Compression](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/data-compression): Reduce data size to save storage and bandwidth, lossless vs lossy, and when each makes sense
- [Memory Management](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/memory-management): How systems allocate, use, and reclaim memory, the difference between systems that scale and systems that crash
- [Thread Pooling](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/thread-pooling): Reuse threads instead of creating them, the pattern that makes web servers handle thousands of concurrent connections
- [Garbage Collection](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/garbage-collection): Automatic memory reclamation, how GC algorithms work and why they cause latency spikes in distributed systems
- [Data Deduplication](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/data-deduplication): Eliminate redundant copies of data to save storage, content-addressable storage and hash-based dedup
- [Checkpointing](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/checkpointing): Save system state at known-good points to enable fast recovery after failures
- [Snapshotting](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/snapshotting): How systems take a consistent snapshot without stopping writes. Compares four techniques, including Redis fork with copy-on-write and Postgres MVCC.
- [Write-Ahead Logging](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/write-ahead-logging): Write changes to a sequential log before applying them, the foundation of database crash recovery
- [Point-in-Time Recovery](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/point-in-time-recovery): Restore a database to any moment in the past using WAL and base backups
- [Data Consistency Models](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/data-consistency-models): The full spectrum from eventual to strong, understand every model and when you need each one
- [Monotonic Reads](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/monotonic-reads): Once you have seen a value, you never see an older one, the minimum useful consistency guarantee
- [Monotonic Writes](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/monotonic-writes): Writes from a single client are applied in order, preventing write reordering across replicas
- [Read-Your-Writes](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/read-your-writes): You always see your own updates immediately, the consistency users expect but rarely get by default
- [Causal Consistency](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/causal-consistency): If A causes B, everyone sees A before B, preserving cause-and-effect in distributed systems
- [Sequential Consistency](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/sequential-consistency): All nodes see operations in the same order, though not necessarily real-time order
- [Snapshot Isolation](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/snapshot-isolation): Snapshot isolation: each transaction reads the database as it was when it began. How MVCC does it, the write skew it allows, and how Postgres stops it.
- [Serializability](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/serializability): Transactions execute as if they ran one at a time, the gold standard of isolation
- [Linearizability](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/linearizability): The strongest consistency guarantee, every operation appears to take effect at a single instant
- [CRDTs](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/crdts): Conflict-free Replicated Data Types, data structures that merge automatically without coordination
- [Operational Transformation](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/operational-transformation): Real-time collaborative editing, how Google Docs lets multiple people type simultaneously
- [Conflict Resolution](https://www.systemdesign.academy/courses/advanced/lessons/consistency-models/conflict-resolution): How replicas settle conflicting writes: last-write-wins, vector clock siblings, CRDTs and custom merge. Shows four strategies with four outcomes.
#### Data Engineering: Lakehouses, Spark and OLAP (advanced)
- [Lakehouse Architecture: What a Folder of Parquet Files Gets Wrong](https://www.systemdesign.academy/courses/advanced/lessons/data-engineering/lakehouse-architecture): What is a lakehouse, and what breaks if a table is just a folder of Parquet files? Warehouse, lake and lakehouse in plain words first. Then a lab on 7,052,769 real NYC taxi trips: while a writer added February as 8 files, a reader that listed the folder saw every half-written state, 9 different row counts; a Delta Lake reader saw only 3,475,226 rows or 7,052,769, never anything between. During an overwrite, the folder reader counted 6,806,334 trips for one month that held 3,331,108.
- [Open Table Formats: What Delta Lake, Iceberg and Hudi Actually Write Down](https://www.systemdesign.academy/courses/advanced/lessons/data-engineering/open-table-formats): Delta Lake, Apache Iceberg and Apache Hudi: what is actually in the metadata? I wrote the same 11,198,026 NYC taxi trips into all three, three commits each, and opened every metadata file. Delta kept 3 JSON commits, Iceberg 10 files in a four-level tree, Hudi a timeline and a metadata table. All three returned January's exact 3,475,226 rows when asked for the first version. After 120 small appends, an Iceberg planner opened 122 metadata files and a Delta reader 22; Hudi packed the rows into 1 file group instead of 120 files. Compaction fixed the count, and cleanup ended time travel.
- [ACID on a Lakehouse: Two Writers, One Table, Who Wins](https://www.systemdesign.academy/courses/advanced/lessons/data-engineering/acid-on-a-lakehouse): Two writers update the same Delta Lake table at once: who wins, and what does the loser see? I ran real writer processes against 3,475,226 NYC taxi trips, 5 runs per case. Two appends both committed every time. Two updates of different rows clashed every time, because they shared the same 9 files: one committed, the other got CommitFailedError. Splitting the table by the filter column let both commit. A check-then-append job started twice stored 317,150 trips for a day of 158,575 in 5 of 5 runs. In delta-rs 1.6.6, a WriteSerializable setting took effect only when spelled writeSerializable.
- [Data Catalogs and Unity Catalog: What a Folder Cannot Do](https://www.systemdesign.academy/courses/advanced/lessons/data-engineering/data-catalogs-and-unity-catalog): What does a catalog like Unity Catalog do that a folder cannot? I put 3,475,226 NYC taxi trips behind three real catalogs: an Iceberg SQL catalog, the Apache Iceberg REST catalog and Unity Catalog OSS 0.6.0. A rename sent one SQL UPDATE and moved 0 of 9 files, while moving the folder broke a reader after 1 of 10 files. A writer that crashed before its swap left a newest metadata file with 123,646 trips nobody committed. Two catalogs on one folder both accepted writes and disagreed. Unity Catalog refused a user with no grants, but the same user read every row straight from the folder.
#### Distributed Systems Core (advanced)
- [State Machine](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/state-machine): Model system behavior as a finite set of states and transitions, the foundation of distributed protocols
- [Finite State Machine](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/finite-state-machine): What a finite state machine is, DFA vs NFA, and how TCP and Kubernetes pods use one. Includes a type-safe state machine in code.
- [CAP Theorem](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/cap-theorem): Consistency, Availability, Partition Tolerance, you can only pick two, and you must pick partition tolerance
- [Network Partitioning](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/network-partitioning): When nodes can't talk to each other, the failure mode that defines distributed systems
- [Split-Brain](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/split-brain): When two halves of a cluster both think they're in charge, the most dangerous failure in distributed systems
- [Quorum](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/quorum): Why quorum reads and writes stay consistent when R + W > N. A grid of every R and W pair for five replicas shows which ones are safe.
- [Leader Election](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/leader-election): How distributed nodes pick a leader without a central authority, bully algorithm, ring, and Raft elections
- [Distributed Locks](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/distributed-locks): Mutual exclusion across network boundaries. Redlock, ZooKeeper locks, and why they're harder than you think
- [Lamport Timestamps](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/lamport-timestamps): How Lamport timestamps order events without synced clocks, using one counter per node. Includes a three-node walkthrough and Python code.
- [Vector Clocks](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/vector-clocks): How vector clocks detect concurrent writes that Lamport timestamps miss. Includes a Python implementation and the Dynamo shopping cart example.
- [Gossip Protocol](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/gossip-protocol): How gossip protocols spread cluster state like a rumor, in O(log N) rounds. Includes a Python simulator, SWIM failure detection and Cassandra.
- [Hinted Handoff](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/hinted-handoff): Store writes temporarily on behalf of unavailable nodes. Dynamo's approach to write availability
- [Read Repair](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/read-repair): How read repair fixes stale replicas during a quorum read in Dynamo-style databases. Compares blocking and background repair, and other repair tools.
- [Anti-Entropy](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/anti-entropy): Background repair using Merkle trees, the last line of defense against data divergence
- [Sloppy Quorum](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/sloppy-quorum): Relaxed quorum rules that prioritize availability over strict replica placement
- [Distributed Transactions](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/distributed-transactions): Coordinating atomic operations across multiple nodes, the hardest problem in databases
- [Two-Phase Commit](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/two-phase-commit): How two-phase commit makes a distributed transaction all-or-nothing, and why it blocks if the coordinator crashes. Includes a 2PC coordinator in code.
- [Three-Phase Commit](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/three-phase-commit): Adding a pre-commit phase to eliminate 2PC's blocking problem, theoretically clean, practically abandoned
- [Saga Pattern](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/saga-pattern): Long-lived transactions without distributed locking, orchestration vs choreography for microservices
- [CQRS](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/cqrs): Command Query Responsibility Segregation, separate read and write models for different scaling and optimization needs
- [Event Sourcing](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/event-sourcing): Store events instead of state, rebuild any point in time from the log of what happened
- [Consensus Algorithms](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/consensus-algorithms): Paxos, Raft, and ZAB, how distributed nodes agree on a single value despite failures
- [Actor Model](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/actor-model): Concurrent computation through message-passing actors. Erlang's secret weapon for building fault-tolerant systems
- [Reactive Systems](https://www.systemdesign.academy/courses/advanced/lessons/distributed-systems-core/reactive-systems): Responsive, resilient, elastic, and message-driven, the Reactive Manifesto and building systems that react to change
#### Reliability & Resilience (advanced)
- [Health Checks](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/health-checks): Verify service health automatically, liveness, readiness, and startup probes
- [Heartbeat](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/heartbeat): Periodic signals that prove a service is alive, the simplest failure detection mechanism
- [Reliability](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/reliability): Building systems that work correctly even when components fail, the engineering discipline of dependable computing
- [Fault Tolerance](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/fault-tolerance): Design systems that continue operating when components fail, redundancy, isolation, and graceful degradation
- [Data Redundancy](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/data-redundancy): Store multiple copies of data to survive hardware failures, replication strategies and trade-offs
- [High Availability](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/high-availability): Design systems for maximum uptime, eliminate single points of failure across every layer
- [Failover](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/failover): Automatically switch to backup systems when primary fails, hot, warm, and cold standby
- [Full Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/full-backup): Complete copy of all data at a point in time, the foundation of any backup strategy
- [Incremental Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/incremental-backup): Back up only changes since the last backup, faster and smaller but slower to restore
- [Differential Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/differential-backup): Back up changes since the last full backup, balanced speed and restore time
- [Snapshot Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/snapshot-backup): Point-in-time copy using filesystem or cloud provider snapshots, fast creation, block-level
- [Hot Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/hot-backup): Back up while the system is running and serving traffic, zero downtime data protection
- [Cold Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/cold-backup): Back up while the system is offline, simplest and most consistent but requires downtime
- [Warm Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/warm-backup): A standby system that receives updates but is not actively serving, minutes to activate
- [Continuous Backup](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/continuous-backup): Capture every change in real-time. WAL streaming for near-zero RPO
- [Clone](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/clone): Create an exact copy of a database or environment, for testing, development, or migration
- [Backup Retention](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/backup-retention): How long to keep backups, policies that balance storage cost with recovery needs
- [Backup Rotation](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/backup-rotation): Grandfather-father-son and other rotation schemes, managing backup lifecycle efficiently
- [Backup Verification](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/backup-verification): Test that backups can actually be restored, because an untested backup is not a backup
- [Restore Testing](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/restore-testing): Regularly practice restoring from backups, the drill that proves your recovery works
- [Severity Levels](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/severity-levels): Classify incidents by impact. SEV1 through SEV4 and what each means for response
- [Incident Classification](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/incident-classification): Categorize incidents by type and impact to route them to the right responders
- [Email Notifications](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/email-notifications): Alert via email, when to use it and how to avoid alert fatigue
- [SMS Notifications](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/sms-notifications): Text message alerts for critical incidents, reaching on-call engineers when other channels fail
- [In-App Notifications](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/in-app-notifications): Alert within the application itself, dashboards, banners, and status indicators
- [Webhook Notifications](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/webhook-notifications): HTTP callbacks for automated incident response, trigger workflows on alert
- [Slack Notifications](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/slack-notifications): Real-time team alerts in Slack channels. ChatOps integration for incident response
- [Teams Notifications](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/teams-notifications): Microsoft Teams alerts and incident channels, enterprise ChatOps
- [On-Call Schedules](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/on-call-schedules): Define who responds to incidents and when, fair rotation that prevents burnout
- [On-Call Rotation](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/on-call-rotation): Rotating on-call duty across team members, weekly, bi-weekly, and follow-the-sun patterns
- [Escalation Policies](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/escalation-policies): Define escalation paths when the primary responder does not acknowledge, tiered response
- [Alert Routing](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/alert-routing): Route alerts to the right team based on service, severity, and time, intelligent dispatch
- [Alert Aggregation](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/alert-aggregation): Group related alerts to reduce noise, one incident, one notification, not 500
- [Alert Suppression](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/alert-suppression): Silence known alerts during maintenance or known issues, prevent alert storms
- [Alert Fatigue Prevention](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/alert-fatigue-prevention): Design alerting to maintain signal quality, because ignored alerts are worse than no alerts
- [PagerDuty Integration](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/pagerduty-integration): Enterprise incident management, on-call, escalation, and incident coordination
- [OpsGenie Integration](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/opsgenie-integration): Atlassian incident management, scheduling, routing, and integration with Jira
- [VictorOps Integration](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/victorops-integration): Splunk On-Call for incident management, timeline-based collaboration
- [ChatOps](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/chatops): Manage operations through chat, run commands, get status, coordinate incidents in Slack
- [Incident Management](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/incident-management): The end-to-end process of detecting, responding to, and resolving production incidents
- [Incident Response](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/incident-response): Structured response when things go wrong, roles, communication, and resolution steps
- [Root Cause Analysis](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/root-cause-analysis): Find the true cause of incidents. Five Whys, fishbone diagrams, and systemic thinking
- [Post-Mortem Analysis](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/post-mortem-analysis): Learn from failures systematically, document what happened, why, and how to prevent recurrence
- [Blameless Culture](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/blameless-culture): Treat failures as learning opportunities, psychological safety enables honest post-mortems
- [Runbook Automation](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/runbook-automation): Automate common operational procedures, from manual checklists to executable workflows
- [Bottleneck Identification](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/bottleneck-identification): Find the constraint limiting system performance, profiling, tracing, and load testing
- [CPU Optimization](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/cpu-optimization): Reduce CPU usage through profiling, algorithm improvement, and concurrency tuning
- [I/O Optimization](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/io-optimization): Reduce disk and network I/O bottlenecks, buffering, batching, and async patterns
- [Network Optimization](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/network-optimization): Reduce latency and bandwidth usage, compression, connection pooling, and protocol tuning
- [Database Profiling](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/database-profiling): Find slow queries and optimize database performance. EXPLAIN plans, index analysis, and query rewriting
- [Application Profiling](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/application-profiling): Identify performance bottlenecks in application code. CPU profiling, memory analysis, and flame graphs
- [RTO](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/rto): Recovery Time Objective, how quickly you must restore service after a disaster
- [RPO](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/rpo): Recovery Point Objective, how much data loss is acceptable in a disaster
- [Disaster Recovery](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/disaster-recovery): Plan and execute recovery from catastrophic failures. DR strategies from backup-restore to active-active
- [DR Testing](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/dr-testing): Regularly test your disaster recovery plan, because untested plans fail when you need them most
- [Business Continuity Planning](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/business-continuity-planning): Keep the business running during disruptions, beyond IT into organizational resilience
- [Self-Healing Systems](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/self-healing-systems): Systems that detect and fix their own problems automatically, auto-scaling, auto-restart, and auto-remediation
- [Backup Strategies Overview](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/backup-strategies-overview): Compare full, incremental, differential, snapshot, and continuous backup approaches side by side
- [Geographic Redundancy](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/geographic-redundancy): Distribute systems across geographic regions to survive regional outages
- [Active-Passive Failover](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/active-passive-failover): One active instance serves traffic while a passive standby waits to take over on failure
- [Active-Active Failover](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/active-active-failover): Multiple active instances serve traffic simultaneously, no wasted standby capacity
- [DNS Failover](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/dns-failover): Route traffic away from failed endpoints using DNS health checks, simple but slow
- [Load Balancer Failover](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/load-balancer-failover): How load balancer failover works with a shared virtual IP, VRRP and keepalived. Also covers ALB health checks and failover across zones.
- [Database Failover](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/database-failover): How a replica is promoted to primary when a database fails, step by step. Covers RPO vs RTO, split brain, fencing, and the 2018 GitHub outage.
- [Circuit Breaker for Resilience](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/circuit-breaker-resilience): Stop calling failing services to prevent cascade failures, open, closed, and half-open states
- [Bulkhead Pattern](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/bulkhead-pattern): Isolate failures to prevent them from sinking the whole ship, resource isolation between components
- [Retry Patterns](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/retry-patterns): Retry failed operations intelligently, exponential backoff, jitter, and retry budgets
- [Timeout Patterns](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/timeout-patterns): Set appropriate timeouts for every external call, deadline propagation and timeout budgets
- [Graceful Degradation](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/graceful-degradation): Provide reduced functionality instead of complete failure, fallbacks, feature flags, and cached responses
- [Load Shedding](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/load-shedding): Deliberately reject excess traffic to protect core functionality, prioritize important requests
- [Backpressure for Resilience](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/backpressure-resilience): Signal upstream to slow down when overwhelmed, flow control that prevents cascading overload
- [Rate Limiting for Resilience](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/rate-limiting-resilience): Protect services from abuse and overload, token bucket, sliding window, and distributed rate limiting
- [Chaos Engineering](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/chaos-engineering): Intentionally inject failures to verify resilience. Chaos Monkey, Gremlin, and Litmus
- [Game Days](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/game-days): Scheduled exercises where teams practice responding to simulated incidents, building muscle memory
- [Capacity Planning](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/capacity-planning): Forecast resource needs before you run out, load testing, traffic modeling, and growth projection
- [Performance Budgets](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/performance-budgets): Set limits on latency, error rate, and resource usage. SLOs as engineering constraints
- [Error Budgets](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/error-budgets): The allowed amount of unreliability, when the budget is spent, freeze features and fix reliability
- [SLO Engineering](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/slo-engineering): Define and measure Service Level Objectives, the practice that aligns reliability with business needs
- [Toil Reduction](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/toil-reduction): Eliminate repetitive manual work through automation, the SRE mission to reduce operational burden
- [Canary Deployments](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/canary-deployments): Roll out changes to a small percentage of traffic first, detect problems before full deployment
- [Blue-Green Deployments](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/blue-green-deployments): Maintain two identical environments and switch traffic between them, instant rollback capability
- [Rolling Deployments](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/rolling-deployments): Update instances gradually, replace old with new one at a time to minimize risk
- [Feature Flags for Resilience](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/feature-flags-resilience): Toggle features on and off without deployment, kill switches for problematic features
- [Immutable Infrastructure](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/immutable-infrastructure): Never modify running servers, replace them entirely with new versions for consistency
- [Infrastructure as Code](https://www.systemdesign.academy/courses/advanced/lessons/reliability-resilience/infrastructure-as-code): Define infrastructure in version-controlled code. Terraform, Pulumi, and reproducible environments
#### Security Testing & Operations (advanced)
- [Code Reviews](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/code-reviews): Human review as the first line of security defense, catching vulnerabilities before they ship
- [Static Code Analysis](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/static-code-analysis): Automated tools that find bugs and vulnerabilities in source code without running it
- [SAST](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/sast): Static Application Security Testing, finding vulnerabilities in source code at build time
- [SCA](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/sca): Software Composition Analysis, finding known vulnerabilities in your dependencies
- [Vulnerability Scanning](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/vulnerability-scanning): Automated detection of known weaknesses in systems, networks, and applications
- [DAST](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/dast): Dynamic Application Security Testing, testing running applications for vulnerabilities by simulating attacks
- [Penetration Testing](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/penetration-testing): Simulated attacks by skilled humans who think like attackers, finding what automated tools miss
- [Red Team / Blue Team](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/red-team-blue-team): Adversarial security exercises, one team attacks, one defends, both learn
- [Bug Bounty Programs](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/bug-bounty-programs): How bug bounty programs pay outside researchers to find security bugs. Compares platforms, shows the report lifecycle, and covers when to launch one.
- [SIEM](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/siem): Security Information and Event Management, centralized security monitoring, detection, and investigation
- [IDS](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/ids): Intrusion Detection Systems, monitoring network traffic and system activity for malicious behavior
- [IPS](https://www.systemdesign.academy/courses/advanced/lessons/security-testing-operations/ips): Intrusion Prevention Systems, actively blocking detected attacks in real-time
#### Stream & Batch Processing (advanced)
- [Batch Processing](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/batch-processing): Process large volumes of data in scheduled jobs, the workhorse of data pipelines
- [Batch Analytics](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/batch-analytics): Run complex queries over massive datasets, data warehousing and OLAP fundamentals
- [ETL](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/etl): Extract, Transform, Load, moving and reshaping data between systems
- [ETL Pipeline](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/etl-pipeline): Orchestrating multi-step data transformations. Airflow, dbt, and pipeline design patterns
- [ELT](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/elt): Extract, Load, Transform, load raw data first, transform in the warehouse
- [Data Catalog](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/data-catalog): An inventory of all data assets in your organization, discovery, metadata, and governance
- [Metadata Management](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/metadata-management): Managing data about data, schemas, lineage, quality metrics, and ownership
- [Data Lineage](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/data-lineage): Tracing data from source to destination, know where every number in your dashboard came from
- [MapReduce](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/mapreduce): The programming model that launched big data, parallel processing across thousands of machines
- [Stream Processing](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/stream-processing): Process data as it arrives, real-time analytics, fraud detection, and event-driven architectures
- [Real-Time Analytics](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/real-time-analytics): How real-time analytics engines query fresh data in under a second. Shows how Pinot ingests from Kafka and compares Druid, ClickHouse and Rockset.
- [Stateless Stream Processing](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/stateless-stream-processing): Transform events independently, filtering, mapping, and routing without maintaining state
- [Micro-Batching](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/micro-batching): Process small batches at short intervals. Spark Structured Streaming approach to near-real-time
- [Window Functions](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/window-functions): Group streaming events by time or count, the fundamental abstraction for bounded stream computation
- [Tumbling Windows](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/tumbling-windows): Fixed-size, non-overlapping time windows, count events per minute, average per hour
- [Hopping Windows](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/hopping-windows): Fixed-size windows that overlap, sliding averages with configurable hop interval
- [Sliding Windows](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/sliding-windows): Windows that slide with every event, continuous recalculation over the most recent N events or T time
- [Session Windows](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/session-windows): Dynamic windows based on activity gaps, group events by user sessions with configurable timeouts
- [Time-Based Windows](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/time-based-windows): Windows defined by wall clock or event timestamps, the time semantics that determine correctness
- [Count-Based Windows](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/count-based-windows): Windows that close after N events regardless of time, useful for batch-size processing
- [Event Time vs Processing Time](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/event-time-vs-processing-time): The critical distinction between when an event occurred and when the system processes it
- [Watermarking](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/watermarking): Tracking progress of event time, how stream processors know they have seen all events up to a certain time
- [Late Data Handling](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/late-data-handling): What to do when events arrive after their window has closed, allowed lateness, side outputs, and retractions
- [Out-of-Order Processing](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/out-of-order-processing): Processing events that arrive in the wrong order, buffering, reordering, and eventual correctness
- [Stateful Stream Processing](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/stateful-stream-processing): Maintaining state across events, aggregations, joins, and pattern detection in streaming data
- [State Management in Streams](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/state-management-streams): Managing operator state in stream processors. RocksDB backends, state TTL, and state migration
- [Checkpointing Strategies](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/checkpointing-strategies): Aligned vs unaligned checkpoints, incremental snapshots, and checkpoint tuning for stream processors
- [Exactly-Once Semantics](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/exactly-once-semantics): Processing every event exactly once despite failures, the holy grail of stream processing
- [Stream Joins](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/stream-joins): Joining two streams, joining a stream with a table, and temporal joins, combining data in motion
- [Change Streams](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/change-streams): Capturing database changes as events. CDC with Debezium, MongoDB change streams, and PostgreSQL logical replication
- [Lambda Architecture](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/lambda-architecture): Batch plus speed layers for complete and fast results. Nathan Marz pattern and its trade-offs
- [Kappa Architecture](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/kappa-architecture): Everything through a single stream processing layer, simplifying Lambda by eliminating the batch layer
- [Data Mesh](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/data-mesh): Decentralized data ownership, domain-oriented, self-serve data infrastructure as a platform
- [Data Fabric](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/data-fabric): Unified data management across distributed environments, automated integration, governance, and delivery
- [Stream-to-Stream Joins](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/stream-to-stream-joins): Joining two unbounded event streams, windowed joins, interval joins, and temporal correlation
- [Stream-to-Table Joins](https://www.systemdesign.academy/courses/advanced/lessons/stream-batch-processing/stream-to-table-joins): Enriching stream events with reference data, lookup joins, broadcast state, and materialized views
#### Capstone (capstone)
- [Design a URL Shortener](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-url-shortener): Design a URL shortening service like Bitly from scratch - hashing, storage, redirection, and analytics at scale
- [Design a Rate Limiter](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-rate-limiter): Design a distributed rate limiting system - token bucket, sliding window, and protecting services at massive scale
- [Design a Notification System](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-notification-system): Design a multi-channel notification system - push, email, SMS with priority queues, fan-out, and delivery guarantees
- [Design a Chat System like WhatsApp](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-chat-system): System design of a chat app like WhatsApp. Covers WebSockets, message ordering, end-to-end encryption, presence and group chat, with Python code.
- [Design Twitter/X Feed](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-twitter-feed): Design a social media feed system - fan-out on write vs read, timeline service, the celebrity problem, and ranking at scale
- [Design YouTube/Netflix](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-youtube-netflix): System design of a video platform like YouTube or Netflix. Covers upload, transcoding, adaptive bitrate (HLS vs DASH), CDN and recommendations.
- [Design Google Search](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-google-search): Design a web search engine - crawling, inverted indexes, PageRank, query processing, and caching at global scale
- [Design Uber/Ride Sharing](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-uber-ride-sharing): Design a ride-sharing platform - geospatial indexing, driver matching, real-time tracking, ETA prediction, and surge pricing
- [Design a Payment System](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-payment-system): Design a payment processing system - ACID transactions, idempotency, reconciliation, retry strategies, and the saga pattern for distributed payments
- [Design an E-Commerce Platform](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-ecommerce-platform): Design an e-commerce system like Amazon - inventory management, cart, checkout, product search, recommendations, and flash sales
- [Design a File Storage System like Dropbox](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-file-storage): Design a cloud file storage system - chunking, deduplication, sync protocols, conflict resolution, and efficient storage at scale
- [Design a Metrics/Monitoring System](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-metrics-monitoring): Design a metrics collection and monitoring system - time-series databases, aggregation pipelines, alerting, and dashboards at scale
- [Design a Key-Value Store](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-key-value-store): Design a distributed key-value store - LSM trees, compaction, consistent hashing, replication, tunable consistency, and failure detection
- [Design a Distributed Task Scheduler](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-task-scheduler): System design of a distributed task scheduler. Covers job queues, cron jobs, exactly-once runs with leases, retries and dead letter queues.
- [Design an API Gateway](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-api-gateway): Design an API gateway - request routing, authentication, rate limiting, circuit breaker, load balancing, and observability
- [Design a RAG System at Scale](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-rag-system): System design of a production RAG system. Covers chunking, embeddings, vector search, hybrid retrieval, reranking, caching, evaluation and cost.
- [Design an AI Agent Platform](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-ai-agent-platform): Design a production AI agent orchestration platform - MCP, A2A protocol, tool calling, memory systems, multi-agent workflows, cost management, and safety guardrails
- [Design a Real-Time Analytics Dashboard](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-realtime-analytics): Design a real-time analytics system - stream processing, WebSocket-powered dashboards, aggregation pipelines, and windowed computations at scale
- [Design a Multi-Region Deployment](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-multi-region): Design a multi-region system - geo-replication, global server load balancing, consistency across regions, failover strategies, and data sovereignty
- [Design a Content Delivery Network](https://www.systemdesign.academy/courses/capstone/lessons/capstone/design-cdn): Design a CDN from scratch - edge caching, origin shield, cache purging, anycast routing, TLS termination, and global distribution
#### Caching Strategies (foundation)
- [Local Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/local-cache) [FREE, no signup]: The fastest cache you can have, and the only one that can disagree with itself
- [Application Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/application-cache) [FREE, no signup]: The cache your own code manages, the race hiding in its default pattern, and how often that race actually fires
- [Browser Caching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/browser-caching) [FREE, no signup]: The only cache with no purge, the only one that can answer with no network, and why revalidation gives up the thing that makes it valuable
- [Client-Side Caching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/client-side-caching) [FREE, no signup]: A cache your code keeps on the user's device, why the usual objection to it is measurably wrong, and what actually limits it
- [Time To Live (TTL)](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/time-to-live-ttl) [FREE, no signup]: How to choose a cache TTL from your own traffic. A sweep of seven TTL values on skewed traffic shows the hit rate ceiling and the cost of staleness.
- [Memoization](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/memoization) [FREE, no signup]: The one cache that cannot serve a wrong answer, what its lookup actually costs, and why its table is never freed
- [Distributed Caching with Redis and Memcached](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/application-level-caching) [FREE, no signup]: A shared cache is a network service, so how you call it matters far more than which product you picked
- [Server-Side Caching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/server-side-caching) [FREE, no signup]: Caching the finished response removes all the work, and one personalised element removes all the benefit
- [Database Query Caching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/database-query-caching) [FREE, no signup]: Your database already cached the disk, so what a query cache saves is CPU, and that varies by 430 times between queries
- [Result Set Caching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/result-set-caching) [FREE, no signup]: Caching a result set freezes it, and the deep pagination it is usually bought to fix disappears when you change the question instead
- [Cache Eviction Policies](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-eviction-policies) [FREE, no signup]: LFU beat LRU by 6 to 15 points, FIFO measured identical to random, and the scan that supposedly destroys LRU cost it 11 points for four hundred requests
- [Cache Invalidation](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-invalidation) [FREE, no signup]: Careful per-row invalidation served more than twice the stale reads of doing nothing, because it raised the hit rate on the entries it cannot see
- [Lazy Loading](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/lazy-loading): A self-filling cache fills itself out of users. Measured on a cold start, 13,259 of them paid the slow path, and warming the 200 most popular keys removed exactly 200
- [Eager Loading](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/eager-loading): Warming a cache has three decisions, not one. Measured, the benefit peaked at exactly the cache's capacity, and loading the same keys in the opposite order was worth 6.7 points
- [Prefetching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/prefetching): Prefetching one page ahead took the p95 from 151 ms to 124. Two pages ahead took it to 991, with the same hit rate, because a prefetch is a real request in a shared queue
- [Predictive Prefetching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/predictive-prefetching): The same one-line rule scored 48.8 percent on one site and zero on another, while the same learned model scored 50.2 on both. A model is worth exactly the distance from the rule you would have written
- [Cache Warming](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-warming): A cold instance in a warm fleet served its users at 8.14 ms while their neighbours got 3.69, and the fleet average moved by 0.68 ms. The largest fix was not a warming job
- [Cache-Aside Pattern](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-aside-pattern): Cache-aside is said to degrade gracefully when the cache fails. Measured, unwrapped it answered 0.0 percent of requests, and wrapped it answered 13.0, which is exactly the database's capacity
- [Read-Through Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/read-through-cache): Read-through removes key drift, shape drift and the stampede by construction, and hides one decision. A three second source failure was served as a wrong answer for 297 seconds, with a flat error rate
- [Write-Through Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/write-through-cache): Write-through took the read hit rate from 71.5 percent to 68.3 on uniform writes, and from 59.5 to 71.9 on correlated ones. It is a bet on whether your writes predict your reads
- [Write-Back Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/write-back-cache): How a write-back cache holds writes and flushes them later. Measured tests show how the flush interval trades fewer database writes for data at risk.
- [Refresh-Ahead Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/refresh-ahead-cache): Refresh-ahead reloads cache entries before they expire. A measured test shows most stalls were evictions it cannot fix, and a longer TTL did better.
- [Cache Stampede Prevention](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/cache-stampede-prevention): Two problems share the name. Single flight took 306,687 loads down to 81,233 and did nothing to the spike. TTL jitter took the peak second from 506 to 339 and removed no work at all
- [Remote Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/remote-cache): Measured against a real Redis on loopback, a hit costs 0.056 ms against 0.00004 for a local one. Work cheaper than that is made slower by caching it, and at 232 KB the parse was 94 percent of the hit
- [Edge Caching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/edge-caching): The same 200 URLs became 200, 501, 7,628 or 18,404 cache entries under four values of one response header, and the origin went from 200 requests to 18,706
- [Fragment Caching](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/fragment-caching): A whole-page cache on a partly personal page measured identically to no cache at all. Fragmenting skipped 93.1 percent of renders, and the page-level hit rate fell to 6.1 percent at the same time
- [Distributed Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/distributed-cache): How a distributed cache spreads keys across nodes with consistent hashing and virtual nodes. Measured tests show why a hot key still overloads a node.
- [Redis Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/redis-cache): The same small GET measured 0.3 ms at worst on an idle server and 59.1 ms while one other client ran KEYS. Reading one field cost 0.934 ms as a blob and 0.040 ms as a hash
- [Memcached](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/memcached): The same big read multiplied a small GET by 1.3 on memcached and by 32 on Redis, from an identical 0.021 ms idle. Ten of sixteen pages stayed with a slab class nobody was using
- [Varnish Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/varnish-cache): The hit rate was 0.0 percent as the framework ships and 99.0 percent with one response header removed. At 80 percent of visitors cookied, 15,913 of 20,000 requests skipped the cache
- [Hazelcast](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/hazelcast): How Hazelcast spreads a map across app processes. Measured tests show only 1 read in N is local, and what a near cache saves and what it costs.
- [Apache Ignite](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/apache-ignite): How Apache Ignite runs SQL on a partitioned cache. Measured tests show a query with the key stays fast, and one without it slows as the cluster grows.
- [Oracle Coherence](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/coherence): How Oracle Coherence entry processors send code to the data. A measured test shows read-modify-write losing updates that the atomic form keeps.
- [Ehcache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/ehcache): How Ehcache works as a cache inside one process, with storage tiers. Measured tests show the cost of each tier and the race a local cache cannot fix.
- [Caffeine Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/caffeine-cache): A batch job taking 30 percent of requests cost LRU 6.3 points of hit rate on the real requests, admission by approximate frequency 4.2, and exact counting 0.8, in 2,048 bytes
- [Guava Cache](https://www.systemdesign.academy/courses/foundation/lessons/caching-strategies/guava-cache): Segmented eviction, the thing everybody warns about, cost 0.0 points at Guava's default. Lazy maintenance, which nobody mentions, left 5,000 expired entries in memory and one read at 20x the median
#### Core Fundamentals (foundation)
- [Latency](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/latency) [FREE, no signup]: Time delay in system response - most basic performance metric
- [Throughput](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/throughput) [FREE, no signup]: How much work a system can handle in a given time period
- [Bandwidth](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/bandwidth) [FREE, no signup]: Maximum data transfer capacity of a network connection
- [Synchronous Processing](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/synchronous-processing) [FREE, no signup]: Sequential request-response pattern where each operation waits for the previous one
- [Asynchronous Processing](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/asynchronous-processing) [FREE, no signup]: Non-blocking concurrent operations that don't wait for each other
- [JSON Schema](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/json-schema): JSON Schema describes what valid JSON looks like, so an API can reject bad data. Shows types, required fields, nested objects and arrays.
- [XML Schema](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/xml-schema): Formal validation rules for XML document structure and data types
- [Input Validation](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/input-validation): Verifying incoming data integrity before processing
- [Output Encoding](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/output-encoding): Securing outgoing data by encoding it for the correct context
- [REST API](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/rest-api): Standard web service architecture using HTTP methods and resource-based URLs
- [API Documentation (Swagger/OpenAPI)](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/api-documentation): How API specification standards like OpenAPI let teams build, test, and consume APIs without guessing
- [Semantic Versioning](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/semantic-versioning): Version numbering conventions that communicate the nature of changes in software
- [Connection Timeout](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/connection-timeout): Maximum time a client waits to establish a connection before giving up
- [Request Timeout](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/request-timeout): A request timeout limits how long a client waits for a full response. Learn how to pick the value, use timeout budgets, and retry without a storm.
- [Session Management](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/session-management): How servers remember who you are between requests in a stateless protocol
- [Stateless vs Stateful Systems](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/stateless-vs-stateful): Two fundamental architecture patterns that shape how systems handle data, scale, and recover from failure
- [Idempotency](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/idempotency): Why some operations are safe to retry and others charge you twice, and how to make everything safe
- [Scalability](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/scalability): How systems grow to handle more users, more data, and more traffic without falling apart
- [Vertical Scaling](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/vertical-scaling): Adding resources to a single node to handle more load
- [Horizontal Scaling](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/horizontal-scaling): Adding more nodes to a system to distribute load
- [Elasticity](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/elasticity): Dynamic resource adjustment based on real-time demand
- [Load Balancing](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/load-balancing): How traffic actually gets spread across many servers, and what breaks when it does not
- [Caching](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/caching): Temporary data storage for faster access and reduced load
- [ACID Properties](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/acid-properties): Database transaction guarantees that keep your data correct
- [BASE Properties](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/base-properties): Distributed system consistency model that trades strong consistency for availability
- [SQL vs NoSQL](https://www.systemdesign.academy/courses/foundation/lessons/core-fundamentals/sql-vs-nosql): Database model comparison, relational vs non-relational data stores
#### Database Fundamentals (foundation)
- [Database Views](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-views): A view and the query it is defined as produced identical plans, character for character. A materialised view answered in 0.017 ms what took 94.929 ms to compute, and is a cache with every problem that implies
- [Database Functions](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-functions): The identical function body took 143.874 ms declared VOLATILE and 0.078 ms declared STABLE, which is 1,845 times. The default is VOLATILE, and nothing validates any of these claims
- [Soft Delete](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/soft-delete): At seventy percent deleted the table with its indexes was 161.9 MB against 33.5 for the same live rows. A unique constraint refused an address a long-deleted row still holds
- [Hard Delete](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/hard-delete): Deleting 507,591 rows changed the table size by 0.0 MB, and VACUUM changed it by nothing either. An unindexed foreign key made 100 parent deletes take 1,011 ms against 8
- [Database Indexing](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-indexing): The planner stopped using an index somewhere between 19.7 and 39.8 percent of the table. A query on the second column of a composite index took 37.269 ms against 0.054, and five indexes took the same inserts from 160 ms to 751
- [Database Normalization](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-normalization): Renaming one category rewrote 200,000 rows in a flat table and 1 in a normalised one. A partial update left the flat table holding two answers to one question, and the join it costs measured 1.34 times
- [Database Denormalization](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-denormalization): A precomputed aggregate answered in 0.024 ms what took 107.729 ms to compute, 4,489 times. Maintaining it cost 3.4 times on every insert, and doing the maintenance the obvious way lost 391 of 480 concurrent increments
- [B-Trees](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/b-trees): Ten million rows sat in three levels and a hundred rows in one. The same million keys took 21.4 MB inserted in order and 29.6 inserted at random, and after churn the index was 21.92 percent full
- [Query Optimization](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/query-optimization): The planner predicted 1,632 rows where 33,334 were waiting, and that estimate is what chooses the join. Two plans touched 32 buffers and 16,667, and one NULL took a NOT IN query from 95,103 rows to zero
- [Stored Procedures](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/stored-procedures): Three thousand updates took 117 ms sent one at a time and 17 inside one CALL. A function that tries to COMMIT fails outright, and an EXCEPTION block inside a loop took 9 ms to 134 while catching nothing
- [Database Triggers](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-triggers): Three triggers created in one order fired in another, three rows were inserted and two exist, and a TRUNCATE emptied an audited table without firing the audit trigger once. None of it raised an error
- [Materialized Views](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/materialized-views): A plain REFRESH made a 7 ms read take 1,231 ms, because it holds an exclusive lock for its whole duration. The concurrent form cost 3 percent more and blocked nobody, and refuses to run without a unique index
- [Database Connection Pooling](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-connection-pooling): PgBouncer, HikariCP, and why opening a new database connection per request is a recipe for disaster
- [Read Replicas](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/read-replicas): Scaling reads by replicating data to follower databases, primary-replica architecture and replication lag
- [Versioned Data](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/versioned-data): Tracking every change to a record, audit trails, undo history, and time-travel queries
- [Temporal Tables](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/temporal-tables): Database-native time-travel queries, system-versioned tables, valid-time, and bi-temporal modeling
- [Tombstoning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/tombstoning): Why databases don't actually delete your data right away, and what they leave behind instead
- [Index Rebuilding](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/index-rebuilding): When your database indexes rot and queries slow to a crawl, how to fix it without taking the system down
- [Vacuum](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/vacuum): PostgreSQL's garbage collector, why dead rows pile up and how VACUUM reclaims them
- [Compaction](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/compaction): How LSM-tree databases like Cassandra, RocksDB, and LevelDB clean up their mess, merging, sorting, and reclaiming space
- [Database Replication](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-replication): Keeping copies of your data on multiple servers, the foundation of every highly available system
- [Master-Slave Replication](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/master-slave-replication): The most common replication topology, one writer, many readers, and the trade-offs that come with it
- [Master-Master Replication](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/master-master-replication): When one writer isn't enough, multiple primaries, write conflicts, and the complexity tax you pay for global writes
- [Table Partitioning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/table-partitioning): Splitting a massive table into smaller, manageable pieces, without your application knowing the difference
- [Vertical Partitioning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/vertical-partitioning): Vertical partitioning splits a wide table into hot and cold columns. Shows a users table before and after, and compares it to horizontal partitioning.
- [Horizontal Partitioning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/horizontal-partitioning): Splitting a table by rows, the partitioning strategy behind every large-scale database deployment
- [Range Partitioning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/range-partitioning): Partitioning by contiguous value ranges, the go-to strategy for time-series data, logs, and anything with a natural ordering
- [Hash Partitioning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/hash-partitioning): Using hash functions to distribute rows evenly across partitions, the antidote to hotspots and skewed data
- [List Partitioning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/list-partitioning): Partitioning by explicit value lists, perfect for categorical data like regions, statuses, and tenant IDs
- [Composite Partitioning](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/composite-partitioning): Combining multiple partitioning strategies, range + hash, list + range, and when the extra complexity is worth it
- [Database Sharding](https://www.systemdesign.academy/courses/foundation/lessons/database-fundamentals/database-sharding): Splitting a database across multiple servers, the nuclear option for scaling writes, and the architecture behind Instagram, Uber, and YouTube
#### Load Balancing & Proxies (foundation)
- [DNS (Domain Name System)](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/dns) [FREE, no signup]: The internet's phone book, how domain names become IP addresses
- [Unicast Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/unicast-routing): One-to-one communication, the default routing model of the internet
- [Broadcast](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/broadcast): One-to-all communication, sending data to every device on a network
- [Multicast](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/multicast): One-to-many communication, efficiently sending data to a group of interested receivers
- [Anycast Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/anycast-routing): One-to-nearest routing, how the same IP address serves users from hundreds of locations
- [Forward Proxy](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/forward-proxy): What a forward proxy is, what changes on the wire, and why companies run one. Also compares forward and reverse proxies and tests the anonymity claim.
- [Reverse Proxy](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/reverse-proxy): An intermediary that receives requests on behalf of servers, for TLS, routing, balancing, caching and protection
- [Load Balancer Algorithms](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/load-balancer-algorithms): How load balancers decide which server gets the next request. Round Robin, Weighted, Least Connections, IP Hash, and more
- [DNS Load Balancing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/dns-load-balancing): Using DNS to distribute traffic across servers, the simplest form of global load balancing
- [Sticky Sessions](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/sticky-sessions): Pinning users to the same server, when stateless just isn't possible
- [Session Affinity](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/session-affinity): Where the session actually lives, and why that one choice decides routing, deploys, scaling and failover
- [Affinity Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/affinity-routing): Routing on a key that names whose data a request is for, so the request lands where that data already is
- [Path-Based Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/path-based-routing): How path-based routing sends parts of one URL to different backends. Covers which rule wins, prefix stripping, and why it is hard to debug.
- [Host-Based Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/host-based-routing): Routing on the hostname, which is also the browser's origin, arrives twice under TLS, and costs a DNS record and a certificate every time
- [Header-Based Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/header-based-routing): Routing on the one field the user cannot see, who is allowed to write it, and what it does to every cache in front of you
- [Weighted Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/weighted-routing): Splitting traffic by proportion, why a percentage of requests is not a percentage of users, and what a small canary can actually tell you
- [Geolocation Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/geolocation-routing): Routing on where a client appears to be, why speed and data residency are two different problems, and how wrong the inference actually is
- [Latency-Based Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/latency-based-routing): Latency-based routing sends users to the fastest region by measuring it. Learn why acting on the measurement changes it, and how to stop the swings.
- [Failover Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/failover-routing): Using a primary until something says it is broken, why the detection is the whole problem, and why the standby is usually broken too
- [Multi-Value Routing](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/multi-value-routing): Handing the client several addresses and letting it choose, what clients actually do with the list, and why the load spreading is mostly folklore
- [Traffic Splitting](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/traffic-splitting): Running two versions at once, everything they have to share, and why the weight change is the easiest part of it
- [Traffic Mirroring](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/traffic-mirroring): Sending a copy of real traffic to a new version and discarding the response, and the two costs that are not discarded with it
- [Connection Draining](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/connection-draining): Removing an instance on purpose, the race between the pool update and the shutdown, and what in-flight leaves out
- [API Gateway](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/api-gateway): Every mechanism in this chapter running in one component in front of everything, what that concentration costs, and how to keep it from becoming an application
- [Global Server Load Balancing (GSLB)](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/global-server-load-balancing): How a GSLB decides which region a user reaches, and why the decision takes minutes to arrive even when it is made in microseconds
- [Content Delivery Network (CDN)](https://www.systemdesign.academy/courses/foundation/lessons/load-balancing-proxies/content-delivery-network): A CDN is thousands of caches you do not operate, and the cache key decides everything about whether it helps you
#### API Design & Protocols (intermediate)
- [API Keys](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/api-keys): The simplest way to identify and authenticate API consumers, a shared secret with limits
- [CORS](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/cors): Cross-Origin Resource Sharing, the browser security model that blocks your API calls and how to fix it
- [API Versioning](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/api-versioning): Evolve your API without breaking existing consumers, strategies for managing change
- [Rate Limiting](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/rate-limiting): Protect your API from abuse and overload by controlling how many requests each consumer can make
- [Throttling](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/throttling): Slow down instead of shut down, degrade gracefully when your system is under pressure
- [Request Batching](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/request-batching): Combine multiple API calls into one request to reduce overhead and improve efficiency
- [Response Batching](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/response-batching): Collect multiple server responses and deliver them together to reduce round trips
- [Keep-Alive Connections](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/keep-alive-connections): Reuse TCP connections across multiple requests to eliminate handshake overhead
- [Connection Reuse](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/connection-reuse): Pool and reuse connections across your application to maximize throughput and minimize overhead
- [Idempotency Keys](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/idempotency-keys): Safely retry failed API requests without causing duplicate side effects
- [Token Bucket Algorithm](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/token-bucket-algorithm): A bucket of tokens that refills at a steady rate, the most popular rate limiting algorithm in production
- [Leaky Bucket Algorithm](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/leaky-bucket-algorithm): Requests drip out at a constant rate, smooth traffic shaping for APIs and networks
- [Sliding Window](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/sliding-window): Rate limit with a moving time window, smoother than fixed windows and more accurate
- [API Rate Limiting](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/api-rate-limiting): Putting it all together, designing rate limiting for production APIs at scale
- [Quota Management](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/quota-management): Manage long-term usage limits, daily quotas, monthly caps, and tiered access
- [SOAP](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/soap): The enterprise protocol that dominated before REST. XML-based, strictly typed, and still running critical infrastructure
- [GraphQL](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/graphql): Ask for exactly what you need, nothing more. The query language that replaced dozens of REST endpoints
- [GraphQL Schema](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/graphql-schema): Design type-safe API contracts with GraphQL's schema definition language
- [gRPC](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/grpc): Google's high-performance RPC framework, binary protocols, HTTP/2 streaming, and auto-generated clients
- [Protocol Buffers](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/protocol-buffers): Google's language-neutral binary serialization, smaller, faster, and strictly typed
- [Apache Avro](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/apache-avro): Schema-evolution-friendly binary serialization, the serialization format built for big data
- [Apache Thrift](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/apache-thrift): Apache Thrift is a cross-language RPC framework built at Facebook. Learn its IDL and code generation, and see how it compares with gRPC and Protobuf.
- [MessagePack](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/messagepack): JSON but binary, a compact serialization format that is schema-less and fast
- [HTTP/2 Multiplexing](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/http2-multiplexing): Multiple requests over one connection, how HTTP/2 eliminated head-of-line blocking at the application layer
- [HTTP/3 QUIC](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/http3-quic): The next generation of HTTP, built on UDP with QUIC for zero round-trip connections and true multiplexing
- [Schema Registry](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/schema-registry): A central repository for API schemas, enforce compatibility, enable evolution, prevent breaking changes
- [API Composition](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/api-composition): Aggregate data from multiple microservices into a single response at the API layer
- [Backend for Frontend (BFF)](https://www.systemdesign.academy/courses/intermediate/lessons/api-design-protocols/backend-for-frontend): A dedicated backend for each frontend, tailor your API to each client's needs
#### Cloud Infrastructure (intermediate)
- [Static IP](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/static-ip): A permanent address for your server that never changes, the foundation of reliable networking
- [Dynamic IP](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/dynamic-ip): Automatically assigned addresses that change, how most devices get online
- [IPv4 vs IPv6](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/ipv4-vs-ipv6): The old address system is full, the new one has 340 undecillion addresses, here's what that means
- [DHCP](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/dhcp): The protocol that automatically assigns IP addresses to every device on a network
- [TCP/IP Stack](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/tcp-ip-stack): The four-layer model that powers every internet connection, from electrical signals to web pages
- [OSI Model](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/osi-model): The seven-layer reference model everyone talks about but nobody actually implements
- [UDP](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/udp): The 'fire and forget' transport protocol that prioritizes speed over reliability
- [CIDR](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/cidr): Classless Inter-Domain Routing, the notation system that replaced wasteful IP address allocation
- [NAT](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/nat): Network Address Translation, how millions of devices share a single public IP address
- [Subnets](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/subnets): Dividing a network into isolated segments for security, organization, and efficiency
- [VPC](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/vpc): Virtual Private Cloud, your own isolated network in someone else's data center
- [Internet Gateway](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/internet-gateway): The door between your private cloud network and the public internet
- [NAT Gateway](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/nat-gateway): Managed NAT for private subnets, outbound internet without inbound exposure
- [Elastic IP](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/elastic-ip): A static public IP you own and can reassign between cloud resources instantly
- [IPAM](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/ipam): IP Address Management, tracking and governing IP allocation across your entire cloud estate
- [VPN](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/vpn): Virtual Private Networks, encrypted tunnels that extend your private network across the public internet
- [Client VPN](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/client-vpn): Connecting individual users to a private cloud network from anywhere in the world
- [Site-to-Site VPN](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/site-to-site-vpn): Permanently connecting your office network to your cloud VPC over encrypted tunnels
- [Regions](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/regions): Geographic locations where cloud providers operate data centers, the first architectural decision
- [Availability Zones](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/availability-zones): Isolated data centers within a region, the building blocks of high availability
- [Network Segmentation](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/network-segmentation): Dividing networks into isolated zones to contain breaches and enforce access policies
- [IaaS](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/iaas): Infrastructure as a Service, renting virtual machines, storage, and networks instead of buying hardware
- [PaaS](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/paas): Platform as a Service, deploy code without managing servers, OS, or infrastructure
- [SaaS](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/saas): Software as a Service, ready-to-use applications delivered over the internet
- [CaaS](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/caas): Containers as a Service, run Docker containers without managing the underlying cluster
- [DBaaS](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/dbaas): Database as a Service, managed databases that handle backups, scaling, and patching for you
- [Serverless](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/serverless): The execution model where you write functions and the cloud handles everything else, scaling to zero and to millions
- [FaaS](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/faas): Functions as a Service, the compute primitive behind serverless, where each function is an independent unit of deployment
- [BaaS](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/baas): Backend as a Service, pre-built backend components so you don't have to build auth, storage, or databases from scratch
- [Cold Start](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/cold-start): The latency penalty when serverless functions wake up, what causes it, how to measure it, and how to minimize it
- [Warm Instances](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/warm-instances): Pre-initialized execution environments that eliminate cold start latency at the cost of idle compute
- [DMZ](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/dmz): The demilitarized zone, a network buffer between the public internet and your private infrastructure
- [Bastion Host](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/bastion-host): A hardened gateway server that provides controlled SSH access to private infrastructure
- [Jump Server](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/jump-server): An intermediary server for accessing isolated network segments, the enterprise evolution of the bastion host
- [VPC Peering](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/vpc-peering): Connecting two VPCs so their resources can communicate using private IP addresses
- [Transit Gateway](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/transit-gateway): A central hub that connects hundreds of VPCs and on-premises networks without N-squared peering
- [Direct Connect](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/direct-connect): A dedicated physical network link between your data center and AWS, bypassing the internet entirely
- [Private Link](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/private-link): Accessing services over private IP addresses without traversing the internet, even across accounts and providers
- [Edge Computing](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/edge-computing): Running computation at the network edge, closer to users, instead of in centralized cloud data centers
- [Fog Computing](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/fog-computing): A distributed computing layer between edge devices and the cloud, processing data at intermediate nodes
- [TCP Optimization](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/tcp-optimization): Tuning TCP parameters and techniques to maximize throughput and minimize latency for cloud workloads
- [Network Latency Optimization](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/network-latency-optimization): Practical ways to reduce network latency in cloud systems, from region choice and CDNs to protocol tuning. Includes a decision tree to pick a fix.
- [QUIC](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/quic): The protocol replacing TCP for the modern web, built on UDP with encryption, multiplexing, and zero-RTT built in
- [WebRTC](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/webrtc): Real-time peer-to-peer communication in the browser, the technology behind video calls, screen sharing, and live streaming
- [Service Mesh](https://www.systemdesign.academy/courses/intermediate/lessons/cloud-infrastructure/service-mesh): A dedicated infrastructure layer for managing service-to-service communication, observability, security, and traffic control without changing application code
#### Data Governance & Compliance (intermediate)
- [Data Validation](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-validation): Verify that data meets expected rules before it enters your system, the first line of defense against garbage data
- [Data Filtering](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-filtering): Select subsets of data that match specific criteria, the WHERE clause of every data operation
- [Data Sorting](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-sorting): Order data by specific criteria, newest first, highest price, alphabetical, or any custom ranking
- [Data Grouping](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-grouping): Aggregate records into meaningful groups, the foundation of reports, dashboards, and analytics
- [Schema Validation](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/schema-validation): Define the expected shape of your data with schemas, and reject anything that does not conform
- [Data Transformation](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-transformation): Convert data from one format, structure, or representation to another, the glue between incompatible systems
- [Data Aggregation](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-aggregation): Combine data from multiple sources into a unified view, building the single source of truth
- [Data Sampling](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-sampling): Analyze a representative subset instead of the full dataset, when processing everything is too slow or too expensive
- [Data Quality](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-quality): Measure and maintain the accuracy, completeness, and reliability of your data, because bad data leads to bad decisions
- [Data Profiling](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-profiling): Analyze your data to understand its structure, patterns, and anomalies before using or migrating it
- [Data Cleansing](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-cleansing): Fix, standardize, and repair dirty data, turning messy real-world inputs into reliable records
- [Data Enrichment](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-enrichment): Enhance your data by adding information from external sources, turning thin records into rich profiles
- [Audit Logging](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/audit-logging): Record who did what, when, and why, the immutable trail that compliance, security, and debugging all depend on
- [Data Retention Policies](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-retention-policies): Define how long data is kept, when it's archived, and when it's deleted, balancing compliance, cost, and utility
- [Data Lifecycle Management](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-lifecycle-management): Manage data from creation to destruction, tiered storage, archival, and the economics of keeping data alive
- [Data Privacy](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-privacy): Protect personal information by design, privacy isn't a feature you add later, it's an architecture decision
- [Data Masking](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-masking): Hide sensitive data while preserving its format and usability, the art of showing enough without showing too much
- [Data Anonymization](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-anonymization): Data anonymization compared: k-anonymity, l-diversity and differential privacy. Includes a hashing test that shows why hashed IDs can be reversed.
- [Data Pseudonymization](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-pseudonymization): Data pseudonymization swaps names and emails for tokens that a key can reverse. Compares five techniques, plus anonymization and encryption.
- [PII Handling](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/pii-handling): Identify, classify, protect, and manage Personally Identifiable Information across your entire system
- [GDPR Compliance](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/gdpr-compliance): The engineer's guide to GDPR: understand the regulation that changed how every tech company handles personal data
- [Compliance Monitoring](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/compliance-monitoring): Continuously verify that your systems meet regulatory requirements, because compliance isn't a one-time checkbox
- [Data Governance Framework](https://www.systemdesign.academy/courses/intermediate/lessons/data-governance-compliance/data-governance-framework): The organizational framework for managing data as a strategic asset: ownership, standards, policies, and accountability
#### Data Replication & Distribution (intermediate)
- [Data Partitioning](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/data-partitioning): Split massive datasets across multiple nodes so no single machine drowns
- [Data Locality](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/data-locality): Store data close to where it's accessed to slash latency and boost throughput
- [Data Locality Optimization](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/data-locality-optimization): Practical techniques to improve data locality at every layer of your stack
- [Replication Lag](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/replication-lag): The delay between a write on the primary and its appearance on replicas, and why it haunts distributed systems
- [Asynchronous Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/asynchronous-replication): The default replication mode, fast writes at the cost of potential data loss
- [Synchronous Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/synchronous-replication): Synchronous replication waits for a replica to confirm each write, so no data is lost on failover. Compares sync, semi-sync and async replication.
- [Semi-Synchronous Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/semi-synchronous-replication): The pragmatic middle ground, wait for one replica, let the rest catch up
- [Logical Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/logical-replication): Replicate data as logical operations, flexible, cross-version, and selective
- [Physical Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/physical-replication): Byte-for-byte WAL shipping, the fastest and simplest way to keep an exact database copy
- [Streaming Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/streaming-replication): Continuous real-time WAL streaming, no waiting for files to fill up
- [Point-to-Point](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/point-to-point): The simplest replication topology, one primary directly connected to one replica
- [Star Topology](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/star-topology): One primary, many replicas, the most common production replication topology
- [Ring Topology](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/ring-topology): Nodes arranged in a circle, each replicating to the next, clean but fragile
- [Hub-and-Spoke](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/hub-and-spoke): A central hub coordinates replication to regional spokes, the enterprise classic
- [Multi-Master Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/multi-master-replication): Multi-master replication lets several nodes accept writes. See how it works, a concrete example of a write conflict, and when it is worth the cost.
- [Bidirectional Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/bidirectional-replication): Two nodes replicate to each other, the simplest form of multi-master
- [Data Synchronization](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/data-synchronization): Keeping multiple data stores consistent when they inevitably drift apart
- [Change Data Capture (CDC)](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/change-data-capture): Capture every database change as a stream of events, the backbone of modern data pipelines
- [Binlog Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/binlog-replication): MySQL's binary log, the engine behind MySQL replication and CDC
- [Write-Ahead Log Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/write-ahead-log-replication): Postgres WAL, the foundation of crash recovery, replication, and point-in-time restore
- [Log Shipping](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/log-shipping): Transfer complete log files from primary to standby, the oldest and simplest replication method
- [Consistent Hashing](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/consistent-hashing): The algorithm that makes adding or removing nodes cheap, an interview favorite and production essential
- [Geographic Distribution](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/geographic-distribution): Spread data across physical locations to reduce latency and survive regional disasters
- [Active-Passive Configuration](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/active-passive-configuration): How active-passive high availability works, with a second-by-second failover flow. Also compares active-passive with active-active setups.
- [Active-Active Configuration](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/active-active-configuration): Both nodes serve traffic simultaneously, maximum utilization, maximum complexity
- [Cross-Region Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/cross-region-replication): How cross-region replication copies data between cloud regions for disaster recovery. Compares sync and async, and shows how lag grows with distance.
- [Geo-Replication](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/geo-replication): Replicate data to geographically distributed nodes for compliance, performance, and resilience
- [Multi-Region Deployment](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/multi-region-deployment): Deploy your entire stack across multiple regions, the full picture
- [Mesh Topology](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/mesh-topology): Every node connects to every other node, maximum redundancy, maximum connections
- [Many-to-Many](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/many-to-many): Multiple sources replicating to multiple destinations, the most flexible and complex pattern
- [Replication Topology](https://www.systemdesign.academy/courses/intermediate/lessons/data-replication-distribution/replication-topology): A decision framework for choosing the right replication topology for your system
#### Database Types & Storage (intermediate)
- [File Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/file-storage): The oldest and most intuitive storage model, hierarchical directories and files
- [Block Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/block-storage): What block storage is and why databases like PostgreSQL run on it, for example on AWS EBS. Also explains how it differs from file and object storage.
- [Object Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/object-storage): The storage model that powers the modern internet, flat, scalable, and dirt cheap
- [Blob Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/blob-storage): Binary Large Objects, the raw bytes behind media, backups, and unstructured data
- [Key-Value Stores](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/key-value-stores): How key-value databases work, how to design good keys, and how Redis, DynamoDB and etcd compare. Includes a diagram of a GET and SET operation.
- [Document Stores](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/document-stores): Databases that think in JSON, flexible schemas for modern applications
- [Row-Oriented Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/row-oriented-storage): How traditional databases store data row by row, and why that matters for your queries
- [Columnar Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/columnar-storage): Store data by column instead of by row, the secret behind modern analytics engines
- [Column-Family Stores](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/column-family-stores): Wide-column databases like Cassandra and HBase, built for massive scale with flexible schemas
- [Time-Series Databases](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/time-series-databases): Databases built for timestamped data, metrics, IoT sensors, financial ticks, and observability
- [Graph Databases](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/graph-databases): Databases that store relationships as native citizens, powering social networks, fraud detection, and recommendation engines
- [In-Memory Databases](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/in-memory-databases): Databases that live entirely in RAM, trading durability for speed measured in microseconds
- [Full-Text Search](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/full-text-search): How search engines find documents by meaning, not just exact match, tokenization, stemming, and relevance scoring
- [Search Engines (Elasticsearch, Solr)](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/search-engines): Distributed search engines that power product search, log analysis, and real-time analytics at massive scale
- [Inverted Index](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/inverted-index): The data structure powering every search engine, mapping words to the documents that contain them
- [Forward Index](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/forward-index): What a forward index stores and how search engines use it with the inverted index for BM25 scoring. Also shows how Lucene and Elasticsearch store it.
- [Bitmap Index](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/bitmap-index): Bit arrays that make low-cardinality queries blazing fast, the secret weapon of data warehouses
- [Geospatial Indexing](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/geospatial-indexing): How databases efficiently answer 'find everything near me'. R-trees, geohashes, and spatial queries
- [Data Warehousing](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/data-warehousing): Centralized repositories optimized for analytical queries, how companies turn raw data into business decisions
- [Data Lakes](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/data-lake): Store everything in its raw form, structured, semi-structured, and unstructured data in one massive repository
- [Hot-Warm-Cold Architecture](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/hot-warm-cold-architecture): Organize data by access frequency to balance performance and cost
- [Tiered Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/tiered-storage): Automatically move data between storage tiers based on policies and access patterns
- [Cold Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/cold-storage): Store rarely-accessed data at minimal cost with acceptable retrieval delays
- [Archive Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/archive-storage): The cheapest storage tier for data you must keep but almost never need to access
- [Hybrid Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/hybrid-storage): Combine on-premises and cloud storage for flexibility, compliance, and cost control
- [Bloom Filters](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/bloom-filters): A space-efficient probabilistic data structure that tells you if an element is definitely NOT in a set
- [Skip Lists](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/skip-lists): A probabilistic data structure that provides O(log n) search with the simplicity of a linked list
- [LSM Trees](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/lsm-trees): How LSM trees turn random writes into fast sequential writes. Covers the write path, levels, size-tiered vs leveled compaction, and the read cost.
- [Merkle Trees](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/merkle-trees): Hash trees that verify data integrity and enable efficient synchronization, powering Git, blockchains, and distributed databases
- [Vector Database](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/vector-database): Databases built for AI, storing and searching high-dimensional embeddings that power semantic search, RAG, and recommendation engines
- [Embedding Storage](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/embedding-storage): How to efficiently store, manage, and serve high-dimensional embedding vectors at scale
- [Similarity Search](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/similarity-search): Finding the most similar items in a dataset, distance metrics, exact vs approximate methods, and real-world applications
- [Approximate Nearest Neighbor (ANN)](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/approximate-nearest-neighbor): The algorithms that make vector search fast. HNSW, IVF, LSH, and how they trade accuracy for speed
- [Database Federation](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/database-federation): Splitting your monolithic database into multiple specialized databases by domain, reducing load and enabling independent scaling
- [Polyglot Persistence](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/polyglot-persistence): Polyglot persistence means using several databases in one system. See an e-commerce example with five databases, and the real costs of running them.
- [Multi-Model Database](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/multi-model-database): One database, many data models, the promise of handling documents, graphs, key-value, and search in a single engine
- [NewSQL Databases](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/newsql-databases): Distributed SQL databases that promise the scalability of NoSQL with the ACID guarantees of traditional SQL. CockroachDB, Google Spanner, and the NewSQL movement
- [Distributed SQL](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/distributed-sql): How SQL databases scale across multiple nodes, sharding, query routing, distributed joins, and the coordination challenges
- [Global Tables](https://www.systemdesign.academy/courses/intermediate/lessons/database-types-storage/global-tables): Fully replicated tables available in every region, low-latency reads worldwide with the trade-off of slower writes
#### DevOps & CI/CD (intermediate)
- [Version Control](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/version-control): Tracking every change to your codebase, the foundation that makes collaboration, rollback, and history possible
- [Environment Variables](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/environment-variables): Separating configuration from code, the key to running the same application in development, staging, and production
- [Code Versioning](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/code-versioning): Semantic versioning and release tagging, communicating what changed and how much it matters
- [Build Automation](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/build-automation): Compiling, bundling, and packaging your code automatically, no manual steps, no forgotten flags, no 'it works on my machine'
- [Test Automation](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/test-automation): Running tests automatically on every code change, catching bugs before humans have to look at them
- [Dependency Management](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/dependency-management): Tracking, updating, and securing the third-party packages your application depends on
- [Package Management](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/package-management): Publishing, distributing, and consuming reusable code packages across your organization and the open-source world
- [Branch Strategies](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/branch-strategies): Organizing parallel development work through Git branching models: trunk-based, GitFlow, and GitHub Flow
- [Configuration Management](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/config-management): Managing application configuration across environments without hardcoding values or leaking secrets
- [Feature Flags](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/feature-flags): Deploying code to production without releasing features to users, decoupling deployment from release
- [Feature Toggles](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/feature-toggles): Fine-grained control over feature behavior at runtime, targeting users, environments, and percentages without redeployment
- [Continuous Integration](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/continuous-integration): Automatically building, testing, and validating every code change, catching issues within minutes of committing
- [Continuous Delivery](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/continuous-delivery): Ensuring that code is always in a deployable state, every commit could go to production if you choose
- [Continuous Deployment](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/continuous-deployment): Automatically deploying every change that passes the pipeline to production, no human gate, no release ceremony
- [Deployment Automation](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/deployment-automation): Scripting the entire deployment process so that deploys are repeatable, auditable, and require no manual steps
- [Release Management](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/release-management): Coordinating what goes out, when, and to whom, versioning, changelogs, and release communication
- [Artifact Repository](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/artifact-repository): Storing and managing build outputs. Docker images, binaries, and packages, in a versioned, access-controlled registry
- [Smoke Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/smoke-testing): Quick sanity checks after deployment to verify the application is alive and core functionality works
- [Regression Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/regression-testing): Verifying that new changes have not broken existing functionality, ensuring forward progress does not come at the cost of backward reliability
- [Integration Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/integration-testing): Testing how components work together, databases, APIs, message queues, and third-party services in combination
- [End-to-End Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/e2e-testing): Testing complete user workflows from browser to database and back, the final verification that everything works together
- [Load Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/load-testing): Simulating realistic user traffic to verify your system handles expected load without degradation
- [Stress Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/stress-testing): Pushing your system beyond its limits to find the breaking point and understand how it fails
- [Performance Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/performance-testing): Measuring and optimizing response times, throughput, and resource usage to meet performance SLAs
- [Shadow Testing](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/shadow-testing): Sending production traffic to a new version without affecting users, testing real-world behavior safely
- [Rolling Deployment](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/rolling-deployment): Replacing application instances one at a time, keeping the service available throughout the update
- [Blue-Green Deployment](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/blue-green-deployment): Maintaining two identical production environments and switching traffic between them for instant deployment and rollback
- [Canary Deployment](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/canary-deployment): Routing a small percentage of production traffic to the new version and monitoring before full rollout
- [Zero-Downtime Deployment](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/zero-downtime-deployment): Deploying new versions without any period of unavailability for users
- [A/B Testing Infrastructure](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/ab-testing-deployment): Running controlled experiments by routing users to different versions and measuring which performs better
- [Backward Compatibility](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/backward-compatibility): Ensuring new versions work with old clients, old data formats, and old APIs without breaking existing consumers
- [Schema Evolution](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/schema-evolution): Changing database schemas and data formats in production without breaking running applications
- [Data Migration](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/data-migration): Moving data between systems, formats, or schemas safely and completely, with validation, rollback, and zero data loss
- [Configuration as Code](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/config-as-code): Managing application and infrastructure configuration in version-controlled files instead of manual settings
- [Infrastructure as Code (IaC)](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/infrastructure-as-code): Defining servers, networks, databases, and cloud resources in declarative code files instead of clicking through consoles
- [GitOps](https://www.systemdesign.academy/courses/intermediate/lessons/devops-cicd/gitops): Using Git as the single source of truth for infrastructure and application deployment, pull-based, declarative, and auditable
#### Kubernetes & Containers (intermediate)
- [Containerization](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/containerization): Packaging applications with their dependencies into isolated, portable units that run anywhere
- [Container Registry](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/container-registry): Centralized storage for container images, the Docker Hub, ECR, and GHCR of the world
- [Image Versioning](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/image-versioning): Tagging, semantic versioning, and immutable deployments for container images
- [Declarative vs Imperative](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/declarative-vs-imperative): Two fundamentally different approaches to managing infrastructure, and why Kubernetes chose declarative
- [Orchestration](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/orchestration): Coordinating hundreds of containers across dozens of machines, the problem Kubernetes was built to solve
- [Auto-Scaling](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/auto-scaling): Automatically adjusting container count and resources based on demand. HPA, VPA, KEDA, and cluster scaling
- [Volume Management](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/volume-management): Giving ephemeral containers access to persistent data through Kubernetes volume abstractions
- [Persistent Volumes](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/persistent-volumes): Decoupling storage from pods so data survives container restarts, rescheduling, and failures
- [Init Containers](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/init-containers): Running setup tasks before the main container starts: migrations, config fetching, dependency checks
- [Jobs and CronJobs](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/jobs-cronjobs): Running one-off tasks and scheduled batch work in Kubernetes
- [Controllers](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/controllers): The control loop pattern that makes Kubernetes self-healing: watching, comparing, and reconciling state
- [Reconciliation Loops](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/reconciliation-loops): How the Kubernetes reconciliation loop observes, compares and acts to fix drift. Also explains level-triggered vs edge-triggered controllers.
- [DaemonSets](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/daemon-sets): Running exactly one pod on every node, for monitoring agents, log collectors, and network plugins
- [StatefulSets](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/stateful-sets): Managing stateful applications in Kubernetes, stable network identities, ordered deployment, and persistent storage per pod
- [Network Policies](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/network-policies): Firewall rules for Kubernetes pods, controlling which pods can communicate with each other
- [Ingress Controllers](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/ingress-controllers): Routing external HTTP traffic to the right Kubernetes service, the cluster's front door
- [Operators](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/operators): Encoding human operational knowledge into software, automating complex application lifecycle management in Kubernetes
- [Custom Resource Definitions](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/crds): Extending Kubernetes' API with your own resource types, the foundation of the operator pattern
- [Immutable Infrastructure](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/immutable-infrastructure): Never modify running systems, replace them entirely with new, tested versions
- [Config Drift Prevention](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/config-drift-prevention): Detecting and preventing configuration drift, when reality slowly diverges from your declared infrastructure
- [Desired State Configuration](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/desired-state-config): Defining what your infrastructure should look like and letting systems converge toward it automatically
- [Container Security](https://www.systemdesign.academy/courses/intermediate/lessons/kubernetes-containers/container-security): Securing containers from image to runtime, scanning, hardening, least privilege, and runtime protection
#### Messaging & Event Systems (intermediate)
- [Polling](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/polling): The simplest way to check for new data, ask repeatedly until something changes
- [Long Polling](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/long-polling): Hold the connection open until new data arrives, polling's smarter cousin
- [Webhooks](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/webhooks): Don't call us, we'll call you, server-to-server event notifications over HTTP
- [Push Notifications](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/push-notifications): Deliver messages to users even when they're not actively using your app
- [Message Queues](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/message-queues): Decouple producers from consumers with a buffer that holds messages until they're processed
- [Pub/Sub Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/pub-sub-pattern): One publisher, many subscribers, the pattern that powers event-driven systems at scale
- [WebSockets](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/websockets): Full-duplex, persistent connections, how the real-time web actually works
- [Server-Sent Events (SSE)](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/server-sent-events): One-way server push over HTTP, real-time updates without the complexity of WebSockets
- [Event Bus](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/event-bus): A central channel where components publish and subscribe to events without knowing each other
- [Command Bus](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/command-bus): Route explicit instructions to the one handler responsible for executing them
- [Request-Response Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/request-response-pattern): The most fundamental messaging pattern, send a request, wait for a reply
- [Request-Reply Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/request-reply-pattern): Request-response over messaging, decouple the caller from the responder with queues
- [Fire-and-Forget Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/fire-and-forget-pattern): Send a message and move on, no waiting, no acknowledgment, no looking back
- [At-Most-Once Delivery](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/at-most-once-delivery): Send it once, hope for the best, the fastest delivery guarantee, with a risk of loss
- [At-Least-Once Delivery](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/at-least-once-delivery): Never lose a message, but you might see it twice
- [Message Ordering](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/message-ordering): Guaranteeing messages arrive in the right sequence, harder than it sounds
- [Message Expiration](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/message-expiration): Messages that self-destruct, setting time limits on message relevance
- [Message Filters](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/message-filters): Let consumers receive only the messages they care about, filter at the broker, not in code
- [Priority Queue](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/priority-queue): Not all messages are equal, process the urgent ones first
- [Delayed Queue](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/delayed-queue): Schedule messages for future delivery, process them later, not now
- [Dead Letter Queue](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/dead-letter-queue): Where failed messages go to die, or to be investigated and rescued
- [Poison Message Handling](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/poison-message-handling): Detect and isolate messages that crash your consumers, before they crash them forever
- [Message Deduplication](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/message-deduplication): Detect and discard duplicate messages before they cause double-processing
- [Idempotent Consumer](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/idempotent-consumer): Design consumers that produce the same result whether a message is processed once or ten times
- [Competing Consumers](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/competing-consumers): Scale message processing by having multiple consumers race to handle messages from the same queue
- [Fan-Out/Fan-In](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/fan-out-fan-in): The fan-out/fan-in pattern splits work across parallel workers and combines the results. Includes a search across shards example and a sequence view.
- [Scatter-Gather](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/scatter-gather): Broadcast a request to many services, collect their responses, and pick the best one
- [Content-Based Routing](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/content-based-routing): Content-based routing sends each message to a destination based on what it contains. Includes a decision tree and a comparison of routing strategies.
- [Dynamic Router](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/dynamic-router): A router that learns and adapts its rules at runtime, no redeployment needed
- [Splitter Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/splitter-pattern): Break a complex message into smaller pieces that can be processed independently
- [Aggregator Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/aggregator-pattern): Collect related messages from multiple sources and combine them into a single output
- [Event-Driven Architecture](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/event-driven-architecture): Build systems where everything reacts to events, the architecture powering Netflix, Uber, and LinkedIn at massive scale
- [Exactly-Once Delivery](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/exactly-once-delivery): The holy grail of messaging, process every message once and only once. Here's why it's nearly impossible and how to fake it.
- [Transactional Outbox Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/transactional-outbox-pattern): Guarantee that your database write and message publish either both happen or neither does
- [Inbox Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/inbox-pattern): The consumer-side counterpart to the outbox, receive and deduplicate messages atomically with processing
- [Resequencer](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/resequencer): Reassemble out-of-order messages back into the correct sequence before processing
- [Process Manager](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/process-manager): Coordinate multi-step workflows across services by tracking state and routing messages through a sequence
- [Routing Slip](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/routing-slip): Attach a travel itinerary to the message itself, each service processes it and forwards to the next stop
- [Claim Check Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/claim-check-pattern): Store large payloads externally and pass a reference through the messaging system instead
- [Normalizer Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/normalizer-pattern): Convert messages from different formats into a common structure before processing
- [Canonical Data Model](https://www.systemdesign.academy/courses/intermediate/lessons/messaging-event-systems/canonical-data-model): Define one shared data language for all services, the standard that makes integration possible
#### Microservices Architecture (intermediate)
- [Monolithic Architecture](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/monolithic-architecture): The single-unit application architecture that every system starts with, and why it works until it doesn't
- [Singleton Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/singleton-pattern): Ensure a class has only one instance, the most loved and hated design pattern in software
- [Factory Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/factory-pattern): Delegate object creation to a factory, decouple what you use from how it's built
- [Observer Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/observer-pattern): Let objects subscribe to events and react when something changes, the foundation of event-driven systems
- [Strategy Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/strategy-pattern): Swap algorithms at runtime without touching the code that uses them, polymorphism done right
- [Adapter Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/adapter-pattern): Make incompatible interfaces work together, the universal translator of software architecture
- [Proxy Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/proxy-pattern): Control access to an object by placing a surrogate in front of it, caching, security, and lazy loading in one pattern
- [Decorator Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/decorator-pattern): Add responsibilities to objects dynamically by wrapping them, like middleware for your classes
- [Facade Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/facade-pattern): Simplify complex subsystems by providing a unified interface, the front desk of your software architecture
- [Command Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/command-pattern): Encapsulate requests as objects, enabling undo, queuing, logging, and distributed task execution
- [Chain of Responsibility](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/chain-of-responsibility): Pass requests along a chain of handlers, each handler decides to process it or pass it on
- [Mediator Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/mediator-pattern): The mediator pattern sends all communication through one central coordinator. Compares mediator, observer and event bus, and the god object risk.
- [Repository Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/repository-pattern): Abstract data access behind a clean interface, swap databases without touching business logic
- [Aggregate Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/aggregate-pattern): Group related domain objects into a single unit with one entry point, the consistency boundary of your domain
- [Service-Oriented Architecture (SOA)](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/service-oriented-architecture): The predecessor to microservices, how enterprise systems first broke apart monoliths using shared services and an ESB
- [Microservices Architecture](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/microservices-architecture-deep): The complete guide to microservices, principles, communication patterns, data ownership, and the real trade-offs nobody talks about
- [Service Discovery](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/service-discovery): How services find each other in a dynamic environment where instances spin up and die constantly
- [Retry Logic](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/retry-logic): When network calls fail, and they will, retries turn transient failures into invisible hiccups
- [Exponential Backoff](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/exponential-backoff): Instead of hammering a failing service, wait longer between each retry, giving the system time to recover
- [Jitter](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/jitter): Add randomness to retry timing to prevent the thundering herd, the missing piece of exponential backoff
- [Graceful Degradation](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/graceful-degradation): When parts of your system fail, give users a reduced experience instead of a broken one
- [Back Pressure](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/back-pressure): When a system is overwhelmed, push back on the sender instead of silently drowning, the flow control of distributed systems
- [Circuit Breaker Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/circuit-breaker-pattern): Prevent cascading failures in distributed systems by failing fast when a downstream service is unhealthy
- [Bulkhead Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/bulkhead-pattern): Isolate components into compartments so a failure in one doesn't sink the entire system, inspired by ship hull design
- [Domain-Driven Design (DDD)](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/domain-driven-design): Domain-driven design shapes software around the business, not the database. Covers bounded contexts, the building blocks, and DDD with microservices.
- [Bounded Context](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/bounded-context): Draw clear boundaries around domain models, where 'Customer' means one thing inside and something different outside
- [Database per Service](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/database-per-service): Each microservice owns its data, no shared databases, no secret backdoor queries, no coupled schemas
- [Shared Database Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/shared-database-pattern): When multiple services share one database, the anti-pattern that sometimes makes sense
- [Service Mesh. Concepts](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/service-mesh-concepts): Dedicated infrastructure for service-to-service communication, observability, security, and traffic management without changing application code
- [Service Mesh. Implementation](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/service-mesh-implementation): Istio vs. Linkerd vs. Cilium, choosing and deploying a service mesh in practice
- [Sidecar Container Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/sidecar-pattern): Attach a helper container to your application, handling logging, monitoring, proxying, or configuration without modifying the app
- [Ambassador Container Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/ambassador-pattern): A specialized sidecar that handles outbound connectivity, connection pooling, retries, and protocol translation for external services
- [Adapter Container Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/adapter-container-pattern): Standardize the output of your application, transform logs, metrics, and data into formats your infrastructure expects
- [Strangler Fig Pattern](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/strangler-fig-pattern): Migrate from monolith to microservices incrementally, replacing pieces one at a time while the old system stays alive
- [Anti-Corruption Layer](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/anti-corruption-layer): Protect your clean domain model from the messy reality of legacy systems and third-party APIs
- [Choreography vs. Orchestration](https://www.systemdesign.academy/courses/intermediate/lessons/microservices-architecture/choreography-vs-orchestration): Two fundamentally different ways to coordinate microservices: decentralized dance vs. centralized conductor
#### Observability & Monitoring (intermediate)
- [Log Levels](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/log-levels): The severity hierarchy that determines which messages matter and which are noise
- [Alerting](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/alerting): Getting notified when things go wrong, before your users tell you
- [Uptime Monitoring](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/uptime-monitoring): Knowing whether your service is alive, the most basic health check
- [Downtime Tracking](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/downtime-tracking): Recording, measuring, and learning from every minute your service is unavailable
- [Availability Metrics](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/availability-metrics): Measuring how reliably your system serves requests, the nines that define your reputation
- [Metrics Collection](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/metrics-collection): Gathering numerical measurements from your systems, the raw data behind every dashboard
- [Log Aggregation](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/log-aggregation): Collecting logs from every service into one searchable place, finding the needle in the haystack
- [Centralized Logging](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/centralized-logging): One place for all logs, the operational backbone of distributed systems
- [Structured Logging](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/structured-logging): Logs as data, not text, making every log line queryable and machine-parseable
- [Log Management](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/log-management): Retention, rotation, access control, and lifecycle policies, keeping logs useful without drowning in them
- [Counter Metrics](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/counter-metrics): Monotonically increasing numbers that count events, requests served, errors thrown, bytes transferred
- [Gauge Metrics](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/gauge-metrics): Values that go up and down, temperature readings for your systems
- [Time-Series Metrics](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/time-series-metrics): Data points indexed by time, the heartbeat of every monitoring system
- [Request IDs](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/request-ids): A unique identifier for every request, the thread that ties your distributed system's logs together
- [Correlation IDs](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/correlation-ids): Linking related requests across async boundaries, when one user action spawns many system actions
- [Infrastructure Monitoring](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/infrastructure-monitoring): Watching the machines underneath your application. CPU, memory, disk, and network
- [Network Monitoring](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/network-monitoring): Watching the wires, bandwidth, latency, packet loss, and connectivity between your services
- [Database Monitoring](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/database-monitoring): Watching your most critical dependency, query performance, connections, replication, and locks
- [SLI (Service Level Indicators)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/sli): The specific measurements that tell you whether your service is healthy, the numbers behind your promises
- [SLO (Service Level Objectives)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/slo): The reliability targets you set for your SLIs, the line between 'acceptable' and 'we need to fix this'
- [SLA (Service Level Agreements)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/sla): The legally binding reliability promise, when missing your targets has financial consequences
- [SLA/SLO/SLI Overview](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/sla-slo-sli-overview): Putting it all together, how SLIs, SLOs, and SLAs work as a unified reliability framework
- [MTTD (Mean Time to Detect)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/mttd): How long it takes to realize something is broken, the gap between failure and awareness
- [MTTA (Mean Time to Acknowledge)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/mtta): How long until a human says 'I'm on it', the gap between alert and action
- [MTTR (Mean Time to Resolve)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/mttr): How long it takes to fix the problem, from acknowledgment to all-clear
- [MTBF (Mean Time Between Failures)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/mtbf): How often things break, the frequency metric that shapes your reliability investment
- [Error Budget](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/error-budget): The quantified amount of unreliability you're allowed, the most powerful tool in SRE
- [Histogram Metrics](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/histogram-metrics): Bucketed distributions that show you the shape of your data, not just the average, but the full picture
- [Summary Metrics](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/summary-metrics): Pre-calculated percentiles computed on the client, exact numbers when you need precision
- [Percentiles](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/percentiles): P50, P95, P99, measuring the experience of your happiest and most frustrated users
- [Metrics Aggregation](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/metrics-aggregation): Combining metrics across instances, time windows, and dimensions, from raw data to actionable dashboards
- [Observability Overview](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/observability-overview): The three pillars of observability, logs, metrics, and traces, and how they work together to make complex systems understandable
- [APM (Application Performance Monitoring)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/apm): End-to-end visibility into your application's performance, code-level insights without reading logs
- [RUM (Real User Monitoring)](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/rum): Measuring performance from the user's browser, the truth about what your customers actually experience
- [Synthetic Monitoring](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/synthetic-monitoring): Simulated user requests from controlled environments, catching problems before real users do
- [Trace IDs](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/trace-ids): The unique identifier for a distributed trace, one ID that ties together every service a request touches
- [Span IDs](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/span-ids): Individual operations within a trace, each span is one unit of work with a start time, duration, and parent
- [Distributed Context Propagation](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/distributed-context-propagation): Passing trace context, baggage, and metadata across service boundaries, the glue of distributed tracing
- [Baggage](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/baggage): Custom key-value pairs that travel with the trace context, carrying business data through your distributed system
- [Distributed Tracing](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/distributed-tracing): End-to-end request visibility across microservices, seeing the full journey from entry to response
- [Apdex Score](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/apdex-score): A single number from 0 to 1 that rates user satisfaction with response time, the happiness meter for your application
- [Golden Signals](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/golden-signals): The four metrics Google monitors for every service, latency, traffic, errors, and saturation
- [RED Method](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/red-method): The request-centric monitoring framework. Rate, Errors, and Duration for every microservice
- [USE Method](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/use-method): The resource-centric monitoring framework. Utilization, Saturation, and Errors for every component
- [Four Golden Signals](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/four-golden-signals): Implementing latency, traffic, errors, and saturation monitoring across your entire infrastructure
- [OpenTelemetry](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/opentelemetry): The vendor-neutral observability framework, one standard for traces, metrics, and logs across every language and platform
- [Prometheus](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/prometheus): The open-source metrics system that became the industry standard, pull-based collection, powerful queries, and native Kubernetes integration
- [Grafana](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/grafana): The visualization layer for your observability stack, dashboards, alerts, and exploration across every data source
- [Jaeger](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/jaeger): Uber's open-source distributed tracing system, built for microservices at massive scale
- [Zipkin](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/zipkin): Twitter's original distributed tracing system, the pioneer that inspired modern tracing
- [Datadog](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/datadog): The all-in-one observability platform, metrics, logs, traces, and more in a single SaaS product
- [New Relic](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/new-relic): The original APM pioneer, now a full observability platform with a generous free tier
- [AppDynamics](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/appdynamics): Cisco's enterprise APM solution, business transaction tracing and AI-powered root cause analysis
- [Dynatrace](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/dynatrace): AI-driven full-stack observability, automatic discovery, baselining, and root cause analysis
- [Splunk](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/splunk): The data platform that started with logs, now full observability with powerful search and analytics
- [ELK Stack](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/elk-stack): Elasticsearch, Logstash, and Kibana, the open-source log management stack used by thousands of organizations
- [Fluentd](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/fluentd): The CNCF log collector that unifies data collection, input, filter, output plugins for every data source
- [Vector](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/vector): A high-performance observability data pipeline, collect, transform, and route logs and metrics with Rust-powered speed
- [Telegraf](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/telegraf): InfluxData's plugin-driven metrics agent, collect metrics from 300+ sources with a single binary
- [StatsD](https://www.systemdesign.academy/courses/intermediate/lessons/observability-monitoring/statsd): The simplest metrics protocol, fire-and-forget UDP packets for lightweight application instrumentation
#### Security Architecture (intermediate)
- [Authentication](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/authentication): Verifying who someone is, the foundation of every security system
- [Authorization](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/authorization): Controlling what authenticated users are allowed to do: permissions, roles, and access policies
- [Encryption at Rest](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/encryption-at-rest): Protecting stored data from unauthorized access, even if an attacker gets physical access to the disk
- [Encryption in Transit](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/encryption-in-transit): Protecting data as it travels across networks, preventing eavesdropping and tampering
- [Security Headers](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/security-headers): HTTP response headers that instruct browsers to enforce security policies
- [Input Sanitization](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/input-sanitization): Cleaning and validating all user input before processing, the first line of defense against injection attacks
- [SQL Injection](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/sql-injection): The oldest and most devastating web attack, injecting malicious SQL through unsanitized user input
- [XSS](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/xss): Cross-Site Scripting, injecting malicious scripts into web pages viewed by other users
- [CSRF](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/csrf): Cross-Site Request Forgery, tricking a user's browser into making authenticated requests to a target site
- [Parameterized Queries](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/parameterized-queries): Why parameterized queries stop SQL injection by keeping SQL code and user data apart. Shows code examples and what happens inside the database.
- [Prepared Statements](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/prepared-statements): Pre-compiled SQL templates that combine security with performance, parse once, execute many times
- [Secure Coding](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/secure-coding): Principles and practices for writing code that resists attacks, from input validation to dependency management
- [Cookie-Based Sessions](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/cookie-sessions): Using HTTP cookies to maintain user state across requests, the traditional session model
- [Token-Based Authentication](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/token-auth): Stateless authentication using signed tokens instead of server-side sessions
- [JSON Web Tokens (JWT)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/jwt): The most popular token format: compact, self-contained, and cryptographically signed
- [JWT Sessions](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/jwt-sessions): Using JWTs as session tokens: the trade-offs, pitfalls, and best practices
- [API Keys](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/api-keys): Simple tokens for identifying and authenticating API clients, not users
- [OAuth 2.0](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/oauth): How OAuth 2.0 lets an app use your data without your password. Covers the four roles, grant types, the authorization code flow, PKCE and common risks.
- [Single Sign-On (SSO)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/sso): How single sign-on works between the user, the identity provider and the app. Compares SAML, OIDC and CAS, with a SAML login flow diagram.
- [Multi-Factor Authentication (MFA)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/mfa): Requiring multiple forms of proof before granting access, something you know, have, and are
- [Distributed Session Management](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/distributed-session-management): Managing user sessions across multiple servers, sticky sessions, centralized stores, and token-based approaches
- [Role-Based Access Control (RBAC)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/rbac): Assigning permissions to roles instead of individuals, the most widely used access control model in production systems
- [Access Control Lists (ACL)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/acl): Attaching permission lists directly to resources, who can do what to this specific object
- [Permission Inheritance](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/permission-inheritance): How permissions flow from parent resources to children, folders to files, organizations to projects, roles to sub-roles
- [Attribute-Based Access Control (ABAC)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/abac): Making access decisions based on attributes of users, resources, and environment, the most flexible access control model
- [Policy-Based Access Control](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/policy-based-access-control): Defining access rules as declarative policies evaluated by a central decision engine, OPA, Cedar, and beyond
- [Time-Based Access Control](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/time-based-access-control): Restricting access based on time windows, business hours, maintenance windows, temporary grants, and expiring permissions
- [Location-Based Access Control](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/location-based-access-control): Restricting access based on geographic location, IP address, or network, geofencing for security
- [Context-Aware Access Control](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/context-aware-access-control): Combining multiple signals, device, location, behavior, risk score, to make dynamic access decisions in real time
- [Principle of Least Privilege](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/least-privilege): Granting the minimum permissions necessary to perform a task, nothing more, nothing less
- [Separation of Duties](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/separation-of-duties): Requiring multiple people to complete sensitive operations, no single person should have end-to-end control
- [Transport Layer Security (TLS)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/tls): The protocol that encrypts data in transit between clients and servers, the S in HTTPS
- [SSL/TLS Certificates](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/ssl-tls-certificates): How SSL/TLS certificates prove who a website is. Compares DV, OV and EV certificates, and walks through the certificate trust chain.
- [Certificate Management](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/certificate-management): The lifecycle of TLS certificates, issuance, renewal, revocation, and the automation that prevents outages
- [Biometric Authentication](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/biometric-auth): Using physical characteristics, fingerprints, face scans, voice patterns, as authentication factors
- [Certificate-Based Authentication](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/certificate-based-auth): Using X.509 certificates to authenticate clients and services, proving identity through cryptographic proof instead of passwords
- [Mutual TLS (mTLS)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/mtls): Both client and server verify each other's certificates, two-way authentication for zero-trust architectures
- [Service-to-Service Authentication](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/service-to-service-auth): How microservices prove their identity to each other, mTLS, JWTs, API keys, and SPIFFE
- [Identity Federation](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/identity-federation): How identity federation lets people log in across organizations with one account. Compares SAML and OpenID Connect, and hub-and-spoke vs mesh.
- [SAML](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/saml): Security Assertion Markup Language, the XML-based protocol that powers enterprise single sign-on and identity federation
- [OpenID Connect](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/openid-connect): An identity layer built on OAuth 2.0, the modern protocol for 'Sign in with Google/GitHub/Apple'
- [Defense in Depth](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/defense-in-depth): Layering multiple security controls so that if one fails, others still protect the system
- [Security by Design](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/security-by-design): Building security into systems from the start instead of bolting it on after the fact
- [Threat Modeling](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/threat-modeling): Systematically identifying and prioritizing threats to your system before attackers find them
- [Attack Surface Reduction](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/attack-surface-reduction): Minimizing the number of entry points an attacker can exploit, less surface means fewer opportunities for breach
- [Zero Trust Architecture](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/zero-trust): Never trust, always verify, eliminating implicit trust from network architecture
- [DDoS Protection](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/ddos-protection): Defending against distributed denial-of-service attacks that overwhelm your infrastructure with traffic
- [Web Application Firewall (WAF)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/waf): Filtering malicious HTTP traffic before it reaches your application, blocking SQL injection, XSS, and other web attacks at the edge
- [API Gateway Security](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/api-gateway-security): Centralizing authentication, rate limiting, and threat protection at the API gateway layer
- [Certificate Pinning](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/certificate-pinning): Binding a client to a specific server certificate to prevent man-in-the-middle attacks even when a trusted CA is compromised
- [Public Key Infrastructure (PKI)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/pki): The system of CAs, certificates, and trust hierarchies that enables secure communication across the internet
- [Key Management Service (KMS)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/kms): Centralized management of cryptographic keys, creation, rotation, access control, and audit
- [Hardware Security Module (HSM)](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/hsm): Dedicated tamper-resistant hardware for protecting cryptographic keys, the most secure key storage available
- [Secrets Management](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/secrets-management): Storing, accessing, and rotating sensitive credentials. API keys, database passwords, tokens, and certificates
- [Vault Systems](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/vault-systems): HashiCorp Vault and similar tools for dynamic secrets, encryption as a service, and centralized credential management
- [Security Audit Logging](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/security-audit-logging): Recording security-relevant events with tamper-proof, queryable audit trails for compliance and incident investigation
- [Incident Response](https://www.systemdesign.academy/courses/intermediate/lessons/security-architecture/incident-response): Structured processes for detecting, containing, eradicating, and recovering from security incidents
#### Web & Content Delivery (intermediate)
- [Browser Storage](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/browser-storage): Understanding the different ways browsers persist data on the client side, cookies, localStorage, sessionStorage, and IndexedDB
- [Web Storage API](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/web-storage-api): The localStorage and sessionStorage APIs, simple key-value storage in the browser with cross-tab communication
- [Static Asset Optimization](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/static-assets): Managing and optimizing CSS, JavaScript, images, and fonts for fast page loads
- [Gzip Compression](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/gzip-compression): Compressing HTTP responses to reduce transfer size by 60-80%, the lowest-effort highest-impact performance optimization
- [Image Optimization](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/image-optimization): Reducing image file sizes without visible quality loss, the biggest opportunity for most web pages
- [Lazy Loading](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/lazy-loading): Deferring the loading of off-screen content until the user scrolls to it, reducing initial page weight and improving LCP
- [Responsive Images](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/responsive-images): Serving different image sizes for different screen sizes so mobile users do not download desktop-sized images
- [Image Formats](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/image-formats): JPEG, PNG, WebP, AVIF, SVG: understanding when to use each format and why it matters for performance
- [Brotli Compression](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/brotli-compression): A modern compression algorithm that outperforms gzip by 15-25%, supported by all major browsers and CDNs
- [Content Encoding](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/content-encoding): How HTTP content encoding works, negotiating compression between client and server transparently
- [Progressive Images](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/progressive-images): Loading images that render a blurry preview immediately and sharpen as more data arrives, perceived performance at its best
- [Thumbnail Generation](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/thumbnails): Creating and serving properly sized preview images for galleries, lists, and social sharing
- [Client-Side Rendering (CSR)](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/csr): Rendering pages entirely in the browser with JavaScript, how SPAs work and when they make sense
- [Server-Side Rendering (SSR)](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/ssr): Generating HTML on the server for each request, fast first paint, SEO-friendly, and dynamic content
- [Static Site Generation (SSG)](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/ssg): Pre-building HTML pages at build time, the fastest possible delivery for content that does not change per request
- [Dynamic Content Delivery](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/dynamic-content): Serving personalized, real-time, or user-specific content that cannot be pre-built or cached uniformly
- [Asset Pipeline](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/asset-pipeline): The build process that transforms source files into optimized, production-ready assets with hashing, minification, and bundling
- [IndexedDB](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/indexeddb): A full database in the browser, storing large structured data for offline apps, caching, and client-side search
- [Service Workers](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/service-workers): Background scripts that intercept network requests, enabling offline support, push notifications, and intelligent caching
- [Progressive Web Apps (PWA)](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/pwa): Web applications that feel like native apps, installable, offline-capable, and push-notification-enabled
- [Offline-First Architecture](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/offline-first): Designing applications that work without a network connection by default and sync when connectivity returns
- [Online-First Architecture](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/online-first): Traditional web architecture where the server is the source of truth and offline is handled as a graceful degradation
- [Sync Mechanisms](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/sync-mechanisms): Resolving conflicts when offline changes sync with the server. CRDTs, last-write-wins, and merge strategies
- [Video Encoding](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/video-encoding): Converting raw video into compressed formats for web delivery, codecs, containers, and quality vs size tradeoffs
- [Audio Encoding](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/audio-encoding): Compressing audio for web delivery. MP3, AAC, Opus, and when to use each
- [Transcoding](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/transcoding): Converting media between formats, resolutions, and bitrates, the backbone of video platforms
- [Incremental Static Regeneration (ISR)](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/isr): Updating static pages after build time without rebuilding the entire site, the best of SSG and SSR combined
- [Edge-Side Includes (ESI)](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/edge-side-includes): Edge Side Includes (ESI) lets a CDN build a page from cached and dynamic parts. Includes a comparison of ESI, SSI and edge functions.
- [Streaming Protocols](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/streaming-protocols): HLS, DASH, and WebRTC, delivering live and on-demand video over the internet at scale
- [Adaptive Bitrate Streaming](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/adaptive-bitrate): Automatically adjusting video quality based on the viewer's network speed, smooth playback across all connections
- [Media Processing Pipeline](https://www.systemdesign.academy/courses/intermediate/lessons/web-content-delivery/media-processing-pipeline): End-to-end architecture for ingesting, processing, encoding, storing, and delivering media at scale