System Design Glossary

Security Architecture

Attribute-Based Access Control: grants permissions based on attributes (user department, resource type, time of day, IP range). More flexible but more complex than RBAC.

Database Fundamentals

Four guarantees for database transactions: Atomicity (all or nothing), Consistency (valid states only), Isolation (no interference), Durability (changes persist).

Microservices Architecture

A structural design pattern that wraps a class with an incompatible interface so it matches the interface your code expects, like a travel plug adapter for software.

LLM and GenAI Ops

Reusable ways to structure an AI agent's work, such as ReAct, plan-and-execute, reflection and multi-agent setups. Each trades extra model calls for a chance at a better answer.

LLM and GenAI Ops

Agentic RAG is RAG where a language model runs the search in a loop. It decides whether to search, what to search for and where. Then it checks if the text it found answers the question.

LLM and GenAI Ops

A language model placed in a loop: it decides an action, calls a tool, reads the result, and repeats until done. The hard part is that per-step errors compound over many steps.

LLM and GenAI Ops

Attacking your own AI system on purpose, the way a real attacker would, to find what it can be tricked into doing before someone else does.

Observability & Monitoring

Automatically notifying engineers when metrics cross predefined thresholds. Good alerts are actionable, not noisy. PagerDuty and Opsgenie route alerts to the right on-call person.

LLM and GenAI Ops

Search that finds vectors close to a query without scanning every item, trading a small chance of missing a true neighbor for a large gain in speed. HNSW and IVF are the common indexes.

Cloud Infrastructure

An agentless automation tool for configuration management, application deployment, and orchestration. Uses YAML playbooks and connects over SSH.

Consistency Models

A background process that compares data between replicas and fixes differences. Uses Merkle trees to efficiently identify which data ranges are out of sync.

Stream & Batch Processing

A distributed stream processing framework that handles both real-time streams and batch data with exactly-once guarantees. Used by Alibaba, Netflix, and Uber at massive scale.

Stream & Batch Processing

A unified analytics engine for large-scale data processing with built-in modules for SQL, streaming, ML, and graph computation. Processes data in-memory for speed.

Core Fundamentals

Application Programming Interface: a contract defining how two pieces of software talk to each other. The waiter between your frontend and your backend.

Microservices Architecture

A pattern where a composer service queries multiple microservices and joins the results in memory. The simplest way to implement cross-service queries.

Core Fundamentals

A single entry point for all client requests that routes them to the appropriate microservice. Handles auth, rate limiting, and request transformation.

Security Architecture

A simple token passed with API requests to identify the calling project or application. Not a substitute for user authentication but useful for rate limiting and usage tracking.

API Design & Protocols

Strategies for evolving an API without breaking existing clients. Common approaches: URL path (/v2/users), header (Accept-Version), or query param (?version=2).

Observability & Monitoring

Application Performance Monitoring: tools that track request latency, error rates, and dependencies in real time. Datadog, New Relic, and Grafana are popular APM platforms.

DevOps & CI/CD

A repository for storing build outputs like Docker images, JAR files, and npm packages. Docker Hub, GitHub Container Registry, and Artifactory are common registries.

Core Fundamentals

A communication model where the caller fires off a request and continues without waiting for a response. Essential for non-blocking I/O and event-driven systems.

Messaging & Event Systems

A messaging guarantee where every message is delivered one or more times. Simpler than exactly-once but requires consumers to handle duplicates via idempotency.

Cloud Infrastructure

Automatically adding or removing compute instances based on current demand. Scales out during traffic spikes and scales in during quiet periods to save cost.

Core Fundamentals

The percentage of time a system is operational and accessible. Measured in 'nines', 99.99% availability means about 52 minutes of downtime per year.

Cloud Infrastructure

An isolated data center within a cloud region, with independent power, cooling, and networking. Deploying across multiple AZs protects against single-facility failures.

Database Fundamentals

A self-balancing search tree in which each node is a page holding many sorted keys, so it stays wide and shallow. Most database indexes use it: finding one key among millions takes only a few page reads.

Reliability & Resilience

A flow control mechanism where a slow consumer signals upstream producers to slow down. Prevents systems from being overwhelmed by data they can't process.

Microservices Architecture

A dedicated backend service tailored for a specific frontend (mobile, web, TV). Each frontend gets an API shaped to its exact needs instead of sharing one generic API.

Stream & Batch Processing

Retroactively populating a new data store, index, or column with historical data. Typically done as a batch job when adding a new feature that needs past data.

Core Fundamentals

The maximum amount of data that can be transferred over a network in a given time. It's the width of the pipe, not how fast the water flows.

Database Fundamentals

An alternative to ACID for distributed systems: Basically Available, Soft state, Eventually consistent. Trades strong consistency for availability.

Stream & Batch Processing

Processing large volumes of data in scheduled chunks rather than in real time. Think nightly reports, ETL jobs, and data warehouse loads.

Cloud Infrastructure

Storage that splits data into fixed-size blocks, each with its own address. Used for databases and boot volumes where low-latency random I/O matters. AWS EBS is block storage.

Distributed Systems Core

A space-efficient probabilistic data structure that tells you if an element is 'possibly in the set' or 'definitely not in the set.' Used by databases to avoid expensive lookups.

DevOps & CI/CD

A deployment strategy using two identical environments. Traffic switches from blue (current) to green (new) instantly, with easy rollback.

Microservices Architecture

A pattern that isolates different parts of a system so a failure in one part doesn't sink the whole ship. Named after the compartments in a ship's hull.

LLM and GenAI Ops

The rule a tokenizer uses to learn its pieces: starting from single characters or bytes, it repeatedly merges the most frequent pair of neighbouring pieces into a new token.

Caching Strategies

The process of removing entries from a cache when it's full. Common policies: LRU (least recently used), LFU (least frequently used), FIFO (first in, first out).

Caching Strategies

Removing or updating stale cache entries when the underlying data changes. One of the two hard problems in computer science (along with naming things).

Caching Strategies

When many requests hit the database simultaneously because a popular cache entry expired. Solved with locking, probabilistic early expiration, or request coalescing.

Caching Strategies

A caching pattern where the application checks the cache first; on a miss, it fetches from the database and populates the cache. Also called lazy loading.

Core Fundamentals

Storing frequently accessed data in a faster storage layer so you don't have to fetch it from the original (slower) source every time.

Reliability & Resilience

Automated statistical comparison of metrics between the canary (new version) and the baseline (current version) to decide whether to promote or roll back a deployment.

DevOps & CI/CD

Rolling out a new version to a small percentage of users first, then gradually increasing. Like sending a canary into a coal mine to test for danger.

Distributed Systems Core

When a network failure splits a distributed data store, each node must either refuse requests to stay consistent or keep answering with possibly stale data. It cannot guarantee both during the partition.

Reliability & Resilience

Estimating future resource needs based on current usage trends and expected growth. Answers questions like 'how many servers do we need for 10x traffic in 6 months?'

Consistency Models

A consistency model that preserves cause-and-effect ordering: if operation A causally precedes B, all nodes see A before B. Weaker than linearizability but stronger than eventual consistency.

Caching Strategies

A network of servers distributed globally that caches content close to users. Netflix uses CDNs to stream video from servers near you, not from one central location.

Microservices Architecture

A behavioral design pattern that passes a request along a chain of handlers, where each handler either deals with it or passes it to the next one. HTTP middleware is the everyday example.

Stream & Batch Processing

Capturing row-level changes in a database and streaming them to other systems in real time. Debezium reads the write-ahead log and publishes changes to Kafka.

Reliability & Resilience

Deliberately injecting failures into a system to test its resilience. Netflix's Chaos Monkey randomly kills servers to ensure the system survives.

Stream & Batch Processing

Periodically saving the state of a stream processing job so it can recover from failures without reprocessing everything from the beginning. Flink and Spark use distributed checkpoints.

LLM and GenAI Ops

The rules for splitting documents into pieces before they are embedded for search. Chunk size and boundaries decide what retrieval can find.

DevOps & CI/CD

Continuous Integration and Continuous Deployment: automating the process of testing and deploying code. Push code, tests run, and it ships to production automatically.

Microservices Architecture

A pattern that stops calling a failing service after repeated failures, preventing cascade failures. Like an electrical circuit breaker that cuts power to prevent fires.

Microservices Architecture

A circuit breaker cycles through three states: Closed (requests flow normally), Open (requests are blocked after failures), Half-Open (a few test requests check if the service recovered).

Core Fundamentals

The foundational architecture of the web: clients (browsers, apps) send requests and servers process them and return responses. Every web interaction follows this pattern.

Distributed Systems Core

The difference in time readings between clocks on different machines. Physical clocks drift; NTP helps but can't guarantee perfect sync. Why distributed systems use logical clocks.

Cloud Infrastructure

A geographic area containing one or more data centers (availability zones). Choosing the right region reduces latency and satisfies data residency requirements.

Database Types & Storage

A NoSQL database that groups columns into families, optimized for reading and writing large amounts of data across many machines. Cassandra and HBase use this model.

Microservices Architecture

A behavioral design pattern that turns a request into an object holding everything needed to run it, so requests can be queued, logged, retried or undone.

Microservices Architecture

Gracefully finishing in-flight requests before removing a server from a load balancer's pool. Prevents dropping active connections during deployments or scale-ins.

Database Fundamentals

A cache of reusable database connections that avoids the overhead of opening and closing a new connection for every query. Critical for high-throughput applications.

Distributed Systems Core

The process of getting multiple nodes in a distributed system to agree on a single value. The foundation of distributed databases and coordination services.

Data Replication & Distribution

A hashing technique where adding or removing servers only moves a small fraction of keys. Used by Amazon DynamoDB and Cassandra for data distribution.

Kubernetes & Containers

A lightweight, isolated environment that packages an application with its dependencies. Shares the host OS kernel, unlike VMs. Starts in milliseconds.

Kubernetes & Containers

A lightweight, immutable package containing everything needed to run an application: code, runtime, libraries, and config. Built with a Dockerfile and stored in a registry.

Web & Content Delivery

The process of distributing and serving content to users from locations geographically close to them for faster load times.

Web & Content Delivery

The process where client and server agree on the response format using HTTP Accept headers. Enables serving JSON, XML, or HTML from the same endpoint.

Security Architecture

An HTTP response header that restricts which resources (scripts, styles, images) a page can load. A powerful defense against XSS attacks by whitelisting trusted sources.

LLM and GenAI Ops

Deciding what goes into a model's limited context window and in what order. Production data shows quality degrades well before the advertised token limit, so what you leave out matters as much as what you put in.

LLM and GenAI Ops

The drop in a language model's accuracy as its input gets longer, even when the answer is in the text and well inside the context window.

LLM and GenAI Ops

The maximum number of tokens a language model can take in at once, counting the prompt, any documents and its own reply.

LLM and GenAI Ops

Rebuilding the inference batch after every token so a finished sequence leaves and a waiting request joins immediately, instead of every request waiting for the slowest one. It is what keeps a serving GPU busy.

Core Fundamentals

A small piece of data the server sends to the browser, which the browser stores and sends back with every subsequent request. Powers sessions, tracking, and preferences.

Security Architecture

Cross-Origin Resource Sharing: a security mechanism that controls which domains can access your API. The browser enforces it; the server configures it.

LLM and GenAI Ops

A measure of how close two embeddings are, based on the angle between them rather than their length. The standard way to rank how related two pieces of text are in vector search.

Distributed Systems Core

Command Query Responsibility Segregation: using different models for reading and writing data. Reads and writes have different performance needs, so separate them.

Distributed Systems Core

Conflict-free Replicated Data Type: a data structure that can be updated independently on different nodes and merged automatically without conflicts. Powers real-time collaboration like Google Docs.

Microservices Architecture

The family of design patterns about how objects get created: Singleton, Factory Method, Abstract Factory, Builder and Prototype. They hide the construction details from the code that uses the object.

Security Architecture

Cross-Site Request Forgery: an attack that tricks a logged-in user's browser into sending unwanted requests to a site where they're authenticated. Prevented with tokens.

Data Governance & Compliance

A searchable inventory of all datasets in an organization, with metadata like schema, owner, freshness, and lineage. The 'Google for your data.'

Data Governance & Compliance

Transforming data into an unreadable format using cryptographic algorithms. Encryption at rest protects stored data; encryption in transit protects data over the network.

Data Replication & Distribution

Splitting databases by function (users in one DB, orders in another) so each database handles only its domain. Simpler than sharding but limits cross-domain joins.

Core

Attaching the correct answer to each example, such as a category or a yes or no, so a model can learn from it or be tested against it.

Stream & Batch Processing

A centralized repository that stores raw data at any scale in its native format. Unlike a data warehouse, data doesn't need to be structured or cleaned before loading.

Core

A design that keeps data as open files, usually Parquet, in cheap object storage and adds a log on top that makes changes all-or-nothing, checks column types and keeps old versions readable.

Data Governance & Compliance

Tracking data from its origin through every transformation and system it passes through. Answers 'where did this number come from?' for audits and debugging.

Data Governance & Compliance

Replacing sensitive data with realistic but fake values so developers and testers can work with production-like data without exposing real PII.

Stream & Batch Processing

A decentralized data architecture where each domain team owns and publishes its data as a product. Shifts responsibility from a central data team to domain teams.

Stream & Batch Processing

A central repository of structured, cleaned data optimized for analytical queries. Snowflake, BigQuery, and Redshift are purpose-built data warehouses.

Database Fundamentals

An organized collection of data that can be easily accessed, managed, and updated. The backbone of almost every application.

Data Replication & Distribution

Dividing a large table into smaller, more manageable pieces while keeping them in the same database. Sharding is partitioning across servers.

Microservices Architecture

A microservices pattern where each service owns its private database. No other service can access it directly. Enforces loose coupling but complicates cross-service queries.

Database Fundamentals

A piece of SQL that automatically executes in response to INSERT, UPDATE, or DELETE events on a table. Useful for audit logs and enforcing complex constraints.

Database Fundamentals

A virtual table defined by a SQL query. Simplifies complex joins, enforces access control, and presents data in a specific shape without duplicating storage.

Messaging & Event Systems

A queue that stores messages that couldn't be processed after multiple attempts. Prevents poison messages from blocking the main queue and lets you debug failures.

Database Fundamentals

When two or more transactions are each waiting for the other to release a lock, creating a cycle where none can proceed. Databases detect and break deadlocks by aborting one.

Microservices Architecture

A structural design pattern that wraps an object in another object with the same interface to add behavior, such as logging, caching or retries, without changing the original class.

Database Fundamentals

Intentionally adding redundant data to database tables to speed up read queries by avoiding expensive joins. Trades storage and write complexity for read performance.

Microservices Architecture

Named, reusable solutions to problems that keep coming up in object-oriented code, such as creating objects, wrapping interfaces or reacting to events. The classic catalogue of 23 was published by the Gang of Four in 1994.

Reliability & Resilience

A plan and set of procedures for restoring systems after a catastrophic failure. Defined by RPO (how much data you can lose) and RTO (how long you can be down).

Caching Strategies

A cache spread across multiple nodes that acts as a shared layer between application servers and the database. Redis Cluster and Memcached are common implementations.

Distributed Systems Core

A lock that coordinates access to a shared resource across multiple machines. Implemented via Redis (Redlock), ZooKeeper, or etcd. Much harder than local locks.

Distributed Systems Core

The algorithms machines use to agree and coordinate over an unreliable network, such as leader election, consensus with Raft or Paxos, logical clocks, gossip and two-phase commit.

Observability & Monitoring

Tracking a request as it flows through multiple services in a distributed system. Each service adds its trace, creating a full picture of the request journey.

Distributed Systems Core

A transaction whose changes span more than one database or service, which must either all commit or all be undone. Two-phase commit and sagas are the two main ways to get there.

Load Balancing & Proxies

The phonebook of the internet. Translates human-readable domain names (google.com) into IP addresses that computers understand.

Load Balancing & Proxies

Different types of DNS records map domains to resources: A records point to IPs, CNAME aliases one domain to another, MX routes email, TXT stores verification data.

Kubernetes & Containers

A platform for packaging applications into lightweight, portable containers. 'Works on my machine' becomes 'works everywhere.'

Database Types & Storage

A NoSQL database that stores data as flexible JSON-like documents. MongoDB and CouchDB let each document have a different structure.

Microservices Architecture

A way of designing software around the business domain: the code uses the same words as the business, and the system is split into bounded contexts, each with its own model.

Web & Content Delivery

Running computation at the network edge, close to the user, instead of in a central data center. Reduces latency for real-time applications like IoT and streaming.

Database Types & Storage

A distributed search and analytics engine built on Apache Lucene. Powers full-text search, log analysis, and real-time analytics at scale.

LLM and GenAI Ops

A list of numbers that represents the meaning of a piece of text, so that similar meanings sit close together in vector space. The same model must embed both the corpus and the query.

LLM and GenAI Ops

Reading real failure traces and coding them into categories to find where a system actually breaks, before building any metric or dashboard. Most of eval effort should go here, not into tooling.

Reliability & Resilience

The allowed amount of unreliability derived from SLOs. If your SLO is 99.9% uptime, your error budget is 0.1% (about 43 minutes/month). Once exhausted, freeze deployments.

Data Replication & Distribution

Extract, Transform, Load: a pipeline that extracts data from sources, transforms it into the desired format, and loads it into a destination like a data warehouse.

Core Fundamentals

A programming pattern that waits for and dispatches events in a single thread. Node.js uses it to handle thousands of concurrent connections without creating a thread per connection.

Distributed Systems Core

Storing every state change as an immutable event instead of just the current state. You can rebuild any past state by replaying events.

Messaging & Event Systems

A design pattern where services communicate by producing and consuming events rather than making direct calls. Promotes loose coupling and asynchronous processing.

Consistency Models

A consistency model where updates propagate asynchronously and all replicas will eventually converge to the same value. Trades immediacy for availability.

Stream & Batch Processing

A processing guarantee where each message is processed exactly one time, even in the face of failures. Achieved through idempotent consumers and transactional producers.

Microservices Architecture

A retry strategy that doubles the wait time between attempts (1s, 2s, 4s, 8s...) with random jitter. Prevents thundering herd problems when many clients retry simultaneously.

Microservices Architecture

A structural design pattern that puts one simple interface in front of a complicated set of classes or services, so callers make one call instead of coordinating many.

Microservices Architecture

A creational design pattern where one method or class decides which concrete object to build, so the calling code asks for an object by type or config instead of calling a constructor directly.

Reliability & Resilience

Deliberately introducing faults (latency, errors, crashes) into a system to verify that resilience mechanisms work. A specific technique within chaos engineering.

Core Fundamentals

A system's ability to keep operating correctly even when some of its components fail. Achieved through redundancy, replication, and graceful degradation.

Core

Turning raw data into the signals a model actually learns from. Often the highest-leverage work in a classical ML system, and where a feature store keeps definitions consistent.

Microservices Architecture

A toggle in code that enables or disables a feature without redeploying. Lets you ship code to production behind a flag and turn it on for a percentage of users.

Core

A single source of truth for features, so training and serving compute them from one definition and cannot drift apart. It ends the train-serve skew bug and lets teams reuse features across models.

LLM and GenAI Ops

Continuing to train a pretrained model on your own data to change its behavior, style, or format. Best for teaching a skill, not for injecting facts, which is what retrieval is for.

Core Fundamentals

A communication pattern where the sender dispatches a message and doesn't wait for or expect a response. Used for logging, analytics events, and non-critical notifications.

Reliability & Resilience

A visualization of profiled software showing which functions consume the most CPU or wall-clock time. The x-axis is the stack population and the y-axis is call stack depth.

Database Fundamentals

A column in one table that references the primary key of another table, enforcing referential integrity between related tables.

Load Balancing & Proxies

A proxy that sits in front of clients and forwards their requests to the internet. Used for anonymity, content filtering, and bypassing geo-restrictions.

Reliability & Resilience

A planned exercise where teams simulate production failures to test incident response procedures and system resilience. Like a fire drill for your infrastructure.

Data Governance & Compliance

General Data Protection Regulation: EU law governing how personal data is collected, stored, and processed. Requires consent, data portability, and the right to be forgotten.

Load Balancing & Proxies

DNS that returns different IP addresses based on the geographic location of the requester. Routes users to the nearest data center for lower latency.

DevOps & CI/CD

Using Git as the single source of truth for infrastructure and application configuration. Changes are made via pull requests and automatically reconciled by tools like ArgoCD or Flux.

Load Balancing & Proxies

A load balancer that routes users to the nearest data center or region based on geography, latency, or health. DNS-based (Route 53, Cloudflare) or anycast-based.

LLM and GenAI Ops

A curated set of inputs with known-correct expected outputs, used to measure quality and catch regressions when you change a prompt, a model, or the retrieval stack.

Distributed Systems Core

A peer-to-peer communication protocol where nodes share information with random neighbors, spreading it like gossip. Used for cluster membership and failure detection.

Reliability & Resilience

A strategy where a system continues to function with reduced capability when a component fails, instead of crashing entirely. Show cached results when the database is down.

Observability & Monitoring

A visualization platform for creating dashboards from any data source. Connects to Prometheus, Elasticsearch, CloudWatch, and dozens more to display metrics, logs, and traces.

Database Types & Storage

A database built for storing and querying relationships between entities. Nodes are entities, edges are relationships. Neo4j is the most popular.

API Design & Protocols

A query language for APIs where the client specifies exactly what data it needs. No over-fetching, no under-fetching. One endpoint to rule them all.

LLM and GenAI Ops

Requiring a model to answer from provided evidence and to cite it, rather than from its own memory. The core defense against hallucination in a RAG system.

API Design & Protocols

A high-performance RPC framework by Google using Protocol Buffers and HTTP/2. Much faster than REST for service-to-service communication.

LLM and GenAI Ops

The checks around a model that screen its input for abuse and its output for grounding and safety, plus limits on what its tools can do. Containment, not just filtering, is what actually holds.

LLM and GenAI Ops

When a language model produces fluent, confident text that is not supported by any source and is often simply false. Retrieval, grounding, and evaluation are the main defenses.

Database Fundamentals

An index that uses a hash function to map keys directly to storage locations. O(1) lookups for exact matches but useless for range queries. Memcached and Redis use hash indexes.

Observability & Monitoring

An endpoint or mechanism that reports whether a service is running and healthy. Load balancers use health checks to route traffic away from unhealthy instances.

Load Balancing & Proxies

Periodic probes a load balancer sends to backend servers to verify they're alive. Unhealthy servers get pulled from the rotation until they recover.

Kubernetes & Containers

A package manager for Kubernetes that bundles K8s manifests into reusable charts. Lets you install complex applications with a single command and manage versioned releases.

Consistency Models

When a replica is temporarily down, another node stores writes intended for it as 'hints.' Once the replica recovers, the hints are replayed to catch it up.

LLM and GenAI Ops

Hierarchical Navigable Small World, a graph-based nearest neighbor index that gives high recall at low latency by navigating layers of links, at the cost of more memory than IVF.

Kubernetes & Containers

A K8s controller that automatically scales the number of pods based on CPU utilization, memory usage, or custom metrics. The Kubernetes equivalent of auto-scaling groups.

Core Fundamentals

Adding more machines to handle increased load (scaling out). Like opening more checkout lanes instead of making one cashier faster.

Distributed Systems Core

An uneven distribution of load where one node, shard, or partition receives disproportionately more traffic than others. Caused by poor shard keys or skewed access patterns.

Core Fundamentals

The protocol powering the web. A request-response model where clients ask for resources and servers respond. Stateless by design.

Web & Content Delivery

Browser and proxy caching controlled by HTTP headers like Cache-Control, ETag, and Last-Modified. Eliminates redundant network requests for unchanged resources.

API Design & Protocols

A major revision of HTTP that adds multiplexing (multiple requests over one connection), header compression, and server push. Much faster than HTTP/1.1 for modern web apps.

Core Fundamentals

HTTP over TLS: the encrypted version of HTTP that protects data in transit. Every production site should use it; browsers flag plain HTTP as insecure.

LLM and GenAI Ops

Running dense vector search and sparse keyword search together and fusing the results, so exact terms like an error code and paraphrased meaning both count.

Cloud Infrastructure

Software that creates and manages virtual machines by abstracting physical hardware. Type 1 (bare-metal) runs directly on hardware; Type 2 runs on an OS.

Core Fundamentals

An operation that produces the same result whether you run it once or multiple times. Critical for safe retries in distributed systems.

Stream & Batch Processing

A message consumer that produces the same result whether it processes a message once or multiple times. Achieved by tracking processed message IDs or using natural idempotency.

API Design & Protocols

Interface Definition Language: a schema that defines the contract between client and server. Protobuf, Thrift, and OpenAPI are IDLs that generate client/server code.

DevOps & CI/CD

Servers are never modified after deployment. To update, you build a new image and replace the old instance entirely. Eliminates configuration drift.

Security Testing & Operations

The structured process for detecting, containing, eradicating, and recovering from security incidents. Includes communication plans, runbooks, and post-incident reviews.

Database Fundamentals

A data structure that speeds up database lookups. Like the index at the back of a book that lets you jump to the right page instead of reading every page.

LLM and GenAI Ops

The engineering that serves more tokens per GPU-second: KV caching, continuous batching, quantization, prefix caching, and speculative decoding. Serving cost is dominated by how well these are done.

Cloud Infrastructure

Managing servers, networks, and cloud resources through declarative configuration files instead of manual setup. Terraform, Pulumi, and CloudFormation are IaC tools.

Messaging & Event Systems

Reusable designs for connecting separate systems through messages, such as the content-based router, splitter, aggregator and normalizer, catalogued as Enterprise Integration Patterns.

Core Fundamentals

Internet Protocol: the addressing scheme that routes packets across the internet. Every device gets an IP address (IPv4 or IPv6) so packets know where to go.

Database Fundamentals

Controls how much one transaction can see changes made by other concurrent transactions. Ranges from Read Uncommitted (fastest, least safe) to Serializable (slowest, safest).

Web & Content Delivery

Incremental Static Regeneration: a Next.js feature that re-generates static pages in the background after a specified time interval, combining SSG speed with fresh data.

Core Fundamentals

JavaScript Object Notation: a lightweight text format for data interchange using key-value pairs and arrays. The lingua franca of web APIs.

Security Architecture

JSON Web Token: a compact, self-contained token for transmitting claims between parties. The server can verify it without a database lookup.

Messaging & Event Systems

A distributed event streaming platform that handles millions of events per second. Used by LinkedIn, Netflix, and Uber for real-time data pipelines.

Messaging & Event Systems

A set of consumers that cooperatively read from topic partitions. Each partition is assigned to exactly one consumer in the group, enabling parallel processing.

Messaging & Event Systems

A subset of a Kafka topic that provides ordering guarantees and parallel processing. Each partition lives on one broker and can be consumed by one consumer per group.

Messaging & Event Systems

A named feed or category to which producers publish records. Topics are split into partitions for parallelism, and each partition is an ordered, immutable log.

Database Types & Storage

The simplest NoSQL model: store data as key-value pairs. Blazing fast lookups by key. Redis, DynamoDB, and etcd are key-value stores.

Kubernetes & Containers

An orchestration platform that automates deploying, scaling, and managing containerized applications. K8s is the operating system for your cloud.

Kubernetes & Containers

A K8s resource that stores non-sensitive configuration data as key-value pairs. Pods read ConfigMaps as environment variables or mounted files, decoupling config from code.

Kubernetes & Containers

A K8s resource that manages rolling updates and rollbacks for a set of pods. You declare the desired state and K8s converges to it.

Kubernetes & Containers

A K8s resource that manages external HTTP/HTTPS access to services inside the cluster. Routes traffic by hostname or URL path to different backend services.

Kubernetes & Containers

A virtual cluster within a physical K8s cluster that isolates resources. Use namespaces to separate environments (dev, staging, prod) or teams.

Kubernetes & Containers

The smallest deployable unit in Kubernetes: one or more containers sharing network and storage. Pods are ephemeral; if one dies, K8s creates a replacement.

Kubernetes & Containers

A K8s resource for storing sensitive data like passwords, tokens, and certificates. Base64-encoded by default; use encryption at rest and RBAC to properly secure them.

Kubernetes & Containers

A stable network endpoint that load-balances traffic across a set of pods. Pods come and go, but the Service's IP and DNS name stay constant.

LLM and GenAI Ops

The stored keys and values of every token processed so far, kept so attention does not recompute the whole history each step. It grows with every token and is what limits how many conversations a GPU can serve.

Stream & Batch Processing

A data processing design that runs a batch layer for complete, accurate results and a speed layer for fresh, approximate results, then merges the two when a query is answered.

Distributed Systems Core

A simple logical clock where each event increments a counter. If event A causes event B, A's timestamp is always less than B's. The foundation of logical time in distributed systems.

Core Fundamentals

The time delay between sending a request and getting a response. Amazon found every 100ms of extra latency costs 1% in sales.

Load Balancing & Proxies

Load balancing at the transport layer (TCP/UDP) based on IP addresses and ports. Fast because it doesn't inspect packet contents.

Load Balancing & Proxies

Load balancing at the application layer (HTTP) that can route based on URL paths, headers, cookies, or request content. More flexible but more CPU-intensive.

Distributed Systems Core

The process of choosing one node in a cluster to coordinate actions. If the leader fails, a new one is elected. Used by Kafka, ZooKeeper, and etcd.

Data Replication & Distribution

A replication approach where any node can accept reads and writes. Uses quorum reads/writes for consistency. Cassandra and DynamoDB use this model.

LLM and GenAI Ops

The dangerous combination of access to private data, exposure to untrusted content, and a way to exfiltrate. An agent with all three can be turned against its own user by a prompt injection.

Consistency Models

The strongest consistency guarantee: every operation appears to take effect atomically at some point between its invocation and completion. As if there's a single copy of the data.

LLM and GenAI Ops

Reusing earlier work to avoid paying for the same model call twice: exact caches for identical requests, semantic caches for similar ones, and prompt caching on the provider's side.

LLM and GenAI Ops

Measuring whether a model or pipeline actually works, using error analysis, labeled datasets, and validated judges rather than a glance. The single most in-demand and least-taught AI engineering skill.

LLM and GenAI Ops

Recording what an LLM application actually did on each request, the prompts, model calls, tool calls, tokens, cost and latency, so you can explain and fix a bad answer.

LLM and GenAI Ops

A component that decides which language model should answer each request, so easy requests go to a cheap model and hard ones to a strong model.

LLM and GenAI Ops

A setting that controls how random a language model's word choice is. Low temperature picks the most likely next token almost every time; high temperature spreads the choice wider.

LLM and GenAI Ops

Using a language model to score another model's output. Fast and scalable, but it has real failure modes such as self-preference and position bias, so the judge itself must be validated against human labels.

Core Fundamentals

Distributes incoming traffic across multiple servers so no single server gets overwhelmed. Like a traffic cop directing cars to different lanes.

Load Balancing & Proxies

The strategy a load balancer uses to distribute requests. Round-robin, least-connections, weighted, IP-hash, and random are common algorithms, each with different trade-offs.

Reliability & Resilience

Deliberately dropping low-priority requests during overload to protect the system's ability to serve high-priority traffic. Better to serve some requests than crash serving none.

Observability & Monitoring

Collecting logs from all services into a central searchable store. The ELK stack (Elasticsearch, Logstash, Kibana) and Grafana Loki are common solutions.

Observability & Monitoring

Recording discrete events with timestamps, severity levels, and context. Structured logs (JSON) are searchable; unstructured logs (plaintext) are not. Ship them to a central system.

API Design & Protocols

The client sends a request and the server holds it open until new data is available or a timeout is reached. A workaround for real-time updates before WebSockets existed.

LLM and GenAI Ops

A parameter-efficient fine-tuning method that freezes the base weights and trains a small pair of low-rank matrices instead, cutting memory and cost while keeping most of the quality.

Caching Strategies

A cache that, when full, throws away the item nobody has used for the longest time. It keeps recently used data on the bet that it will be used again soon.

Database Types & Storage

Log-Structured Merge Tree: a write-optimized data structure that buffers writes in memory and periodically flushes sorted runs to disk. Used by Cassandra, RocksDB, and LevelDB.

Stream & Batch Processing

A programming model for processing massive datasets in parallel across a cluster. Map splits data into key-value pairs; Reduce aggregates them. Pioneered by Google, implemented by Hadoop.

Database Types & Storage

A precomputed query result stored as a physical table and refreshed periodically. Trades storage for read performance on expensive aggregations.

LLM and GenAI Ops

Embeddings trained so their first numbers carry the most meaning, which lets you cut them shorter to save storage and search time while losing little accuracy.

LLM and GenAI Ops

An open protocol that lets AI apps such as Claude, ChatGPT, Cursor and VS Code use outside tools and data. An MCP server offers the tools once, and any app with an MCP client can find and call them.

Database Types & Storage

A simple, high-performance distributed memory caching system. Stores key-value pairs in RAM. Simpler than Redis but less feature-rich.

Messaging & Event Systems

Middleware that translates messages between different protocols, routes them to the right consumers, and provides guarantees like ordering and delivery. RabbitMQ and ActiveMQ are brokers.

Messaging & Event Systems

A buffer that stores messages between producers and consumers. Messages are processed one by one, in order. Think of it as a to-do list for your services.

Observability & Monitoring

Numerical measurements collected over time that describe system behavior: request rate, error rate, latency percentiles, CPU utilization. Prometheus is the standard collector.

Microservices Architecture

An architecture where an application is split into small, independent services that communicate over the network. Each service owns its own data and can be deployed separately.

Microservices Architecture

The recurring designs teams use to build and run microservices, such as API gateway, service discovery, circuit breaker, database per service, saga and event-driven messaging.

Core

When a model's accuracy decays over time because the live data has moved away from what it was trained on. Detected by monitoring inputs and outputs, and fixed by retraining.

Core

A versioned catalog of trained models with their metadata, stage, and lineage, so you know exactly what is deployed and can roll back. The control plane for shipping models safely.

Core

Exposing a trained model behind an API that answers predictions within a latency budget, with batching, autoscaling, and versioning. Where most of the production cost and the request-path complexity lives.

Microservices Architecture

A single, unified application where all features share the same codebase and deployment. Simpler to start with but harder to scale individual parts.

Consistency Models

A guarantee that once you read a value, subsequent reads will never return an older value. Prevents the disorienting experience of data appearing to go backward in time.

Security Architecture

Mutual TLS: both client and server present certificates to authenticate each other. Standard in service mesh architectures where every service verifies its peers.

Reliability & Resilience

Mean Time To Detect: the average time between a failure occurring and being noticed. Shorter MTTD means better monitoring and alerting. You can't fix what you don't know is broken.

Reliability & Resilience

Mean Time To Recovery: the average time from when a failure is detected to when the service is restored. A key reliability metric that drives investment in automation and runbooks.

LLM and GenAI Ops

A design where more than one LLM-driven loop works on one task, each with its own context, usually coordinated by an orchestrator that hands out parts and merges the results. It adds token cost and handoffs that can fail.

Data Replication & Distribution

A replication topology where multiple nodes accept writes independently and sync with each other. Useful for multi-datacenter setups but creates conflict resolution challenges.

Cloud Infrastructure

A single software instance serving multiple customers (tenants) while keeping their data isolated. Can be achieved at the database, schema, or row level.

Cloud Infrastructure

Network Address Translation gateway: allows instances in a private subnet to access the internet while preventing inbound connections. Keeps your backend servers invisible to the public.

Core Fundamentals

A break in communication between nodes in a distributed system. Some nodes can't reach others. The 'P' in CAP theorem that forces the trade-off between consistency and availability.

Database Fundamentals

Organizing database tables to reduce redundancy by splitting data into related tables connected by foreign keys. Follows normal forms (1NF, 2NF, 3NF).

Database Fundamentals

Databases that don't use traditional table-based relational models. Includes document stores, key-value, graph, and column-family databases.

Security Architecture

An authorization framework that lets users grant third-party apps limited access to their accounts without sharing passwords. Powers 'Sign in with Google.'

Database Types & Storage

A storage architecture that manages data as objects (file + metadata + ID) rather than blocks or files. S3 is the gold standard. Infinitely scalable, cheap, and durable.

Observability & Monitoring

The ability to understand a system's internal state from its external outputs. Built on three pillars: metrics, logs, and traces.

Microservices Architecture

A behavioral design pattern where an object keeps a list of subscribers and notifies each one when its state changes, so the object does not need to know what the subscribers do.

Observability & Monitoring

A vendor-neutral open standard for collecting metrics, logs, and traces from applications. Provides SDKs and a collector that ships telemetry to any observability backend.

Database Fundamentals

Object-Relational Mapping: a library that lets you interact with a database using your programming language's objects instead of raw SQL. Drizzle, Prisma, and SQLAlchemy are ORMs.

Distributed Systems Core

A pattern where a service writes events to an outbox table in the same database transaction as its state change. A separate process reads the outbox and publishes events, ensuring atomicity.

LLM and GenAI Ops

A way of storing a language model's KV cache in small fixed-size blocks instead of one big reserved slab, so GPU memory is not wasted and more requests fit at once.

API Design & Protocols

Breaking large result sets into smaller pages. Offset-based (page=2&limit=20) is simple; cursor-based (after=abc123) handles real-time data without skipping or duplicating items.

Distributed Systems Core

A family of protocols for solving consensus in unreliable networks. Famously difficult to understand but mathematically proven correct.

Core Fundamentals

A decentralized architecture where each node acts as both client and server. No central authority. Used by BitTorrent, blockchain, and WebRTC.

LLM and GenAI Ops

A family of methods, including LoRA and QLoRA, that adapt a large model by training a tiny fraction of its parameters, so tuning fits on a single GPU instead of a cluster.

Security Testing & Operations

Authorized simulated attacks against a system to find security vulnerabilities before real attackers do. White-hat hackers probe for weaknesses in a controlled environment.

Data Governance & Compliance

Personally Identifiable Information: any data that can identify a specific individual, like name, email, SSN, or IP address. Must be encrypted and access-controlled.

Reliability & Resilience

A blameless analysis conducted after an incident to document what happened, why, and how to prevent it from recurring. The most important output is the list of action items.

Database Fundamentals

A column (or combination of columns) that uniquely identifies each row in a database table. Must be unique and not null.

Data Replication & Distribution

A replication topology where one node (primary) handles writes and replicates changes to one or more read-only replicas. The foundation of most database scaling strategies.

Observability & Monitoring

An open-source monitoring system that scrapes metrics endpoints, stores time-series data, and supports powerful PromQL queries. The de facto standard for Kubernetes monitoring.

LLM and GenAI Ops

Writing and structuring the instructions given to a language model to get reliable output. In 2026 the craft has moved from clever wording to schemas, context layout, and evaluation.

LLM and GenAI Ops

An attack where untrusted text, often returned by a tool, carries an instruction the model obeys, because the model cannot reliably tell instructions from data. The top security risk for agents, and unsolved by prompting alone.

API Design & Protocols

Google's language-neutral, binary serialization format. Smaller and faster than JSON. Defines schemas in .proto files that generate typed code for any language.

Load Balancing & Proxies

An intermediary server that sits between the client and the destination server. Forward proxies act on behalf of clients; reverse proxies act on behalf of servers.

Microservices Architecture

A structural design pattern that puts a stand-in object with the same interface in front of a real object to control access to it, for example to cache, check permissions or load it lazily.

Messaging & Event Systems

A messaging pattern where publishers send messages to topics, and subscribers receive messages from topics they care about. Publishers don't know who's listening.

LLM and GenAI Ops

Storing model weights in fewer bits, such as FP8 or INT8, to shrink memory and speed up inference. Modern methods lose little quality on most tasks.

Distributed Systems Core

The minimum number of nodes that must agree for a read or write to succeed. With N replicas, W+R > N ensures overlap between write and read sets for consistency.

Distributed Systems Core

A consensus algorithm designed to be understandable. Uses leader election and log replication. Powers etcd (used by Kubernetes) and CockroachDB.

LLM and GenAI Ops

A pattern that retrieves relevant text from an external index and puts it in the model's prompt so the answer is grounded in current, verifiable facts rather than the model's memory.

LLM and GenAI Ops

Measuring a retrieval-augmented generation system in two parts, whether retrieval found the right text and whether the answer used it, so a failure can be traced to one of them.

API Design & Protocols

Controlling how many requests a client can make in a given time window. Protects your API from abuse and ensures fair usage.

Security Architecture

Role-Based Access Control: assigns permissions to roles (admin, editor, viewer), then assigns roles to users. Simpler to manage than per-user permissions.

LLM and GenAI Ops

An AI agent that alternates reasoning and acting: it writes a thought, calls a tool, reads the result, and repeats until it can answer. ReAct is short for reason plus act.

Consistency Models

A technique where a read operation detects stale data on a replica and triggers a background update to bring it in sync. Used by Cassandra and DynamoDB to heal inconsistencies lazily.

Data Replication & Distribution

A copy of your database that handles read queries, reducing load on the primary database. Writes still go to the primary and replicate out.

Consistency Models

A consistency guarantee that after a user performs a write, their subsequent reads will always reflect that write. Without it, a user might save data and see the old version.

Caching Strategies

A cache that loads missing data from the database by itself, through one loader function, so your code only ever asks the cache. What the loader does when the database fails decides whether an outage shows up as errors or as wrong answers.

Database Types & Storage

An in-memory data store used as a cache, message broker, and database. Blazing fast because everything lives in RAM.

Database Types & Storage

Redis supports strings, lists, sets, sorted sets, hashes, streams, bitmaps, and HyperLogLogs. Each structure solves different problems: sorted sets for leaderboards, streams for event logs.

Core Fundamentals

Duplicating critical components or functions so that if one fails, a backup takes over. The reason planes have two engines and databases have replicas.

Database Fundamentals

Keeping copies of the same data on multiple servers. Improves read performance and provides fault tolerance if one server goes down.

Core Fundamentals

Request-response is the most basic way two programs talk. One side sends a request, then waits for exactly one reply before it moves on. HTTP, REST and simple gRPC calls all work this way.

LLM and GenAI Ops

A second retrieval stage where a cross-encoder scores each candidate chunk against the query directly. Slower than the first-stage retriever but far more precise, so it runs only over the top few dozen results.

Core Fundamentals

An architectural style for building APIs using standard HTTP methods (GET, POST, PUT, DELETE). Resources are identified by URLs.

Microservices Architecture

Automatically re-attempting a failed operation, usually with exponential backoff. Essential for handling transient failures in distributed systems.

Load Balancing & Proxies

A server in front of your web servers that receives every request first, passes it to one of them and returns the answer. It handles TLS, load balancing, caching and routing, and hides the servers behind it.

DevOps & CI/CD

Gradually replacing old instances with new ones, a few at a time. No downtime, but both versions run simultaneously during the rollout.

Load Balancing & Proxies

A load balancing algorithm that distributes requests to servers in sequential order, cycling through the list. Simple but ignores server capacity differences.

Reliability & Resilience

Recovery Point Objective: the maximum acceptable amount of data loss measured in time. An RPO of 1 hour means you can afford to lose up to 1 hour of data.

Reliability & Resilience

Recovery Time Objective: the maximum acceptable downtime after a disaster. An RTO of 15 minutes means the system must be back online within 15 minutes of failure.

Reliability & Resilience

A documented set of step-by-step procedures for handling specific operational tasks or incidents. Good runbooks reduce MTTR by giving on-call engineers a clear action plan.

Microservices Architecture

A saga implementation where each service listens for events and decides what to do next independently. No central coordinator. Looser coupling but harder to trace.

Microservices Architecture

A saga implementation where a central orchestrator tells each service what step to execute next. Easier to debug than choreography but creates a single point of coordination.

Distributed Systems Core

A way to manage distributed transactions across microservices using a sequence of local transactions with compensating actions for rollback.

Core Fundamentals

A system's ability to handle growing amounts of work by adding resources. A scalable system maintains performance as load increases.

Stream & Batch Processing

A centralized service that stores and validates schemas for event-driven systems. Confluent Schema Registry ensures Kafka producers and consumers agree on data formats.

Security Architecture

Securely storing and distributing credentials, API keys, and certificates. Tools like Vault, AWS Secrets Manager, and SOPS prevent secrets from leaking into code or logs.

Security Testing & Operations

A systematic evaluation of a system's security posture against standards and best practices. Covers access controls, encryption, logging, vulnerability management, and compliance.

LLM and GenAI Ops

Search that matches meaning instead of exact words. Text is turned into embeddings, and the results are the documents whose embeddings sit closest to the query's.

Consistency Models

A guarantee that concurrent transactions produce the same result as if they were executed one at a time in some serial order. The gold standard for database transaction isolation.

Core Fundamentals

Converting an in-memory data structure into a byte stream for storage or network transmission. Deserialization is the reverse. JSON, Protobuf, and Avro are serialization formats.

Cloud Infrastructure

A cloud execution model where the provider manages all infrastructure and you pay only for actual compute time. AWS Lambda, Vercel Functions, and Cloudflare Workers are serverless.

Microservices Architecture

The mechanism by which microservices find and communicate with each other. Services register themselves and others can look them up by name.

Microservices Architecture

A dedicated infrastructure layer for handling service-to-service communication in microservices. Manages load balancing, encryption, and observability automatically.

Web & Content Delivery

A script the browser runs in the background, separate from the web page. Enables offline caching, push notifications, and background sync for progressive web apps.

Core Fundamentals

A way to maintain state across multiple HTTP requests. The server stores data about a user and gives them a session ID (usually in a cookie).

Distributed Systems Core

The column or field used to determine which shard a piece of data belongs to. A bad shard key creates hot spots; a good one distributes data and queries evenly.

Database Fundamentals

Splitting a database into smaller pieces (shards) distributed across multiple servers. Each shard holds a subset of the data.

Microservices Architecture

A helper container deployed alongside the main application container in the same pod. Handles cross-cutting concerns like logging, monitoring, or TLS without modifying app code.

Microservices Architecture

A creational design pattern that makes sure a class has exactly one instance and gives the whole program one shared way to reach it, used for things like a connection pool or a config object.

Reliability & Resilience

Service Level Agreement: a contractual commitment between provider and customer specifying uptime, response time, and penalties for breaches. The business version of an SLO.

Reliability & Resilience

Service Level Indicator: a quantitative measure of service behavior, like the proportion of requests faster than 300ms. The raw metric that feeds SLOs.

Security Architecture

A rate limiting algorithm that tracks requests in a rolling time window. More accurate than fixed windows because it smooths out spikes at window boundaries.

Reliability & Resilience

Service Level Objective: a target value for an SLI, like '99.9% of requests under 300ms.' The internal engineering goal that drives reliability investment.

DevOps & CI/CD

A quick, basic test run after deployment to verify the most critical paths still work. If the smoke test fails, the deployment is rolled back immediately.

Consistency Models

A consistency level where each transaction reads from a consistent snapshot of the database taken at the transaction's start time. Prevents dirty reads without full serializability overhead.

Observability & Monitoring

A single unit of work in a distributed trace, representing one operation (e.g., an HTTP call or database query). Spans are nested to form a trace tree.

Distributed Systems Core

A failure scenario where a network partition causes two halves of a cluster to operate independently, each believing it's the leader. Can cause data corruption if not handled.

Database Fundamentals

Structured Query Language for managing relational databases. Tables, rows, columns, and powerful joins to query related data.

Security Architecture

An attack where malicious SQL is inserted into a query through user input. Prevented by parameterized queries and prepared statements. Never concatenate user input into SQL.

Reliability & Resilience

Google's discipline for running reliable production systems. Applies software engineering to operations: automation over toil, SLOs over uptime promises, and error budgets for velocity.

API Design & Protocols

Server-Sent Events: a one-way channel where the server pushes updates to the client over HTTP. Simpler than WebSockets when you only need server-to-client streaming.

Load Balancing & Proxies

Decrypting TLS-encrypted traffic at a load balancer or reverse proxy so backend servers receive plain HTTP. Offloads CPU-intensive crypto work and simplifies certificate management.

Security Architecture

Cryptographic protocols that encrypt data in transit between client and server. TLS is the modern successor to SSL. The 'S' in HTTPS.

Web & Content Delivery

Server-Side Rendering: generating HTML on the server for each request. Slower than SSG but always returns fresh data. Good for personalized or frequently changing pages.

Distributed Systems Core

Running the same deterministic state machine on multiple nodes, applying the same commands in the same order. If they start in the same state and see the same inputs, they stay in sync.

Distributed Systems Core

A behavioral design pattern where an object delegates its behavior to a separate object for its current state and swaps that object when the state changes, a clean way to build a finite state machine.

Core Fundamentals

A system that remembers previous interactions. The server keeps track of client state between requests, making it harder to scale but sometimes necessary.

Core Fundamentals

A system where each request contains all the information needed to process it. The server doesn't remember previous requests. Easier to scale horizontally.

Web & Content Delivery

Pre-rendering pages to static HTML at build time. The fastest possible page loads because there's no server-side computation on each request.

Load Balancing & Proxies

A load balancer feature that routes all requests from the same client to the same backend server. Needed when servers store session state locally.

Database Fundamentals

Pre-compiled SQL code stored in the database that can be called by name. Reduces network round trips and keeps business logic close to the data.

Microservices Architecture

A migration strategy where you gradually replace a monolith by routing features one by one to new microservices, until the old system can be decommissioned.

Microservices Architecture

A behavioral design pattern that puts each version of an algorithm in its own class behind one interface, so code can swap the algorithm at runtime instead of growing an if-else chain.

Stream & Batch Processing

Processing data continuously as it arrives, rather than in batches. Powers real-time analytics, fraud detection, and live dashboards.

Consistency Models

A guarantee that after a write completes, all subsequent reads will return the updated value. Safer but slower than eventual consistency.

Microservices Architecture

The family of design patterns about how classes and objects fit together: Adapter, Bridge, Composite, Decorator, Facade, Flyweight and Proxy. They change how pieces connect without changing what each piece does.

LLM and GenAI Ops

Constraining a model to return valid JSON matching a schema, so downstream code can rely on it. The foundation of reliable tool calling.

Core Fundamentals

A communication model where the caller waits for the operation to complete before moving on. Simpler to reason about but blocks the thread.

Reliability & Resilience

The high-percentile response times (p99, p99.9) that affect the slowest requests. A system with 10ms median but 2s p99 latency feels slow for 1 in 100 users.

Core Fundamentals

A reliable transport protocol that guarantees data arrives in order and without errors. It uses a three-way handshake to establish connections.

Cloud Infrastructure

An open-source IaC tool by HashiCorp that provisions infrastructure across any cloud provider using declarative HCL configuration. Plan, apply, destroy.

Security Testing & Operations

A structured process for identifying security threats, attack surfaces, and mitigations during system design. STRIDE and DREAD are common frameworks.

Distributed Systems Core

An extension of two-phase commit that adds a pre-commit phase to reduce the blocking window. Still not widely used in practice due to complexity; sagas are preferred.

Security Architecture

Slowing down the rate of processing requests instead of rejecting them outright. The gentler cousin of rate limiting.

Core Fundamentals

The number of operations a system can handle per unit of time. Think of it as how many cars a highway can move per hour.

Reliability & Resilience

When many clients simultaneously retry or reconnect after a failure, overwhelming the recovering system. Solved by jittered backoff, request coalescing, and admission control.

Database Types & Storage

A database optimized for time-stamped data points like metrics, sensor readings, and financial ticks. InfluxDB and TimescaleDB are purpose-built for this.

Microservices Architecture

A maximum duration to wait for an operation to complete before giving up. Without timeouts, a stalled dependency can hang your entire system indefinitely.

Reliability & Resilience

Manual, repetitive, automatable operational work that scales linearly with service size. SRE teams aim to keep toil below 50% of their time and automate the rest.

LLM and GenAI Ops

The unit a language model reads and writes, roughly a few characters. Cost and context limits are counted in tokens, not words, so token accounting is how you budget an LLM system.

Security Architecture

A rate limiting algorithm where tokens are added to a bucket at a fixed rate. Each request consumes a token; requests are rejected when the bucket is empty. Allows short bursts.

Consistency Models

A marker that indicates a deleted record in a distributed database. Since replicas may not see the delete immediately, the tombstone prevents deleted data from being resurrected.

LLM and GenAI Ops

When a model emits a structured request to run a function, and an orchestrator executes it and feeds the result back. The model never runs anything itself, so the orchestrator validates every call.

Observability & Monitoring

The full end-to-end path of a request through a distributed system, composed of multiple spans. A trace shows which services were called, in what order, and how long each took.

Core

When a feature is computed one way in training and a slightly different way in production, so every prediction is subtly wrong. Nothing crashes, accuracy just leaks away. Feature stores exist to prevent it.

Database Fundamentals

A sequence of database operations treated as a single atomic unit. Either all operations succeed (commit) or none of them do (rollback).

DevOps & CI/CD

A branching model where developers commit to a single main branch frequently with small changes. Feature flags replace long-lived feature branches.

Caching Strategies

Time To Live: how long a cached entry, DNS record, or packet is valid before it expires and must be refreshed or discarded.

Cloud Infrastructure

A methodology for building cloud-native applications with 12 principles: codebase, dependencies, config, backing services, build/release/run, processes, port binding, concurrency, disposability, dev/prod parity, logs, admin processes.

Distributed Systems Core

A protocol ensuring all nodes in a distributed transaction either commit or abort together. Phase 1: prepare (vote). Phase 2: commit or rollback.

Core Fundamentals

User Datagram Protocol: a connectionless transport protocol that trades reliability for speed. No handshake, no ordering, no retransmission. Used for video streaming and gaming.

Distributed Systems Core

A logical clock that tracks causality across distributed nodes using a vector of counters. Each node increments its own counter and merges vectors on message receipt.

Distributed Systems Core

A database that stores embeddings, lists of numbers that capture the meaning of text or images, and finds the ones closest to a question's embedding. It powers semantic search and RAG.

Core Fundamentals

Making a single machine more powerful (more CPU, RAM, storage). Simpler but has physical limits. Also called 'scaling up.'

Cloud Infrastructure

A software emulation of a physical computer running its own OS on shared hardware. Heavier than containers but provides stronger isolation.

Cloud Infrastructure

Virtual Private Cloud: a logically isolated section of the cloud where you launch resources in a virtual network you define. Controls IP ranges, subnets, route tables, and gateways.

Security Architecture

Web Application Firewall: filters and monitors HTTP traffic between a web application and the internet. Blocks SQL injection, XSS, and other OWASP top-10 attacks.

Stream & Batch Processing

A timestamp that tracks how far a stream processing system has progressed through event time. Tells the system when it's safe to close a window and emit results, even with late-arriving data.

Messaging & Event Systems

An HTTP callback triggered by an event. Instead of polling for updates, the source system pushes a notification to your URL when something happens.

API Design & Protocols

A protocol for full-duplex communication over a single TCP connection. Unlike HTTP, the server can push data to the client without being asked.

Stream & Batch Processing

Grouping stream events into finite time-based or count-based windows for aggregation. Tumbling windows don't overlap; sliding windows do; session windows group by activity gaps.

Data Replication & Distribution

A technique where changes are written to a log before being applied to the database. Ensures durability and crash recovery.

Caching Strategies

A caching pattern where writes go to the cache first and are asynchronously flushed to the database later. Fast writes but risks data loss on cache failure.

Caching Strategies

A caching pattern where every write goes to both the cache and the database simultaneously. Ensures consistency but adds write latency.

Core Fundamentals

Extensible Markup Language: a verbose, tag-based format for structured data. Still used in enterprise systems, SOAP, and configuration files.

Security Architecture

Cross-Site Scripting: an attack where malicious scripts are injected into trusted websites. Prevented by sanitizing user input and setting Content-Security-Policy headers.

Security Architecture

A security model that never trusts any request by default, even from inside the network. Every request must be authenticated, authorized, and encrypted regardless of origin.