Write-Ahead Logging

Before any change touches actual data pages, it gets written to a sequential log file. If the database crashes mid-write, the log tells recovery exactly what happened: committed changes are replayed, incomplete changes are rolled back. The database comes back in a consistent state every single time.

WAL is the most important concept in database engineering. PostgreSQL WAL, MySQL InnoDB redo log, SQLite journal, etcd WAL, Kafka commit log, CockroachDB Raft log. All implement the same principle. Without WAL, databases would corrupt data on every power failure.

WAL also enables replication (stream the log to replicas), point-in-time recovery (replay to any timestamp), change data capture (read the log to detect changes), and database branching (Neon's page server reads the WAL). One log file, four capabilities.

Rule 1: Log before data. Every modification is written to the WAL and fsynced to disk BEFORE the data page is modified in memory. This is the "write-ahead" part. The WAL entry must be durable before anything else happens.

Rule 2: Commit lives in the log. A commit record in the WAL is the point of no return. Once that record hits disk, the transaction is committed even if no data pages have been updated yet. Data pages are just a cache of what the WAL already contains.

Rule 3: Lazy data writes. Dirty data pages can be flushed to disk later, in any order, by a background writer. This is safe because the WAL contains everything needed to reconstruct any missing updates.

Performance insight: WAL writes are sequential (always appending to the end of a file). Data page writes are random (scattered across the data directory). Sequential I/O is 10-100x faster than random I/O on spinning disks and 3-10x faster on SSDs due to reduced write amplification. A single fast sequential write replaces N slow random writes.