We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Data Replication Distribution
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Data Replication Distribution
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
Before Postgres 9.0 introduced streaming in 2010, the only way to replicate a Postgres database was to physically copy WAL segment files from the primary to the standby. Fill up a 16MB WAL file, ship it via rsync or scp or NFS, let the standby replay it. Crude? Yes. But it worked, and it has been proven in production for over two decades.
Log shipping has qualities that streaming replication does not. It is dead simple. It is just file copying. You can implement it with a one-line shell script. It works over any network transport, including air-gapped networks where a USB drive is the "network." It does not require a persistent connection between primary and standby. And it naturally creates a disaster recovery archive as a side effect.
Server still uses log shipping as a primary DR mechanism in 2026. Many enterprise shops run it because their DBAs have operated it for 15+ years and trust it implicitly. PostgreSQL shops that need disaster recovery to a remote site (where millisecond lag does not matter, but durability does) still reach for log shipping. pgBackRest, one of the most popular Postgres backup tools, is built entirely around WAL archiving.
You will encounter log shipping in production. Not as the only replication method, but as a complement to streaming replication. The combination of "streaming for HA, archiving for DR" is the standard playbook recommended by every major Postgres consultancy.
The process is straightforward:
archive_command firesrestore_commandThe key trade-off: The standby is always at least one WAL segment behind the primary. On a busy system writing 16MB of WAL every few seconds, the lag might be seconds. On a quiet system, a single WAL segment might take hours to fill, meaning hours of lag and hours of potential data loss.
Postgres mitigates this with archive_timeout, a setting that forces a WAL segment switch after N seconds, even if the segment is not full. Setting archive_timeout = 60 means the standby is at most 60 seconds behind, even on a database with zero write traffic. The cost is slightly more segment files to archive (partially filled segments).
| Configuration | Lag | RPO | When to Use |
|---|---|---|---|
| No archive_timeout | 1 segment (seconds to hours) | Hours on quiet DBs | DR with relaxed RPO |
| archive_timeout = 300 | Up to 5 minutes | 5 minutes | Warm standby |
| archive_timeout = 60 | Up to 1 minute | 1 minute | Near-real-time DR |
| Streaming | Milliseconds | Sub-second | Real-time HA |