We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design and AI Engineering, by Roni Das. 984 interactive lessons.
Start LearningIntermediate · Messaging & Event Systems
A messaging pattern where publishers send messages to topics, and subscribers receive messages from topics they care about. Publishers don't know who's listening.
In short
Pub/Sub (publish/subscribe) is a messaging pattern where senders called publishers write messages to a named channel called a topic, and any number of receivers called subscribers read messages from the topics they care about. The publisher never addresses a specific subscriber, so the two sides are fully decoupled and can be added, removed, or scaled independently.
In short
Pub/Sub (publish/subscribe) is a messaging pattern where senders called publishers write messages to a named channel called a topic, and any number of receivers called subscribers read messages from the topics they care about. The publisher never addresses a specific subscriber, so the two sides are fully decoupled and can be added, removed, or scaled independently.
Learn it in the course
This page is the free part.
₹1,111 in India$25 everywhere elseonce, for the whole course
The System Design course covers Pub/Sub across a run of lessons, not one page. These 4 alone are about 78 minutes of step-by-step reading, every one with a quiz.
One publisher, many subscribers, the pattern that powers event-driven systems at scale.
Guarantee that your database write and message publish either both happen or neither does.
Break a complex message into smaller pieces that can be processed independently.
A central channel where components publish and subscribe to events without knowing each other.
all part of
System Design Masterclass
770 lessons · about 287 hours · 18 free to read
₹1,111in India, by UPI
$25everywhere else, by PayPal
It is really nice and good Please join the course
You own this course
Continue with Pub/Sub Patternpay once,
yours for life
In a direct request, service A calls service B and waits for an answer. The two are tightly coupled: A needs B's address, B must be running, and A blocks until B replies. Pub/Sub breaks that coupling. A publisher sends a message to a topic and moves on. It does not know how many subscribers exist, what they do, or whether any exist at all.
A topic is just a named stream. Subscribers register interest in a topic, and the messaging system delivers every matching message to each of them. One message can fan out to zero, one, or a thousand subscribers, and the publisher's code never changes when you add another consumer.
This is a one-to-many broadcast model, which is what separates it from a plain point-to-point queue. In a work queue, one message goes to exactly one worker. In Pub/Sub, the same message is delivered to every subscriber that asked for the topic.
Between publishers and subscribers sits a broker or message bus. The publisher hands a message to the broker, the broker accepts it (durable brokers also persist it), and then the broker is responsible for getting a copy to each subscriber. Redis Pub/Sub, Google Cloud Pub/Sub, Apache Kafka, NATS, and AWS SNS all play this broker role with different guarantees.
Two filtering models are common. Topic-based filtering routes by a channel name, so a subscriber to orders.created gets every message on that channel. Content-based filtering lets subscribers specify a rule, for example only messages where amount is greater than 1000, and the broker evaluates the rule before delivering.
Delivery semantics matter a lot. Plain Redis Pub/Sub is fire-and-forget: if a subscriber is offline when a message is published, it never sees that message. Durable systems like Kafka and Google Cloud Pub/Sub keep messages on disk and track each subscriber's read position, so a subscriber that was down for an hour catches up when it returns. Most durable brokers offer at-least-once delivery, which means a consumer can occasionally receive the same message twice and must handle that idempotently.
Reach for Pub/Sub when one event needs to trigger several independent reactions. A single order-placed event might need to charge a card, update inventory, send a confirmation email, and feed an analytics pipeline. With Pub/Sub the order service publishes one message and stays ignorant of those four consumers, which can be deployed and scaled by separate teams.
The cost of that decoupling is that you give up the immediate, synchronous answer. The publisher learns nothing about whether processing succeeded, so you need separate mechanisms for failures, retries, and dead-letter handling. Debugging is harder too, because a request no longer flows through one call stack you can trace top to bottom.
There is also a delivery-guarantee decision to make up front. Ephemeral Pub/Sub like Redis is fast and simple but loses messages for absent subscribers, which is fine for live notifications and terrible for financial events. Durable Pub/Sub costs more and adds latency but survives outages. Picking the wrong one is a common and expensive mistake.
Picture a ride-hailing app the moment a driver accepts a trip. The trip service publishes one driver_assigned event to a topic. Four subscribers react in parallel: the rider app updates the map with the driver's location, the pricing service locks the fare, the notifications service texts the rider, and the data warehouse records the event for later analysis.
None of these consumers know about each other, and the trip service does not know they exist. If next quarter the company adds fraud detection, that new service simply subscribes to the same topic. No code in the trip service changes, and nothing else is disrupted.
This is exactly why event-driven architectures lean on Pub/Sub. It lets a system grow by adding listeners rather than rewiring callers, which is the difference between a codebase that scales with the org and one that calcifies.
Redis Pub/Sub
Built-in PUBLISH and SUBSCRIBE commands for low-latency, fire-and-forget messaging like live chat and real-time dashboards.
Apache Kafka
Durable, log-based Pub/Sub where topics are partitioned and subscribers track their own offset, used by LinkedIn and Uber for high-throughput event streams.
Google Cloud Pub/Sub
Fully managed broker with at-least-once delivery and message retention, used to glue together Google Cloud services and ingest analytics events.
AWS SNS
Simple Notification Service fans out a single published message to many SQS queues, Lambda functions, and HTTP endpoints at once.
A message queue is point-to-point: each message is consumed by exactly one worker, which is good for distributing work. Pub/Sub is one-to-many: each message is delivered to every subscriber of the topic, which is good for broadcasting an event to many independent consumers.
Yes, Kafka supports the Pub/Sub pattern. Producers write to topics and consumer groups subscribe to them. Unlike ephemeral Pub/Sub, Kafka persists messages to a log and lets each subscriber track its own read offset, so consumers can replay history or catch up after downtime.
It depends on the broker. In ephemeral systems like plain Redis Pub/Sub the message is simply dropped. In durable systems like Kafka or Google Cloud Pub/Sub the message is retained on disk, so a subscriber that connects later can still read it.
Not always, and not globally. Most brokers only guarantee ordering within a single partition or channel, not across the whole topic. If strict ordering matters, you usually route related messages to the same partition using a key, for example the user ID.
Yes. Most durable Pub/Sub systems offer at-least-once delivery, meaning a subscriber may occasionally receive a duplicate after a retry or network hiccup. Consumers should be idempotent, often by recording a unique message ID and ignoring ones they have already processed.
Message Queue
A buffer that stores messages between producers and consumers. Messages are processed one by one, in order. Think of it as a to-do list for your services.
Kafka
A distributed event streaming platform that handles millions of events per second. Used by LinkedIn, Netflix, and Uber for real-time data pipelines.
Event Sourcing
Storing every state change as an immutable event instead of just the current state. You can rebuild any past state by replaying events.
Webhook
An HTTP callback triggered by an event. Instead of polling for updates, the source system pushes a notification to your URL when something happens.
Kafka Topic
A named feed or category to which producers publish records. Topics are split into partitions for parallelism, and each partition is an ordered, immutable log.
Kafka Partition
A subset of a Kafka topic that provides ordering guarantees and parallel processing. Each partition lives on one broker and can be consumed by one consumer per group.
This page explains the idea. The "Pub/Sub Pattern" lesson builds it up step by step, with diagrams. It is part of the System Design Masterclass course, which has 770 lessons. 18 of them are free to read. You pay once and keep it for life, with no subscription.
Open the Pub/Sub Pattern lessoncourse 1
From absolute beginner to principal engineer, drawn step by step.
Asked about Pub/Sub in an interview?
Each question has why it is asked, a full answer, what the interviewer listens for, the follow-ups, the common mistakes and a diagram (268 diagrams in all). A PDF you keep, junior to staff level.
India: UPI, cards or netbanking through Razorpay. Elsewhere: card or PayPal. Download at once from My packs.
