We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
This idea carries a full system design question on its own. Each walks through the full answer.
System Design Masterclass · Messaging Event Systems
Free preview: the first 2 parts of this 7-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Messaging Event Systems
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
When you walk into a hospital emergency room, you don't get treated in the order you arrived. A broken finger waits while a heart attack gets immediate attention. The ER triages patients by severity, not arrival time.
Priority queues work the same way for messages. Instead of strict FIFO (first in, first out), messages are assigned a priority level and higher-priority messages jump ahead of lower-priority ones. A payment failure alert gets processed before a marketing analytics event. A VIP customer's support ticket gets routed before a free-tier user's.
The need shows up whenever your system has mixed workloads. Some messages are urgent and time-sensitive, others are important but can wait. Without priority, a flood of low-priority analytics events could block the processing of a critical payment retry for minutes or hours. Stripe's webhook retry system, for example, prioritizes payment_intent.failed events over customer.updated events because a failed payment affects revenue in real time.
Amazon found that every 100ms of additional in order processing reduced conversion by 1%. When a payment retry message sits behind 10,000 analytics events, that latency isn't 100ms. It's minutes.
The producer assigns a priority value (typically 0-9, where higher means more urgent) to each message. The broker reorders delivery so higher-priority messages are consumed first.
Queue contents (arrival order): [P1, P3, P1, P5, P2]
Delivery order (by priority): [P5, P3, P2, P1, P1]
Single queue with priority sorting (RabbitMQ). RabbitMQ supports up to 255 priority levels. Internally, it maintains a separate sub-queue per priority level and always dequeues from the highest non-empty level first. The catch: each priority level creates a separate Erlang process and in-memory list. At 255 levels, memory usage spikes roughly 30% compared to a flat queue. In practice, 3-5 levels is plenty: critical, high, normal, low.
Multiple queues (DIY pattern). Create separate queues for each priority level and have consumers read from the high-priority queue first. If it's empty, read from the next level. This works with any broker, even those without native priority support.
Queue: tasks-critical -> Consumer checks first
Queue: tasks-normal -> Consumer checks second
Queue: tasks-low -> Consumer checks last
Weighted consumption. Instead of strict priority (high always before low), process a ratio. Pull 7 messages from critical, 2 from normal, 1 from low per cycle. This prevents starvation where low-priority messages never get processed during busy periods.
| System | Native Priority Support | Max Levels | Notes |
|---|---|---|---|
| RabbitMQ | Yes (x-max-priority) | 255 (recommend 10 or fewer) | Separate sub-queue per level |
| Amazon SQS | No (use multiple queues) | N/A | Combine with SNS for routing |
| Azure Service Bus | Yes (sessions with priority) | 10 | Priority header on messages |
| No (use multiple topics) | N/A | Consumers poll topics in order | |
| ZADD with score as priority | Unlimited | ZPOPMIN pops highest score first |