We use analytics cookies.We use cookies to analyze site traffic and improve your experience.
Loading...
System Design Masterclass · Messaging Event Systems
Free preview: the first 2 parts of this 6-part lesson.
Already bought the course? Sign in to read the whole lesson.
this lesson is part of
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design courseread the rest of this lesson in
System Design Masterclass
Unlock this lesson and the 751 other paid lessons
$25 once, by PayPal
See the System Design coursein Messaging Event Systems
This lesson is part of the full System Design Masterclass. Unlock it along with 769 other lessons for a one-time payment.
One-time payment by card or PayPal
Get Lifetime AccessOne-time payment: UPI, card or netbanking
Get Lifetime Access18 Foundation lessons are free, no payment needed. Try them now
It is easy and good
“It is really nice and good Please join the course”
At a fancy restaurant, you don't carry your heavy winter coat to the table. You hand it to the coat check attendant, receive a small ticket with a number, and carry the lightweight ticket. When you leave, you present the ticket and get your coat back.
The claim check pattern works the same way for messages. Instead of stuffing a 50MB file into a message (which most brokers reject or handle poorly), you store the file in external storage (S3, blob storage) and send a lightweight message containing just a reference, a URL or storage key.
Message brokers are designed for small, fast messages, typically kilobytes. When you push megabytes through them, you get slow , high memory usage, broker instability, and often hard rejections. SQS limits messages to 256KB. Kafka's default is 1MB. RabbitMQ slows dramatically with large messages.
The claim check pattern keeps your messaging system lean while supporting arbitrarily large payloads. The message stays small and fast; the payload lives in storage designed for large data.
Producer side (check in):
// 1. Store the large payload in blob storage
const payloadKey = `uploads/${orderId}/invoice.pdf`;
await s3.putObject(bucket, payloadKey, largePayload);
// 2. Send a lightweight message with just the reference
await queue.publish('invoice-processing', {
orderId: 'order-123',
payloadRef: {
storage: 's3',
bucket: 'my-bucket',
key: payloadKey
}
});
Consumer side (claim):
async function processInvoice(msg) {
// 1. Retrieve the payload using the reference
const payload = await s3.getObject(msg.payloadRef.bucket, msg.payloadRef.key);
// 2. Process the full payload
await processInvoiceData(payload);
// 3. Optionally clean up the stored payload
// await s3.deleteObject(msg.payloadRef.bucket, msg.payloadRef.key);
}
What to store externally:
What to keep in the message: