Idempotent Consumer

Press a light switch once, the light turns on. Press it again, the light turns off. That's a toggle, and it's not idempotent.

Now imagine a different switch labeled "ON." Press it once, the light turns on. Press it again, the light is already on, nothing changes. Press it ten more times, still on. The result is always the same regardless of how many times you press it.

That second switch is idempotent. An idempotent operation produces the same result whether you execute it once or a hundred times.

In messaging systems, idempotent consumers are essential because at-least-once delivery means every consumer will eventually receive duplicate messages. Network glitches, consumer crashes, broker rebalances. Duplicates are not a bug, they're a feature of reliable delivery. The question isn't "will I get duplicates?" but "what happens when I do?"

If your consumer is idempotent, duplicates are harmless. If it's not, duplicates cause double charges, duplicate records, incorrect counts, and corrupted state.

Stripe's entire API is designed around idempotency. Every mutating endpoint accepts an Idempotency-Key header. If you send the same request with the same key twice, the second request returns the cached result from the first. They process 250M+ API requests per day. Roughly 250K of those are duplicates caught by idempotency keys. Without this, those 250K requests would have created duplicate charges, duplicate refunds, or duplicate customer records.

Naturally idempotent operations (no extra work needed):

SET user.status = 'active'          -- Same result every time
DELETE FROM cart WHERE id = '123'   -- Second delete is a no-op
PUT /users/123 { name: "Roni" }    -- Overwrites with same data

Naturally non-idempotent operations (need idempotency protection):

INSERT INTO orders (...)                 -- Second insert creates duplicate
UPDATE balance SET amount = amount + 100 -- Second update doubles the credit
counter.increment()                      -- Second call counts too high

Strategies for making non-idempotent operations idempotent:

1. Unique constraint + upsert:

INSERT INTO orders (id, user_id, amount)
VALUES ('order-abc', '123', 99.99)
ON CONFLICT (id) DO NOTHING;
-- Second execution hits the conflict and does nothing

2. Idempotency key (message ID as dedup token):

async function processPayment(msg) {
 const idempotencyKey = msg.id;

 // Atomic: check + process in one transaction
 await db.transaction(async (tx) => {
  const existing = await tx.query(
   'SELECT 1 FROM processed WHERE key = $1', [idempotencyKey]
  );
  if (existing.length > 0) return; // Already processed

  await tx.query('UPDATE balances SET amount = amount + $1 WHERE user_id = $2',
   [msg.body.amount, msg.body.userId]);
  await tx.query('INSERT INTO processed (key) VALUES ($1)', [idempotencyKey]);
 });
}

3. Conditional updates (optimistic locking):

UPDATE orders SET status = 'shipped'
WHERE id = 'order-abc' AND status = 'paid';
-- Only transitions from 'paid' to 'shipped'
-- If already 'shipped', the WHERE clause matches 0 rows = no-op

4. Version numbers:

UPDATE inventory SET quantity = 50, version = 4
WHERE product_id = 'SKU-123' AND version = 3;
-- Only applies if current version is 3
-- Duplicate with same version check fails silently

Shopify's order processing pipeline uses strategy 2 (idempotency keys) for payment mutations and strategy 3 (conditional updates) for order status transitions. The two patterns complement each other: keys for financial operations, conditions for state machines.