Integration Design Patterns: Routers, Splitters and Normalizers

Most companies run many systems that were never built to talk to each other. A web shop, an old ERP system, a partner's file drop, a payment provider. Integration is the work of making them exchange data reliably.

The common way to do it is messaging. One system puts a message on a channel, such as a message queue or a topic in a message broker, and another system reads it later. Neither has to be online at the same time, and neither needs to know how the other works inside.

Messaging raises the same questions again and again. Where should this message go? What if it holds five items that need different handling? What if three sources send the same thing in three formats? Integration design patterns are named, tested answers to those questions.

The standard catalog is the book Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf, published in 2003. Its site describes it as a pattern language of 65 integration patterns. These are different from the 23 design patterns of the Gang of Four. Those describe how classes inside one program work together. Integration patterns describe how whole systems work together through messages.

The course's messaging chapter covers these patterns. Each line is the job the pattern does, with its lesson.

• Content-based router. Reads each message and sends it to a destination based on what it contains, such as domestic, international, or high-value orders to fraud review.
• Dynamic router. A router whose rules change at runtime, for example when a new participant registers, with no redeploy.
• Message filter. Lets a consumer receive only the messages it cares about, with the filtering done by the broker rather than in consumer code.
• Splitter. Breaks one message holding many items into one message per item, so each can be handled on its own and in parallel.
• Aggregator. Collects related messages and combines them into one. It must answer three questions: which messages belong together, when the group is complete, and how to combine them.
• Scatter-gather. Sends the same request to many providers, collects their answers and picks the best, like a flight price search.
• Resequencer. Buffers messages that arrive out of order and releases them in sequence-number order.
• Routing slip. Attaches the list of steps to the message itself. Each service does its step and forwards the message to the next one on the list, with no central coordinator.
• Process manager. A central component that tracks the state of a multi-step process and decides the next step, including undo steps when something fails.
• Normalizer. Routes each format to its own translator so everything comes out in one common format.
• Canonical data model. One shared format for each business entity, so each service translates only between its own model and the shared one.
• Claim check. Stores a large payload in external storage such as S3 and sends only a small reference through the broker.
• Dead letter queue. Holds messages that could not be processed, so they neither block the queue nor vanish. Lesson.
• Idempotent consumer. A consumer that gives the same result whether a message arrives once or many times, which at-least-once delivery requires. Lesson.

Several patterns look alike. The lessons separate them by intent.

• Aggregator or scatter-gather. An aggregator combines different messages that share a correlation ID. Scatter-gather sends one identical request to many recipients and picks among their answers.
• Normalizer or canonical data model. The canonical data model is the shared format itself. The normalizer is the machinery that converts incoming formats into it. With 20 services, the lesson counts up to 190 point-to-point translations without a canonical model, and 20 with one.
• Routing slip or process manager. A routing slip puts the plan inside the message, which is light but cannot react to results. A process manager holds the plan centrally and can branch and run undo steps.

Do not reach for a pattern when the problem is not there. The normalizer lesson says to skip it when all sources already share a format, or when only one source exists. Skip it too when the only difference is field names and a simple mapping will do. Each pattern adds a moving part to run and monitor.

Many of these patterns also sit inside a wider event-driven architecture or a pub/sub setup, where routers and filters decide which subscribers see which events.

To step through the normalizer flowchart and its comparison with the adapter and transformer, the Normalizer Pattern lesson is the next step.