Enterprise Integration Design Patterns Catalogue

Summary

The section below documents some common Enterprise Integration Patterns (EIP) that can be used across different vendor platforms. The EIPs are considered industry best practices, they have been peer-reviewed by domain experts. For a detailed understanding of the core components visit the Well-defined Integration Architecture post.

*Note, that this catalogue is a work in progress.

EIP Management Systems

TypeDefinitionQuality Attribute
Control Bus (CB)Manage multiple distributed integrations through a single backplane. The CB uses a separate message channel to communicate with endpoints to ensure that all integration components are working.Monitoring
Alerts & NotificationAlerts and Notifications design patterns work with the Control Bus and Retry patterns. When failures occur in an integrated system, the Control Bus issues notifications to Administrators.Monitoring
DetourThe Detour pattern allows a management system to send messages through a separate message channel based on conditions. This Pattern is useful for re-routing messages from the main channel.
Message HistoryEnsures that messages are persisted in storage. Commonly used in Event-driven and Event-Sourcing architectures.Monitoring
Message StoreStore a Command, Event, or Query message in persistent storage for auditing purposes.
Smart ProxyThe Smart Proxy intercepts messages sent by a Sender on a messaging channel. The Proxy then replaces the Return Address with its own address or another address the Proxy is listening in on. When the Receiver responds to the message by sending the response message to the Return Address, the Proxy can then act on behalf of the Sender.

Smart Proxies are good for Message Routing scenarios. Most enterprise integration tools like Azure Service Bus and WSO2 support Smart Proxies.
Test MessageTest Message is an EIP that is designed to monitor the health and stability of any enterprise integration. This can sometimes exist native to the integration platform or can be built to work adjacent to a core service. Test Message EIP sends messages from a Sender to a Receiver without invoking any business logic. Once a Test Message has been sent, it is immediately disposed of, or stored for future reference.
Channel PurgerRemove all unwanted messages from a message channel.
Manual ResubmissionFailed messages may often need to be manually submitted to a message channel. This EIP allows a message to be manually submitted to a message channel.
Failover ManagementThis EIP also known as ‘Endpoint Error Handling’ is useful for managing integration high availability.
Health EndpointMonitor the health of an application endpoint.
Distributed TracingTrack requests as they flow from Sender to Receiver.

Message Endpoint Patterns

TypeDefinitionQuality Attribute
Application AdapterThe Application Adapter pattern takes two incompatible interfaces and creates a bridge between the two. This pattern is very useful when converting from one message format to another (XML to JSON).
Message GatewayThe Messaging Gateway EIP encapsulates message-specific code from the rest of the application. It is a class that wraps messaging-specific method calls and exposes domain-specific methods to the application. Only the Messaging Gateway knows about the actual implementation of the messaging system.
Message MapperThe Messaging Mapper pattern moves data between business domain objects and the underlying messaging infrastructure while keeping the two components independent of each other. It contains the mapping logic between the messaging infrastructure and the domain objects. Neither the objects nor the infrastructure has knowledge of the Messaging Mapper’s existence.
Transactional ClientThe Transactional Client EIP controls transactions with the messaging system. It makes the client’s session with the messaging system transactional so that the client can specify transaction boundaries.

This is considered an anti-pattern.
Anti-Pattern
Polling ConsumerThe Polling Consumer EIP allows a Receiver to explicitly make calls to a Receiver when it wants to receive messages. Polling is carried out at intervals. This Pattern is often used in Async HTTP API calls.
Event-Driven ConsumerProcess event messages based on a Sender emitting an event without the Consumer constantly polling. Event-driven EIPs notify a Consumer when a state change has occurred.
Competing ConsumerThis EIP enables multiple concurrent consumers to process messages received on the same messaging channel. With multiple concurrent consumers, a system can process multiple messages concurrently to optimize throughput, improve scalability and availability, and balance the workload.
Throttled ConsumerThe Throttled Consumer EIP ensures that the Receiver is only allowed to process a set of messages at any given time. This EIP is useful in scenarios where the Receiver destination endpoint may not be ready to commit a transaction. By slowing the Receiver endpoint using other EIPs like Exponential Backoff, the Receiver endpoint is given a chance to process a message.
Singleton ConsumerIn a Publisher & Subscriber model, a Singleton Consumer pattern ensures that only one consumer handles messages in sequence. Some consumer Receivers can process multiple messages in parallel, this design pattern aims to ensure that a single consumer is created and responsible for handling messages.
Selective ConsumerSelect a specific consumer in both EIP Point-to-point Messaging and Publisher/Subscriber by using message headers such as the TO, FROM, and CorrelationID message attributes.
Message DispatcherIn a Point-to-Point Messaging system, that uses a single message channel, use a Message Dispatcher service to distribute messages to the appropriate Receiver.
Durable SubscriberA durable subscription saves messages for an inactive subscriber and delivers these saved messages when the subscriber reconnects. Azure Service Bus works on the Pull model, messages are left in the Queue until the Receiver/Subscriber explicitly moves and removes the message from the Queue.
Idempotent ReceiverThis EIP ensures that a message consumed by a Receiver/Subscriber always produces the same result. This EIP works in parity with message settlement and delivery models.
Message DeduplicationSome Sender Endpoints can manage duplicate messages being sent to a message channel. This reduces the complexity of the Receiver/Subscriber endpoint when implementing Idempotent Receivers.
Stale Message/
Message Expiration
The Message Expiration EIP allows a sender to indicate when a message should be considered stale and shouldn’t be processed. You can set the Message Expiration to specify a time limit in which a message is viable. Message Expiration is built into some Messaging systems; these messages are often dead-lettered.
Service WindowA service window in integration is a specific time period when a system or service is available for integration. It is usually used to refer to a period of time during which a system or service is open and available for integration with other systems. This allows for a more efficient and reliable integration process since developers can plan their integration efforts around known service windows.
Service ActivatorA Service Activator is a method used to create, configure, and manage services in a Service Oriented Architecture (SOA). It is a component that integrates with service containers that are responsible for the creation and management of services. The Service Activator listens for requests and then creates or configures the appropriate service to fulfill the request. It can also manage the lifecycle of the services, such as starting and stopping them.
Priority QueueProcess messages based on priority queues. A Queue with a higher priority will process messages faster than a queue with a lower priority.
Queue-Based Load LevellingUse a Queue to invoke a sequential message system to manage load and costs by restricting a Consumer to processing a single message at a time.
Retry MessageA Sender and Receiver may fail to send/receive a message. Retry patterns help to solve this problem without bottlenecking the overall system.
Election LeaderA distributed system that picks the worker node based on the Bully/Ring algorithm.

Message Construction Patterns

TypeDefinitionDomain
Document MessageUse a Message Construction pattern to create a JSON or XML message to a message channel.
Event MessageSend a Key/Value event message to a message channel.
Command MessageSend a Receiver an asynchronous message to instruct it to do something. This EIP is useful in Point-to-Point messaging systems.
Message Model/
Message Schema
Message modelling is a way of predefining the message formats that are used by your applications. There are various Message Models in places such as CloudEvents, AsyncAPI, or proprietary message models such as Drift.
Request-ResponseReceive a response from a recipient after sending a message. Request-Response works well with the AMQP protocol.
Request/ReplySend a message to one Queue, and receive a reply in another Queue. This is a different pattern from the common request & response pattern. Request & Reply works well with asynchronous architectures.
Return AddressAdd Sender details to a message so that Receivers can identify and send responses back to the Sender if required.
Correlation IdentifierAdd a unique identifier that associates any message to the system or endpoint that can help to identify why the message was constructed in the first place.
Message SequenceAdd a numerical value to the message so that messages can be resequenced.
Message ExpirationAdd an expiry date on a message so that messages are automatically disposed of after a given time period.
Message ActivationMessage activation is the process of activating a message by triggering some kind of action. This could be a customer opening an email, clicking on a link or downloading an attachment, or any other form of engagement that shows the customer is interested in what the message has to offer. By activating a message, the sender can measure and track the effectiveness of the message, which helps inform future marketing efforts.
Format IndicatorAdd details indicating the format of the messages so that Receivers can quickly identify and process messages.
Message SchemaCreate and transform messages from a Schema registry to ensure that a Sender & Receiver conform to a message standard.

Message Channel Patterns

TypeDefinitionDomain
Point-to-pointIntegrate a Sender & Receiver directly without using a message channel. This EIP is also known as Point-to-Point Messaging. In Point-to-Point messaging, the source and targets are known as Sender & Receiver.
Publish-SubscribeProcess messages asynchronously using a message channel where Receivers consume messages when they are ready.
Pull-to-PushTransform a Pull Channel into a Push Channel.
Push-to-PullTransform a Push Channel into a Pull Channel.
DataType ChannelDataType Channels in integration are used to specify how the data is going to be transformed in an integration flow. They provide the ability to define the schema and the mapping of the data that is to be exchanged between two systems. This includes the data type, length, and format of the data. DataType Channels are used to ensure that the data is in the proper format and to ensure data integrity.
Invalid MessageRemove messages that contain invalid Headers/Parameters or Payloads to a separate queue to process these messages independently.
Dead Letter ChannelRemove messages that cannot be delivered from a queue into a ‘Dead-Letter’ queue
Guaranteed DeliveryEnables the Sender to make sure that a message will be delivered, even if the Sender or Receiver application fails.
Message SettlementEnables a Receiver to send a response ACK to the Message Broker advising the message has been completed, abandoned, deferred or Deadlettered.
Fire & ForgetSend messages to a Receiver without expecting an acknowledgment from the Receiver.
Circuit BreakerStop data flow between systems based on implicit rules.
Channel AdapterConnect platform-agnostic systems to a single channel so that the systems can consume messages from a single channel.
Message BusA message broker is a communication channel that operates a push/pull model messaging system. However, there are other message channels such as event streaming architectures that handle streams of data. When lightweight, key/value pair messaging is required, the Message Bus is a very good option.
Message BridgeConsume messages from one queue, and send them to another queue to be processed independently.
Outbox DeliveryA service that uses a relational database inserts messages or events into an outbox table (e.g. OUTBOX ) as part of the local transaction.
Store and ForwardSimilar to the Outbox, this EIP is designed to handle messages where the destination is not reachable. Messages are stored in a temporary datastore and forwarded to the Receiver when the Receiver is online.
Delta SyncTrack database changes in one system and apply these changes to another database.

Message Transformation Patterns

TypeDefinitionDomain
Message TranslatorTranslate one message format into another.
Claim CheckCall an external service that contains a larger payload than the message itself can carry.
NormalizerA message may arrive in a channel in multiple formats, XML, JSON, and AVRO. Use a Normalizer EIP to transform the ingress messages to a standard format for egress.
Canonical Data ModelA Canonical Data Model (CDM) is a data model that defines a common set of data elements, the relationships between them, and the rules governing the data, to be used as a reference for data exchange between different systems. It is designed to provide a consistent view of data across multiple applications, databases, and organizations. The CDM is often used to improve data integration and consistency, reduce data duplication, and create a more efficient data exchange process.
Service CallCall an external service to transform data held in the message.

Message Routing Patterns

TypeDefinitionDomain
Pipes-and-FiltersProcess messages through complex series of conditions and steps.
Message RouterRoute messages based on complex business rules.
Content-Based RouterRoute to a receiver based on the message header content. This design pattern works well with the AMQP protocol using the TO attribute.
Message FilterFilter messages based on rules so that Receivers only consume and process messages intended for them.
Dynamic RouterDynamically route to a destination without explicitly adding the Recipients details to a message.
Recipient ListRoute Messages to a Recipient either by using a static or dynamic Recipient list.
SplitterSplit a message into multiple channels so that they can be processed differently.
AggregatorFan-in individual messages to process them as a single collection.
ResequencerReorder a sequence of messages that have been received out of order.
Composed Msg. ProcessorA Composed Message Processor is an integration pattern used to process a message composed of multiple parts. It is used to process a message that has multiple elements and needs to be handled in a certain order. The Composed Message Processor can be used to convert an incoming message into an intermediate format that can be used by downstream components or to organize and validate the content of a message.
Scatter-GatherA scatter-gather is an integration pattern used in message-driven architectures to send multiple requests to different services in parallel, and then gather the responses into a single response. This approach can reduce the latency of a composite service while increasing its throughput.
Routing SlipRoute messages through a list of processing steps without predetermining the routing.
Message DelayerDelay sending a message for a given period.
Circuit BreakerStop sending messages between service endpoints if a Recipient is offline.
SagaDefine a series of related actions in a router that should be either completed successfully (all of them) or not-executed/compensated.
Service CallCall a remote service in a distributed system where the service is looked up from a service registry.
MulticastRoute single or multiple messages to multiple systems at the same time.
LoopRepeat processing the same message in a loop.
ThreadsDe-couple the routing from the current thread to a new thread to improve routing performance.
Message SamplingSample messages at intervals to a dedicated queue to ensure the overall integration is working effectively.
AbortForce message/s being delivered to a destination
Service CallCall a remote service to validate a schema or a destination address.

Comments (4)

Tony

Feb 14, 2022 at 9:36 AM

One item that’s missing is the Bi-Directional Sync. Thanks for sharing this list.

Reply

syed hussain

Jul 06, 2022 at 6:37 AM

Thank you for your comment. Integration is usually documented as a one-way stream, of course, it can be reversed so that the Receiver is also responding to a Sender making this bi-directional. If you are speaking of two-way communication (duplex) then this is a different communication protocol altogether, for this I have Azure Pub/Sub and Relays documented, but also posts on both Web Sockets and SignalR, both provide duplex capabilities.

Reply

Anton

Jun 06, 2022 at 9:32 AM

I know this page is work in progress, but where would you classify something like X12 and EDIFACT?

Reply

syed hussain

Jul 06, 2022 at 6:33 AM

I have some commentary on the Application Adapter pattern, although not explicit to EDI, it should provide some guidance.

Reply

Leave a comment