Enterprise Integration Design Patterns Catalogue with Azure, AWS & GCP
Post by: syed hussain in All Integration Design Patterns
Summary
The section below documents some common Enterprise Integration Patterns (EIP) that can be used across different vendor platforms. The EIPs are considered industry best practices, they have been peer-reviewed by domain experts. For a detailed understanding of the core components visit the Well-defined Integration Architecture post.
*Note, that this catalogue is a work in progress.
EIP Management Systems
| Type | Definition | Domain | Technologies |
|---|---|---|---|
| Alerts & Notification | Alerts and Notifications design patterns work with the Control Bus and Retry patterns. When failures occur in an integrated system, the Control Bus issues notifications to Administrators. This helps in quickly addressing issues and maintaining system reliability. | Monitoring | AWS CloudWatch, Azure Monitor, Splunk |
| Audit Trail | Tracks and records every action and decision made by the system, ensuring that a complete history is available for auditing purposes. This is crucial for security, compliance, and troubleshooting. | Security, Traceability | Elasticsearch, Logstash, Kibana (ELK Stack), Splunk |
| Channel Purger | Removes all unwanted messages from a message channel to maintain cleanliness and ensure only relevant messages are processed. This helps in avoiding clutter and potential processing delays. | Clean-up | Apache Kafka, RabbitMQ |
| Control Bus (CB) | Manages multiple distributed integrations through a single backplane. The CB uses a separate message channel to communicate with endpoints to ensure that all integration components are working. It centralizes control and monitoring, making it easier to manage complex systems. | Monitoring | MuleSoft, Apache Camel, WSO2 |
| Detour | Allows a management system to send messages through a separate message channel based on conditions. This pattern is useful for re-routing messages from the main channel when certain conditions are met, enabling dynamic and flexible message handling. | Flexibility | Apache Camel, WSO2, Spring Integration |
| Distributed Tracing | Tracks requests as they flow from Sender to Receiver, providing visibility into the end-to-end journey of a message. This is essential for debugging, performance monitoring, and ensuring that messages are handled correctly throughout their lifecycle. | Traceability | Jaeger, Zipkin, AWS X-Ray |
| Failover Management | Manages integration high availability by handling endpoint errors. This includes mechanisms for detecting failures and switching to backup systems or endpoints to ensure continuous operation. | High Availability | Kubernetes, AWS Elastic Load Balancing, Azure Traffic Manager |
| Governance Dashboard | Provides a central interface for monitoring, managing, and auditing integration activities and compliance. This ensures that all integrations comply with organizational policies and regulatory requirements. | Monitoring, Compliance | MuleSoft Anypoint, WSO2 Governance Registry |
| Health Endpoint | Monitors the health of an application endpoint, providing real-time status updates and alerts if the endpoint becomes unavailable or unhealthy. This helps in maintaining the reliability and availability of integrations. | Health Monitoring | Spring Boot Actuator, AWS Health Check, Azure Application Insights |
| Manual Resubmission | Allows a message to be manually submitted to a message channel when it fails to be processed automatically. This is useful in scenarios where automated retry mechanisms are insufficient, and human intervention is required. | Reliability | RabbitMQ Management Plugin, Apache Kafka Manager |
| Message History | Ensures that messages are persisted in storage, commonly used in Event-driven and Event-Sourcing architectures. This allows for tracking, auditing, and replaying messages if necessary. | Monitoring | Apache Kafka, AWS Kinesis, Azure Event Hubs |
| Message Store | Stores a Command, Event, or Query message in persistent storage for auditing purposes. This ensures that all important messages are archived and can be retrieved for future reference or analysis. | Auditing | Amazon S3, Azure Blob Storage, Google Cloud Storage |
| Scheduler | Triggers actions or sends messages based on a pre-defined schedule. This is useful for tasks that need to be performed at regular intervals or specific times. | Timing, Automation | Apache Quartz, AWS CloudWatch Events, Azure Scheduler |
| Smart Proxy | Intercepts messages sent by a Sender on a messaging channel. The Proxy then replaces the Return Address with its own address or another address the Proxy is listening on. When the Receiver responds to the message by sending the response message to the Return Address, the Proxy can then act on behalf of the Sender. This is useful for message routing and handling responses in complex integration scenarios. | Message Routing | NGINX, HAProxy, Envoy |
| Test Message | Monitors the health and stability of any enterprise integration. Sends messages from a Sender to a Receiver without invoking any business logic. Once sent, it is immediately disposed of, or stored for future reference. This ensures that the integration components are functioning correctly without affecting the business process. | Health Monitoring | Postman, SoapUI, Apache JMeter |
Message Endpoint Patterns
| Type | Definition | Domain | Technologies |
|---|---|---|---|
| Application Adapter | The Application Adapter pattern takes two incompatible interfaces and creates a bridge between the two. This pattern is very useful when converting from one message format to another (e.g., XML to JSON), ensuring seamless communication between different systems. | Interoperability | Apache Camel, MuleSoft, Talend |
| Content Enricher | Enriches the message content by adding additional data obtained from an external source. This ensures that messages have all the necessary information before being processed further. | Completeness | Apache Camel, Spring Integration |
| Content Filter | Filters out unnecessary data from a message before it is processed or forwarded. This helps in reducing the message size and ensuring only relevant data is passed along. | Efficiency | Apache Camel, MuleSoft |
| Durable Subscriber | Saves messages for an inactive subscriber and delivers these saved messages when the subscriber reconnects. This ensures that no messages are lost when a subscriber is temporarily unavailable. | Reliability | Apache Kafka, AWS SNS, Azure Service Bus |
| Event-Driven Consumer | Processes event messages based on a Sender emitting an event without the Consumer constantly polling. Event-driven EIPs notify a Consumer when a state change has occurred, improving responsiveness and reducing resource usage. | Responsiveness | Apache Kafka, AWS Lambda, Azure Event Grid |
| Idempotent Receiver | Ensures that a message consumed by a Receiver/Subscriber always produces the same result. This is crucial for preventing duplicate processing and ensuring consistency in message handling. | Consistency | Apache Kafka, RabbitMQ |
| Manual Resubmission | Allows a message to be manually submitted to a message channel when it fails to be processed automatically. This is useful in scenarios where automated retry mechanisms are insufficient, and human intervention is required. | Reliability | RabbitMQ Management Plugin, Apache Kafka Manager |
| Message Deduplication | Manages duplicate messages being sent to a message channel, reducing the complexity of the Receiver/Subscriber endpoint when implementing Idempotent Receivers. | Efficiency | Apache Kafka, RabbitMQ |
| Message Dispatcher | Distributes messages to the appropriate Receiver in a Point-to-Point Messaging system that uses a single message channel. This ensures that messages are routed correctly based on specific criteria. | Routing | Apache Camel, MuleSoft, Spring Integration |
| Message Gateway | Encapsulates message-specific code from the rest of the application. It is a class that wraps messaging-specific method calls and exposes domain-specific methods to the application. Only the Messaging Gateway knows about the actual implementation of the messaging system, promoting encapsulation and separation of concerns. | Encapsulation | Apache Camel, Spring Integration |
| Message Mapper | Moves data between business domain objects and the underlying messaging infrastructure while keeping the two components independent of each other. It contains the mapping logic between the messaging infrastructure and the domain objects. | Decoupling | Apache Camel, MuleSoft, Talend |
| Polling Consumer | Allows a Receiver to explicitly make calls to a Receiver when it wants to receive messages. Polling is carried out at intervals. This pattern is often used in Async HTTP API calls, providing flexibility in message consumption. | Flexibility | Apache Camel, Spring Integration |
| Priority Queue | Processes messages based on priority queues, where a queue with a higher priority will process messages faster than a queue with a lower priority. This ensures that critical messages are handled promptly. | Prioritisation | RabbitMQ, Apache ActiveMQ |
| Queue-Based Load Levelling | Uses a Queue to invoke a sequential message system to manage load and costs by restricting a Consumer to processing a single message at a time. This helps in managing system load effectively. | Load Management | Azure Queue Storage, AWS SQS |
| Selective Consumer / Competing Consumer | Selects a specific consumer in both EIP Point-to-point Messaging and Publisher/Subscriber by using message headers such as the TO, FROM, and CorrelationID message attributes. This allows precise targeting of messages to the appropriate consumer. | Precision | Apache Kafka, RabbitMQ |
| Service Activator | Creates, configures, and manages services in a Service-Oriented Architecture (SOA). The Service Activator listens for requests and then creates or configures the appropriate service to fulfill the request, managing the lifecycle of the services, such as starting and stopping them. | Service Management | Spring Boot, Apache Camel, MuleSoft |
| Service Window | Defines a specific time period when a system or service is available for integration, allowing for a more efficient and reliable integration process. | Availability | AWS Lambda Scheduled Events, Azure Functions Timer Trigger |
| Singleton Consumer | Ensures that only one consumer handles messages in sequence in a Publisher & Subscriber model. This design pattern aims to ensure that a single consumer is created and responsible for handling messages, ensuring consistency in message processing. | Consistency | Apache Kafka, RabbitMQ |
| Smart Proxy | Intercepts messages sent by a Sender on a messaging channel. The Proxy then replaces the Return Address with its own address or another address the Proxy is listening on. When the Receiver responds to the message by sending the response message to the Return Address, the Proxy can then act on behalf of the Sender. This is useful for message routing and handling responses in complex integration scenarios. | Message Routing | NGINX, HAProxy, Envoy |
| Stale Message/Message Expiration | Allows a sender to indicate when a message should be considered stale and shouldn’t be processed. You can set the Message Expiration to specify a time limit in which a message is viable. Message Expiration is built into some Messaging systems; these messages are often dead-lettered. | Timeliness | RabbitMQ, Apache Kafka, Azure Service Bus |
| Test Message | Monitors the health and stability of any enterprise integration. Sends messages from a Sender to a Receiver without invoking any business logic. Once sent, it is immediately disposed of, or stored for future reference. This ensures that the integration components are functioning correctly without affecting the business process. | Health Monitoring | Postman, SoapUI, Apache JMeter |
| Throttled Consumer | Ensures that the Receiver is only allowed to process a set number of messages at any given time. This pattern is useful in scenarios where the Receiver destination endpoint may not be ready to commit a transaction. By slowing the Receiver endpoint using other EIPs like Exponential Backoff, the Receiver endpoint is given a chance to process a message. | Rate Limiting | Apache Camel, Spring Integration |
| Transactional Client | Controls transactions with the messaging system. It makes the client’s session with the messaging system transactional so that the client can specify transaction boundaries. This is considered an anti-pattern because it can lead to tight coupling and reduced flexibility. | Anti-Pattern | JMS (Java Message Service), Spring Transaction Management |
Message Construction Patterns
| Type | Definition | Domain | Technologies |
|---|---|---|---|
| Command Message | Sends a Receiver an asynchronous message to instruct it to perform a specific action. This pattern is useful in Point-to-Point messaging systems where commands need to be executed reliably. | Asynchronous Messaging | Apache Kafka, RabbitMQ, Azure Service Bus |
| Correlation Identifier | Adds a unique identifier that associates any message to the system or endpoint, helping to identify why the message was constructed in the first place. This is crucial for tracking and managing messages across systems. | Message Tracking | Apache Camel, MuleSoft, WSO2 |
| Document Message | Uses a Message Construction pattern to create a structured message (e.g., JSON or XML) to send to a message channel. This pattern is useful for exchanging complex data structures between systems. | Data Exchange | JSON, XML, Avro |
| Event Message | Sends a key-value event message to a message channel. This pattern is essential for event-driven architectures where events need to be propagated across different components or systems. | Event-Driven Architecture | Apache Kafka, AWS SNS, Azure Event Grid |
| Format Indicator | Adds details indicating the format of the messages so that Receivers can quickly identify and process messages. This ensures that the message format is correctly interpreted by the receiving system. | Data Formatting | MIME types, Content-Type headers |
| Message Activation | Activates a message by triggering an action, such as a customer opening an email, clicking a link, or downloading an attachment. By activating a message, the sender can measure and track the effectiveness of the message. | User Engagement | Email tracking systems, Marketing automation tools |
| Message Compensator | is a pattern used to undo or reverse the effects of previously executed actions or transactions in a messaging system, typically as part of a saga or distributed transaction. | Event Rollback | ACID Transactions |
| Message Expiration | Adds an expiry date on a message so that messages are automatically disposed of after a given time period. This ensures that outdated messages do not clog the system. | Message Lifecycle Management | RabbitMQ, Apache Kafka |
| Message Model/Message Schema | Predefines the message formats that are used by applications. This ensures that all messages conform to a specified standard, facilitating interoperability and reducing errors. | Data Standardization | JSON Schema, Avro Schema, Protocol Buffers |
| Message Sequence | Adds a numerical value to the message so that messages can be resequenced if they arrive out of order. This is important for maintaining the correct order of processing in systems where message order is critical. | Ordered Processing | Apache Kafka, RabbitMQ |
| Notification Message | Sends a brief, informative message to notify a system or user of an event. This pattern is useful for alerting and informing stakeholders about significant events or changes. | Event Notification | AWS SNS, Twilio, Azure Notification Hubs |
| Request-Reply | Receives a response from a recipient after sending a message. This pattern works well with protocols like AMQP and is useful for synchronous interactions. | Synchronous Messaging | Apache Camel, Spring Integration, RabbitMQ |
| Request / Response | Sends a message to one Queue and receives a reply in another Queue. This pattern is useful in asynchronous architectures where decoupled interactions are required. | Asynchronous Messaging | Apache Kafka, RabbitMQ, Azure Service Bus |
| Return Address | Adds Sender details to a message so that Receivers can identify and send responses back to the Sender if required. This pattern is essential for enabling two-way communication in message-driven systems. | Message Routing | JMS, Apache Camel, MuleSoft |
Message Channel Patterns
| Type | Definition | Domain | Technologies |
|---|---|---|---|
| Broadcast Channel | A message channel that sends the same message to multiple receivers simultaneously. This pattern is useful for scenarios where the same information needs to be disseminated to many endpoints. | Broadcast Messaging | Apache Kafka, AWS SNS, Azure Event Grid |
| Channel Adapter | Connects platform-agnostic systems to a single channel so that the systems can consume messages from a single channel. This pattern is useful for integrating heterogeneous systems. | Integration | Apache Camel, Spring Integration, MuleSoft |
| Channel Pool | Manages a pool of message channels to optimize resource usage and message throughput. This helps in balancing the load and improving the efficiency of message processing. | Resource Management | RabbitMQ, ActiveMQ, Apache Kafka |
| Circuit Breaker | Stops the flow of messages between systems based on predefined rules, typically to prevent system overload or failure propagation. This pattern helps in maintaining system stability and reliability. | Fault Tolerance | Hystrix, Resilience4j, Envoy |
| DataType Channel | Specifies how the data is going to be transformed in an integration flow, including the schema and the mapping of data elements. This ensures data integrity and consistency. | Data Transformation | Apache Kafka, Apache Camel |
| Dead Letter Channel | Removes messages that cannot be delivered from a queue and places them in a ‘Dead-Letter’ queue for further investigation. This helps in managing undeliverable messages and maintaining system health. | Error Handling | Amazon SQS, RabbitMQ, Azure Service Bus |
| Fire & Forget | Sends messages to a Receiver without expecting an acknowledgment from the Receiver. This pattern is useful for scenarios where reliability is not a primary concern. | Asynchronous Messaging | RabbitMQ, Apache Kafka, AWS SQS |
| Guaranteed Delivery | Ensures that a message will be delivered, even if the Sender or Receiver application fails. This pattern is crucial for systems that require high reliability and message integrity. | Reliability | Apache Kafka, Amazon SQS, Azure Service Bus |
| Invalid Message | Removes messages that contain invalid headers, parameters, or payloads to a separate queue for independent processing. This helps in maintaining the integrity of the main message processing channel. | Error Handling | RabbitMQ, ActiveMQ, Apache Kafka |
| Message Bridge | Consumes messages from one queue and sends them to another queue to be processed independently. This pattern is useful for decoupling message producers and consumers. | Decoupling | Apache Camel, Spring Integration, MuleSoft |
| Message Bus | Operates a push/pull model messaging system, handling streams of data for lightweight, key/value pair messaging. This pattern is useful for large-scale message distribution. | Messaging Infrastructure | Apache Kafka, RabbitMQ, NATS |
| Message Settlement | Enables a Receiver to send a response ACK to the Message Broker advising that the message has been completed, abandoned, deferred, or dead-lettered. This pattern helps in managing message lifecycle and delivery status. | Message Lifecycle Management | RabbitMQ, Azure Service Bus, AWS SQS |
| Outbox Delivery | Uses a relational database to insert messages or events into an outbox table as part of the local transaction, ensuring reliable message delivery even in the case of system failures. | Reliability | Apache Kafka, Debezium, Spring Boot |
| Point-to-Point | Integrates a Sender and Receiver directly without using a message channel, known as Point-to-Point Messaging. This pattern is useful for direct and simple integrations. | Direct Messaging | JMS, Apache Kafka, RabbitMQ |
| Publish-Subscribe | Processes messages asynchronously using a message channel where receivers consume messages when they are ready. This pattern is useful for decoupling message producers and consumers. | Asynchronous Messaging | Apache Kafka, AWS SNS, Azure Event Grid |
| Pull-to-Push | Transforms a Pull Channel into a Push Channel, enabling proactive message delivery to receivers. This pattern is useful for systems that need to react to events promptly. | Messaging Transformation | Apache Camel, Spring Integration, MuleSoft |
| Push-to-Pull | Transforms a Push Channel into a Pull Channel, allowing receivers to pull messages when they are ready to process them. This pattern is useful for load management and controlling message consumption rates. | Messaging Transformation | Apache Camel, Spring Integration, MuleSoft |
| Service Window | Defines a specific time period when a system or service is available for integration, allowing for efficient and reliable integration processes. | Availability | Custom Implementation |
| Store and Forward | Handles messages where the destination is not reachable by storing them in a temporary datastore and forwarding them when the receiver is online. This pattern ensures reliable message delivery even in the case of temporary unavailability. | Reliability | Apache Kafka, RabbitMQ, ActiveMQ |
| Delta Sync | Tracks database changes in one system and applies these changes to another database, ensuring data consistency across systems. | Data Synchronization | Debezium, Apache Kafka, AWS DMS |
Message Transformation Patterns
| Type | Definition | Domain | Technologies |
|---|---|---|---|
| Canonical Data Model | Defines a common set of data elements, relationships, and rules to be used as a reference for data exchange between different systems. It provides a consistent view of data across multiple applications, databases, and organizations, improving data integration and reducing duplication. | Data Standardization | MuleSoft, Apache Camel, IBM Integration Bus |
| Claim Check | Extracts the larger payload from the message and stores it in an external service, leaving only a claim check in the message. This allows messages to remain lightweight while still being able to reference large data sets. | Data Offloading | Apache Camel, MuleSoft, Spring Integration |
| Content Enricher | Adds additional data to a message from an external source, enriching the message content. This is useful when the initial message does not contain all the required information for processing. | Data Enrichment | Apache Camel, MuleSoft, Spring Integration |
| Content Filter | Filters out unnecessary data from a message before it is processed or forwarded. This helps in reducing the payload size and ensuring that only relevant data is processed. | Data Filtering | Apache Camel, MuleSoft, Spring Integration |
| Format Transformer | Transforms the message format to meet the requirements of different systems. This is essential for interoperability between systems that use different data formats. | Data Transformation | Apache Camel, MuleSoft, Spring Integration |
| Message Adapter | Adapts a message from one format to another to ensure compatibility between systems. This is useful for integrating heterogeneous systems that use different protocols or message formats. | Data Transformation | Apache Camel, MuleSoft, Spring Integration |
| Message Compression | Compresses message data to reduce the size for transmission. This is useful for optimizing bandwidth usage and improving performance. | Data Efficiency | Gzip, Snappy, Zstandard |
| Message Decompression | Decompresses message data that was previously compressed, restoring it to its original size and format for processing. | Data Efficiency | Gzip, Snappy, Zstandard |
| Message Filter | Filters messages based on specified criteria, ensuring that only messages meeting the criteria are processed. This is useful for routing and handling messages selectively. | Data Filtering | Apache Camel, MuleSoft, Spring Integration |
| Message Normalizer | Transforms incoming messages to a standard format, ensuring consistency in how messages are processed. This is useful when messages arrive in multiple formats, such as XML, JSON, and AVRO. | Data Standardization | Apache Camel, MuleSoft, Spring Integration |
| Message Translator | Translates one message format into another to ensure compatibility between different systems. This is essential for integrating systems that use different data formats. | Data Transformation | Apache Camel, MuleSoft, Spring Integration |
| Normalizer | Ensures that messages arriving in different formats are converted to a single standard format for processing. This helps in maintaining consistency and reducing complexity. | Data Standardization | Apache Camel, MuleSoft, Spring Integration |
| Service Call | Calls an external service to transform data held in the message. This pattern is useful for offloading complex data transformation tasks to specialized services. | Data Transformation | REST APIs, SOAP Web Services, gRPC |
| Transformer | Transforms the content of a message according to specified rules. This is useful for converting data formats, enriching messages, or applying business logic. | Data Transformation | Apache Camel, MuleSoft, Spring Integration |
Message Routing Patterns
| Type | Definition | Domain | Technologies |
|---|---|---|---|
| Aggregator | Combines multiple related messages into a single message. This is useful for collecting and processing related data in a batch. | Message Aggregation | Apache Camel, MuleSoft, Spring Integration |
| Composed Message Processor | Processes a message composed of multiple parts, handling each part in a specific order. This is useful for complex message workflows that require multiple steps. | Message Processing | Apache Camel, MuleSoft, Spring Integration |
| Content-Based Router | Routes messages to different endpoints based on the content of the message. This pattern is useful for directing messages to the appropriate service based on their content. | Message Routing | Apache Camel, MuleSoft, Spring Integration |
| Dynamic Router | Dynamically routes messages to a destination without explicitly adding the recipients’ details to a message. This is useful for scenarios where the routing logic needs to be flexible and context-aware. | Dynamic Routing | Apache Camel, MuleSoft, Spring Integration |
| Message Filter | Filters messages based on specified rules so that receivers only consume and process messages intended for them. This helps in reducing processing overhead and focusing on relevant messages. | Data Filtering | Apache Camel, MuleSoft, Spring Integration |
| Message Router | Routes messages based on complex business rules. This pattern is useful for implementing sophisticated routing logic that considers various factors. | Message Routing | Apache Camel, MuleSoft, Spring Integration |
| Message Sampling | Samples messages at intervals to a dedicated queue to ensure the overall integration is working effectively. This is useful for monitoring and quality assurance. | Monitoring | Apache Camel, MuleSoft, Spring Integration |
| Multicast | Routes single or multiple messages to multiple systems at the same time. This pattern is useful for scenarios where the same message needs to be processed by multiple recipients. | Message Broadcasting | Apache Camel, MuleSoft, Spring Integration |
| Pipes-and-Filters | Processes messages through a complex series of conditions and steps. This pattern is useful for breaking down message processing into discrete, reusable components. | Message Processing | Apache Camel, MuleSoft, Spring Integration |
| Recipient List | Routes messages to a list of recipients, either statically or dynamically determined. This pattern is useful for scenarios where messages need to be sent to multiple endpoints. | Message Routing | Apache Camel, MuleSoft, Spring Integration |
| Resequencer | Reorders a sequence of messages that have been received out of order. This is useful for maintaining the correct order of messages in scenarios where order is important. | Message Ordering | Apache Camel, MuleSoft, Spring Integration |
| Routing Slip | Routes messages through a list of processing steps without predetermining the routing. This pattern is useful for dynamic and flexible message workflows. | Dynamic Routing | Apache Camel, MuleSoft, Spring Integration |
| Saga | Defines a series of related actions that should be either completed successfully (all of them) or not executed/compensated. This is useful for managing complex, long-running business processes. | Transaction Management | Apache Camel, Spring Boot, Axon Framework |
| Scatter-Gather | Sends multiple requests to different services in parallel and then gathers the responses into a single response. This approach can reduce the latency of a composite service while increasing its throughput. | Parallel Processing | Apache Camel, MuleSoft, Spring Integration |
| Selective Consumer | Selects a specific consumer in both EIP Point-to-Point Messaging and Publisher/Subscriber models by using message headers such as TO, FROM, and CorrelationID message attributes. | Message Filtering | Apache Camel, MuleSoft, Spring Integration |
| Service Call | Calls a remote service in a distributed system where the service is looked up from a service registry. This is useful for integrating with external services and APIs. | Service Integration | REST APIs, SOAP Web Services, gRPC |
| Splitter | Splits a message into multiple parts so that they can be processed individually. This pattern is useful for breaking down large messages into manageable pieces. | Message Processing | Apache Camel, MuleSoft, Spring Integration |
| Throttled Consumer | Ensures that the receiver is only allowed to process a set number of messages at any given time. This pattern is useful in scenarios where the receiver destination endpoint may not be ready to commit a transaction. | Rate Limiting | Apache Camel, MuleSoft, Spring Integration |
Security Patterns
| Type | Definition | Domain | Example Technologies |
|---|---|---|---|
| Authentication Gateway | Manages and verifies the identity of users or systems attempting to access a service. This ensures that only authenticated entities can interact with the system. | Identity Management | OAuth2, OpenID Connect, Keycloak, AWS Cognito |
| Authorization Pattern | Ensures that authenticated entities have the appropriate permissions to perform actions or access resources. This pattern helps in enforcing access control policies. | Access Control | RBAC (Role-Based Access Control), ABAC (Attribute-Based Access Control), AWS IAM, Azure AD |
| Data Masking | Obscures sensitive data within a message before it is transmitted or logged. This helps in protecting sensitive information from unauthorized access. | Data Privacy | Apache Kafka, DataMasker, IBM Guardium |
| Digital Signature | Uses cryptographic signatures to ensure the authenticity and integrity of a message. This helps in verifying that the message has not been tampered with and confirms the sender’s identity. | Data Integrity | RSA, DSA, ECDSA, OpenSSL, AWS KMS |
| Encrypted Communication | Encrypts data in transit to protect it from being intercepted or tampered with. This ensures that data remains confidential and secure while being transmitted between systems. | Data Security | TLS/SSL, HTTPS, VPN, AWS KMS, Azure Key Vault |
| Encrypted Storage | Encrypts data at rest to protect it from unauthorized access. This ensures that stored data remains confidential and secure. | Data Security | AES, RSA, AWS S3 Server-Side Encryption, Azure Blob Storage Encryption |
| Federated Identity | Allows users to use a single set of credentials to access multiple systems. This pattern helps in simplifying user management and enhancing security across integrated systems. | Identity Management | SAML, OAuth2, OpenID Connect, Microsoft ADFS |
| Firewall | Monitors and controls incoming and outgoing network traffic based on predetermined security rules. This helps in protecting systems from unauthorized access and attacks. | Network Security | Cisco ASA, Palo Alto Networks, AWS WAF, Azure Firewall |
| Intrusion Detection System (IDS) | Monitors network or system activities for malicious activities or policy violations and produces reports. This helps in detecting and responding to security threats. | Threat Detection | Snort, Suricata, AWS GuardDuty, Azure Security Center |
| Intrusion Prevention System (IPS) | Identifies and prevents potential security threats in real-time. This helps in blocking malicious activities before they can cause harm. | Threat Prevention | Cisco Firepower, Palo Alto Networks, AWS WAF, Azure Firewall |
| Key Management | Manages cryptographic keys, including their generation, distribution, storage, and destruction. This ensures that keys are handled securely and used appropriately. | Cryptographic Key Management | AWS KMS, Azure Key Vault, HashiCorp Vault |
| Message Integrity | Ensures that the content of a message has not been altered during transit. This helps in maintaining the integrity of the data being transmitted. | Data Integrity | HMAC, SHA-256, MD5, OpenSSL |
| Multi-Factor Authentication (MFA) | Requires more than one form of verification to authenticate a user. This enhances security by making it more difficult for unauthorized users to gain access. | Identity Management | Google Authenticator, Authy, AWS MFA, Azure MFA |
| Secure Session | Manages user sessions securely, ensuring that session data is protected from unauthorized access and tampering. | Session Management | JWT (JSON Web Tokens), OAuth2, Secure Cookies |
| Security Token Service (STS) | Issues security tokens that assert the identity and privileges of the user. This is useful for enabling Single Sign-On (SSO) and federated identity management. | Identity Management | AWS STS, Azure AD, Auth0 |
| SSL/TLS Offloading | Handles the encryption and decryption of SSL/TLS traffic to reduce the load on backend servers. This improves performance while maintaining secure communications. | Network Security | F5 BIG-IP, Citrix ADC, NGINX |
Data Architectures and Design Patterns
| Type | Definition | Domain | Example Technologies |
|---|---|---|---|
| CQRS (Command Query Responsibility Segregation) | Separates the read and write operations of a data store to optimize performance, scalability, and security. Commands handle updates, while queries handle data retrieval. | Data Management | Axon Framework, EventStore, Microsoft Azure Cosmos DB |
| Data Lake | A centralized repository that allows you to store all your structured and unstructured data at any scale. It enables flexible data processing and analytics. | Data Storage | AWS S3, Azure Data Lake, Google Cloud Storage |
| Data Mart | A subset of a data warehouse focused on a specific business line or team. It provides targeted insights and analytics capabilities. | Data Storage | AWS Redshift, Google BigQuery, Microsoft SQL Server |
| Data Mesh | A decentralized data architecture where different business domains manage their own data products. This approach promotes scalability, ownership, and agility. | Data Management | DataHub, Starburst, Dremio |
| Data Warehouse | A centralized repository for integrated data from multiple sources. It supports analytical reporting, structured queries, and data mining. | Data Storage | AWS Redshift, Google BigQuery, Snowflake, Microsoft SQL Server |
| Database Sharding | Divides a large database into smaller, more manageable pieces called shards. This approach improves performance and scalability by distributing the load across multiple servers. | Data Partitioning | MongoDB, Apache Cassandra, Amazon DynamoDB |
| Domain-Driven Design (DDD) | An approach to software development that emphasizes collaboration between technical and domain experts to create a shared understanding of the problem space and design solutions. | Software Architecture | Axon Framework, EventStore, Spring Boot |
| Event Sourcing | Stores the state of a system as a sequence of events. This pattern provides a reliable way to reconstruct past states and ensures that all changes are recorded. | Data Management | EventStore, Apache Kafka, Axon Framework |
| ETL (Extract, Transform, Load) | A data integration process that involves extracting data from different sources, transforming it into a suitable format, and loading it into a data warehouse or other repository. | Data Integration | Apache Nifi, Talend, AWS Glue |
| Kappa Architecture | An architecture designed for processing data streams. It simplifies the Lambda Architecture by processing both real-time and historical data using a stream processing engine, reducing complexity. | Data Processing | Apache Kafka, Apache Flink, Apache Samza |
| Lambda Architecture | Combines batch and real-time processing to provide a comprehensive data processing solution. This architecture supports both high throughput and low latency. | Data Processing | Apache Spark, Apache Kafka, AWS Lambda |
| Master Data Management (MDM) | Ensures the consistency, accuracy, and accountability of an organization’s shared master data assets. This involves processes and tools for data governance and quality management. | Data Governance | Informatica MDM, IBM InfoSphere, Talend |
| Polyglot Persistence | Uses multiple data storage technologies to handle different data storage needs within the same application. This approach leverages the strengths of various databases to optimize performance and scalability. | Data Management | MongoDB, Cassandra, Neo4j, PostgreSQL |
| Shared Nothing Architecture | Each node is independent and self-sufficient, avoiding shared resources that can become bottlenecks. This architecture is designed for high scalability and reliability. | System Design | Apache Cassandra, Amazon DynamoDB, Google Cloud Spanner |
| Stream Processing | Processes data in real-time as it is produced or received. This architecture is ideal for applications that require immediate insights or actions based on streaming data. | Real-time Data Processing | Apache Kafka, Apache Flink, Amazon Kinesis |
| Time-Series Database | Optimized for storing and querying time-stamped data. This type of database is ideal for applications that involve time-series data such as monitoring, IoT, and financial data. | Data Storage | InfluxDB, TimescaleDB, Prometheus |
| Data Virtualization | Provides a unified data layer that integrates data from disparate sources in real-time or near-real-time, without physically moving the data. This allows for seamless data access and integration. | Data Integration | Denodo, Tibco Data Virtualization, IBM Cloud Pak for Data |