Priority Queue Pattern with Azure and Dynamics 365

Summary

This pattern explores the idea of prioritising message requests so that messages with a higher priority are processed differently to messages with a lower priority.

In traditional systems, messages are processed in a FIFO queue.

However, in a Priority Queue system, the messages are placed in different streams.

Both streams are handled independently.

*The architecture below describes the Priority Queue pattern with Microsoft Dynamics 365 CE, however this patterns can be applied for any application that can produce messages for consumption.

Systems Context

A Sales Manager first submits a request on a web platform to see the total number of sales created in the last year. A few moments later a Customer also submits another request to update their Customer account details.

In the above scenario, two processes have been created; one that returns the total sum of sales in the last year, and another process that updates a Customer record. If a single worker process is fulfilling both request, the request will be fulfilled in a FIFO order:

1. Process A: Return total number of sales in the last year.
2. Process B: Update Customer details.

In this scenario, Process B might be deemed high priority because the process updates a customer record. Process A, while still very important, may run for an several minutes before the data is aggregated and fed back to the Sales Manager. Process B will have to wait until Process A has completed before committing to the job. This scenario may be unacceptable in many business scenarios.

Architecture

In Microsoft Dynamics 365 CE, both custom assemblies and plugins work in a FIFO order, in the event execution pipeline. Long-running queries such as total sales roll-up should be performed outside of the platform (out-of-band).

For both scenarios we’ll consider the usage of the Service Bus Topics.

This architecture for the above scenario can be represented as:

1. In a Priority Queue pattern, Dynamics 365 creates several messages using an Azure-aware plugin and assigns a custom property called Priority in each message with a value set to either High or Low.
2. Dynamics 365 CE posts these messages to the defined Azure Service Bus Topic. The Topic has two associated subscriptions that both filter messages by examining the Priority property. One subscription accepts messages where the Priority property is set to High, and the other accepts messages where the Priority property is set to Low.
3. A pool of consumers reads messages from each subscription. The high priority subscription has consumers that can process large volumes of data such as to calculate the total sales volume. The low priority subscription has consumers like Azure Functions to execute the request to update a Customer record.

Service Bus Partition Architecture

While a queue is often used for point-to-point communication, Service Bus Topics are useful in publish/subscribe scenarios.

Each partitioned queue or topic consists of multiple partitions. Each partition is stored in a different messaging store and handled by a different message broker. When a message is sent to a partitioned queue or topic, Service Bus assigns the message to one of the partitions. The selection is done randomly by Service Bus or by using a partition key that the sender can specify.

When a client wants to receive a message from a partitioned queue, or from a subscription to a partitioned topic, Service Bus queries all partitions for messages, then returns the first message that is obtained from any of the messaging stores to the receiver. Service Bus caches the other messages and returns them when it receives additional receive requests. A receiving client is not aware of the partitioning; the client-facing behaviour of a partitioned queue or topic (for example, read, complete, defer, deadletter, prefetching) is identical to the behaviour of a regular entity.

The peek operation on a non-partitioned entity always returns the oldest message, but not on a partitioned entity. Instead, it returns the oldest message in one of the partitions whose message broker responded first. There is no guarantee that the returned message is the oldest one across all partitions.

Considerations

1. Define what constitutes a priority and which Service Bus Topic should receive High and Low priorities.
2. Decide the order in which High/Low priority processes output to a consumer application. For example, if you deem that a high priority process is a customer transaction from a web form entry, and low priority message sits in a batch process, the low-priority batch process may eventually override any value that high-priority process has written to a database.
3. When provisioning Service Bus Topics, note that Topics are not available in the Basic Plan.
4. There is a base cost of 0.01GBP per day for provisioning the namespace.

Resources

More information can be found here: Azure Service Bus.