Event-Driven Enterprise Integration Pattern using Azure Functions and Stripe Payment Gateway

Post by: syed hussain in All Integration Design Patterns

Summary

In this post, I’ll demonstrate how to build an event-driven integration using Microsoft Azure Functions, Azure Service Bus Queues and Stripe. This integration demonstrates the event-driven architecture design pattern; an event (state change) occurs in one system (Stripe) which is then handled by Azure Functions.

Event-driven architectures are asynchronous communications between services; in that, an Event is produced by a Sender and consumed by a Recipient.

  1. Event Emitters (Senders or Agents) have the responsibility to detect, capture, and transfer events (state changes). Event emitters do not know that Event Receivers exist, the sole responsibility is to transfer an event change to an event channel.
  2. Event Sinks have the responsibility to consume messages. A sink does not need to be a target application either, a Sink might just have the responsibility to filter, transform and forward the event to another system or it might provide a self-contained reaction to such an event.
  3. Event channels are conduits in which events are transmitted from Event Emitters to event consumers (Event Sinks). The physical implementation of event channels might be based on components such as message-oriented middleware or point-to-point channels.

Event-driven Architectures

Event-driven architectures can often be confusing to people that have heard the term in different domains of computer science. You can have event-driven architectures as a low-level implementation in application design, integration design and as part of a large elaborate ecosystem like Azure or AWS where resources emit events that are ingested by other components. They are essentially the same thing just implemented differently.

Events are Facts that have occurred.

A very simple example where it’s possible to demonstrate an application managing events internally is when using the Vue.js framework. Vue.js the JavaScript reactive programming framework handles events or state changes within the framework itself. This is still considered an ‘event’ and part of larger event-based systems where components interact with each other based on events emitted.

Take this simple example of a button click event.

<button @click="$emit('someEvent')">click me</button>

When the button is clicked, it emits an event that is captured by a function in the program. The Button itself is a component and does not ‘care’ what happens to the event after it submits the event. The sole responsibility of the button is to transfer the event to an event channel.

In systems integration, the Emitter may detect a change within an application and then transfer this state over to an Event Channel ready for the Sink to consume the events.

Large ecosystems like Azure (Azure Event Grid) and AWS (EventBridge) have event-driven solutions that enable detecting changes in many of their resources. This is also considered an event-driven architecture.

Within the context of this post, I will be discussing event-driven architectures in software and integration architectures. In that Event-driven architectures will mean asynchronous communication between a Publisher (Event Emitter) and a Consumer (Event Sink).

Conceptual Architecture

This Enterprise Integration Pattern (EIP) will demonstrate an event-driven integration using Webhooks and Azure Service Bus Queues.

A Customer registers on Stripe and makes a payment for a product. A webhook built with Azure Functions is invoked after an action. The Azure Function pushes the event message to a Service Bus queue. An Event Processor, then Consumes the message and attempts to insert a record into Microsoft Dataverse.

Stripe provides two systems when it comes to tracking event changes; Webhooks and API to query the platform. As we want to build an event-driven architecture, we need to be notified of a state change immediately when it happens. For this reason, we’ll use the webhook instead of querying the API at regular intervals.

The concept for event-driven architecture remains the same despite the technologies used. I’m using Azure Functions here; however, this technology can easily be replaced by Amazon Lambda or Google Cloud Functions.

HTTP Webhooks

In an event-driven architecture, using webhooks is a good way to rapidly build out an event-driven architecture without needing to prepare the additional infrastructure. However, there are some drawbacks when it comes to using webhooks.

Webhooks are user-defined callbacks built on the HTTP architecture. When an event occurs in an application, if there are registered endpoints, the application will make a POST request with a JSON payload to the endpoint. Once the POST request has been completed, the Sender application no longer concerns itself with what the Receiver intends to do with the message.

HTTP is an asymmetric protocol that operates on a request and response architectural pattern. It is also state-less, which means that every request met with a response is independent. A server will never maintain transaction details of every session request. If sessions are not managed in a state, how can we be sure that the Receiver has received the event message? There is no practical way to do this without introducing other technologies and protocols. This is one of the known challenges with HTTP and webhooks.

Now as the name suggests, webhooks are actions that are ‘hooked’ to an event. When an event occurs, the endpoint registered in a webhook is invoked. Endpoints are HTTP Post requests in JSON as the message format. This is different to a Polling Consumer, where a client might continuously call the sender to check if there are any updates. In a webhooks architecture, the Sender pushes out a notification when an event occurs.

Polling Consumer

Webhooks

In the conceptual architecture, a webhook endpoint (Azure Functions HTTP Trigger) is registered in the Stripe Developer Portal. Stripe makes the registration process very simple – just log in to the Developer Portal and create a new webhook with multiple events.

There are some common pitfalls when it comes to using webhooks, these are:

Benefits

  1. Works over HTTP/HTTPS.
  2. Enables the de-coupling of systems (Sender & Receiver).
  3. Easy integration & consumption – just register a HTTP endpoint.
  4. Good to use when a Third-Party service has applied rate limits.

Disadvantages

  1. Endpoints need to be secured correctly to prevent illegitimate HTTP Posts requests. For this reason, the HMAC (hash-based message authentication code) signature is often included in the Header.
  2. Mutual TLS authentication is also often used to secure communication between Sender & Receiver.
  3. Webhooks rely on the sender tracking the delivery of a message. Some services track the delivery of a message others do not. Stripe attempts to ‘deliver’ the webhook event message up to three times.
  4. Missed notifications can also be an occurrence if the Sender does not have the infrastructure or the Sender application does not track notifications.
  5. Duplicate event notifications can also be quite common if the Sender does not have de-duplication logic.

Target Reference Architecture

The reference architecture for this project is as follows:

  1. Stripe (Event Emitter) will send an event when a new Customer record has been added.
  2. HTTPS connections will only be permitted in Azure Functions.
  3. CORS to be enabled in Azure Functions.
  4. IP to be restricted to Stripe only.
  5. Webhook endpoint registered.
  6. Azure Function to forward create event messages in Azure Service Bus.
  7. An Event Processor will be used to consume the message, attempting to create a record in Microsoft Dataverse.

Note that for this demo I will not be implementing a hash-based message authentication code (HMAC). HMAC should be used to secure webhook calls to the registered Azure HTTP endpoint to ensure that they are genuinely coming from Stripe. There are other tools available in Azure that can assist in authenticating a request; however, you should always implement HMAC if the opportunity arises.

Platform Setup

  1. Register on Stripe.
  2. Create an Azure Functions HTTP project.
  3. Implement necessary security configurations (enable HTTPS only, restrict IPs, enable CORS). A list of security configurations can be found here.

Stripe Payment Gateway and API Registration

Register on the Stripe website to obtain access to the Developer portal. This provides access to two keys, test mode and live mode. Use test mode keys for development only. The process documented on this page works for both test and production scenarios.

Azure Functions Project Setup

  • Create a new Azure Functions HTTP Trigger Project.
  • Ensure the Function is scoped to run at the Function scope.
  • Remove the GET HTTP method, only keeping the POST method.
Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req, ILogger log)

Stripe Webhook Setup

Log in to the Developer Portal, and register a new Webhook. For now, we’ll just select the Customer Created event. When you’re configuring the endpoint, copy and paste the Azure HTTP Function link with the token.

https://stripeapi31.azurewebsites.net/api/GetCustomerEvent?code=Wsa3IXMEETJnl7dV2MuEIVNMH6Jfhbfjewfr68ssks9a5dbfMpTPVWlpnYoO2g==

This is all that is required to set up the Stripe webhook.

C# Implementation

For demonstration purposes, I have created both the Stripe and Dataverse methods in a single class. The full project can be found in my Github repository.

This class does two things:

  1. When a new customer is created in Stripe, the webhook emits an event to the HTTP endpoint. This message is posted using the HTTP POST method. The Stripe message payload is a JSON object which is deserialized into a custom class object called Root.
  2. Once the JSON POST request has been deserialized, a connection is made to the Dataverse and the Customer record is created.
public static async Task<IActionResult> Run([HttpTrigger(AuthorizationLevel.Function, "post", Route = null)] HttpRequest req, ILogger log)
{
    log.LogInformation("C# HTTP trigger function processed a request.");

    // Read contents from Post stream
    var content = await new StreamReader(req.Body).ReadToEndAsync();

    // Deserialise the event message
    var msg = JsonConvert.DeserializeObject<Root>(content);

    // Connect to Power Platform
    try
    {
// Connect to Power Platform Source
ServiceClient service = DataverseConnection();

// Test connection
Entity contact = new Entity("contact");
{
    contact["firstname"] = msg.Data.Object.Name;
    contact["description"] = msg.ToString();

    service.Create(contact);
}
    }
    catch (Exception e)
    {
log.LogInformation(e.ToString());
    }

    return new OkObjectResult($"Event captured for: {msg.Data.Object.Name}");
}

Testing the webhook

Create a new Customer in Stripe, within seconds you should see a new Customer created in the Dataverse Environment. If you look in the Stripe Developer portal under the webhook you created, you should see a Sucess: 200 messages with a response.

Remote Debugging

Important: Ensure you can start and run and debug the Local Azure Functions project. As of 04/2022, there appears to be a bug in Visual Studio 2019 where the Symbols are not loaded correctly. If this occurs load the symbols manually.

In Azure functions go to Settings > Configurations > General Settings to turn on Debugging.

Download the Get Publish Profile, you will need this later for authentication.

In Visual Studio, you remote debug like any other service by attaching to the process.

Debug > Attach to Process.

Connection Type: Default

Connection Target: Your Azure Functions URL followed by a port number.

Ports for remote debugging are:

  • Visual Studio 2022: 4026
  • Visual Studio 2019: 4024
  • Visual Studio 2017: 4022
  • Visual Studio 2015: 4020
  • Visual Studio 2013: 4018
  • Visual Studio 2012: 4016

When prompted to authenticate, use the details from the Get Publish Profile:

userName="eax360stripeapi\$eax360stripeapi" userPWD="4uYrGubFvQxdMKpAMeNM4T71RgXfx0yMHdhnYhSxC7doh5sP5KEnaFGMWdzw"

Finally, connect to the w3wp.exe service.

Implementation Challenges

Webhooks rely on a response being sent back to the Sender. There was an issue with the code I constructed, the middleware (the logic in between the request and response) was failing, but the response was being sent back as a 200 OK. The failure was quite simply the fact that I was using environment variables and had forgotten to create them in the Configuration section in Azure Functions. It did get me thinking that the middleware logic does need to be thoroughly tested especially with a tool like Azure Functions where the remote debugging experience isn’t so great.

There is a multitude of options when it comes to securing the Azure Function endpoint. However, the real concern to some readers will be ensuring that messages are only coming from Stripe. Using HMAC is an option, however, in a production setting the API Management Gateway should always be used. APIM allows for tighter control of security and governance.

Tags: azure functions dataverse event-driven
16 Apr 2022
1