Building an idempotent event-driven microservices

Creating a microservice to use in the example

We will use a microservice based on the Devprime platform

• Use an active account on Devprime and activate the Devprime CLI.
• Start the containers from docker (MongoDB, RabbitMQ and Redis)
• Create the orderevents and paymentevents queue in RabbitMQ.
• Create a new microservice following the example of “payment” presented.
• This microservice will listen to the “orderevents” queue/topic

Starting the idempotency configuration

Setting the idempotency in the “Devprime_App” key
Change the Idempotency configuration to “Enabled” to “true” and “Flow” to “frontend”. We changed the “key” to “my-idempotency-key”. With this configuration, the API will require the idempotency key in the Headar.
code .\src\app\appsettings.json

1
2
3
4
5
6
7
8
9

    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "auto"
    }

Include a second persistence in the “Devprime_State” key
In this example we will use a local Redis with a default password
code .\src\app\appsettings.json

1
2
3
4
5
6
7
8
9

{
  "enable": "true",
  "alias": "State2",
  "dbtype": "redis",
  "connection": "127.0.0.1:6379,password=LltF8Nx*yo",
  "timeout": "5",
  "retry": "2",
  "durationofbreak": "45"
}

For this scenario we will add the event directly to RabbitMQ to simulate a duplicate event.

Run the dp-stream microservice and sign in to the Rabbitmq portal at
http://localhost:15672 with the guest/guest user. Locate the Queues menu > orderevents and locate the “Publish message” item

Locate the payload field and paste the json below and then go to “Publish message” to queue that event.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

{
   "Headers":{
      "uber-trace-id":"126e838875d34dd3:5644a477603d0a55:bc20635b06c79bc2:1"
   },
   "Id":"36dbd9d0-7b8b-41fb-bdaf-2ff6a0672bd4",
   "CorrelationId":"18c71a48-0253-4247-9c8b-d576c4f8394e",
   "AppId":"21cef89c-229e-4291-8d6a-1ccf5c401484",
   "AppName":"ms-idempotency",
   "TenantID":"",
   "TenantUserID":"",
   "Version":1,
   "Name":"OrderCreated",
   "CreationDate":"2022-03-13T11:14:01.0284863-03:00",
   "Payload":{
      "ID":"ccfc8248-2ab5-4423-92fc-b16c4555add2",
      "CustomerName":"Bill",
      "CustomerTaxID":"string",
      "Total":100
   }
}

The microservice is listening to this queue and will already react to the event and process successfully.

Now return to RabbitMQ and send the same event again and you will notice that when it reaches the microservice it is automatically rejected.

To repeat the test, change a value in the json payload by putting another guid in the ID, for example.

Starting manual idempotency

Devprime allows you to manually enter a specific method that will support idempotency. The first step is to change the main setting and then go directly to the method to add the setting.

** Changing the “Action” key to “Manual” **

1
2
3
4
5
6
7
8
9

    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "manual"
    }

In manual mode, when performing a test by sending duplicate events, you will observe that they will be accepted. You need to visit each desired method and add “Dp.Idempotency.Enable = true;” to enable the idempotency control.

** Modifying the Stream to have idempotency**
Open the Event Hub and locate the subscribe of the desired event which in this context is the “OrderCreated” and add the “Dp.Idempotency.Enable = true;”.
code src\Adapters\Stream\EventStream.cs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20

namespace Devprime.Stream;
public class EventStream : EventStreamBase, IEventStream
{
    public override void StreamEvents()
    {
        Subscribe<IPaymentService>("Stream1", "OrderCreated", (payload, paymentService, Dp) =>
        {
            Dp.Idempotency.Enable = true;

            var dto = System.Text.Json.JsonSerializer.Deserialize<OrderCreatedEventDTO>(payload);
            var command = new Payment()
            {
                CustomerName = dto.CustomerName,
                OrderID = dto.OrderID,
                Value = dto.Value
            };
            paymentService.Add(command);
        });
    }
}

When you try to run again with duplicate events, you will already notice that it is working again.

More articles on idempotency

Idempotency in microsservices and REST APIs
Building an idempotent event-driven microservices