Building an idempotent event-driven microservices

Building an idempotent event-driven microservices

Creating a microservice to use in the example

We will use a microservice based on the DevPrime platform

  • Use an active account on the DevPrime and activate the DevPrime CLI.
  • Start containers through Docker (MongoDB, RabbitMQ and Redis)
  • Create the orderevents and paymentevents queue in RabbitMQ.
  • Create a new microservice by following the example of “payment” presemtado.
  • This microservice will listen to the queue/topic “orderevents”

Starting Idempotency setup

Setting the idempotency in the “DevPrime_App” key
Change the Idempotency setting in the “Enabled” option to “true” and “Flow” to “frontend”. We changed the “key” to “my-idempotency-key”. With this setting the API will require the Idempotency key in Headar.
code .\src\app\appsettings.json

1
2
3
4
5
6
7
8
9
    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "auto"
    }

Include a second persistence in the “DevPrime_State” key
In this example we will use a local Redis with a default password
code .\src\app\appsettings.json

1
2
3
4
5
6
7
8
9
{
  "enable": "true",
  "alias": "State2",
  "dbtype": "redis",
  "connection": "127.0.0.1:6379,password=LltF8Nx*yo",
  "timeout": "5",
  "retry": "2",
  "durationofbreak": "45"
}

For this scenario we will add the event directly to RabbitMQ to simulate a duplicate of events.

Run the dp-stream microservice and enter the Rabbitmq portal at
http://localhost:15672 with the guest/guest user. Find the Queues menu > orderevents and find the item “Publish message”

Find the payload field and paste the json below and dpeois go to “Publish message” to queue this event.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
{
   "Headers":{
      "uber-trace-id":"126e838875d34dd3:5644a477603d0a55:bc20635b06c79bc2:1"
   },
   "Id":"36dbd9d0-7b8b-41fb-bdaf-2ff6a0672bd4",
   "CorrelationId":"18c71a48-0253-4247-9c8b-d576c4f8394e",
   "AppId":"21cef89c-229e-4291-8d6a-1ccf5c401484",
   "AppName":"ms-idempotency",
   "TenantID":"",
   "TenantUserID":"",
   "Version":1,
   "Name":"OrderCreated",
   "CreationDate":"2022-03-13T11:14:01.0284863-03:00",
   "Payload":{
      "ID":"ccfc8248-2ab5-4423-92fc-b16c4555add2",
      "CustomerName":"Bill",
      "CustomerTaxID":"string",
      "Total":100
   }
}

The microservice is listening to this queue and will already react to the event and successfully process.

DevPrime Capabilities Idempotency Stream

Now return to RabbitMQ and resubmit the same event and you will notice that upon reaching the microservice it is automatically rejected.

DevPrime Capabilities Idempotency Stream

To repeat the test change a value in the json Payload by placing another guid in the ID for example.

Starting manual Idempotency

DevPrime allows you to manually enter a specific method that will support idempotência. The first step is to change the main configuration and then go directly into the method to add the configuration.

** Changing the “Action” key to “Manual”**

1
2
3
4
5
6
7
8
9
    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "manual"
    }

In manual mode when performing a test by submitting duplicate events you will notice that they will be accepted. You need to visit each desired method and add “Dp.Idempotency.Enable = true;” to enable idempotency control.

** Modifying the Stream to have idempotência"**
Open the Event Hub and find the subscribe of the desired event that in this context is “OrderCreated” and add “Dp.Idempotency.Enable = true;”.
code src\Adapters\Stream\EventStream.cs

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
namespace DevPrime.Stream;
public class EventStream : EventStreamBase, IEventStream
{
    public override void StreamEvents()
    {
        Subscribe<IPaymentService>("Stream1", "OrderCreated", (payload, paymentService, Dp) =>
        {
            Dp.Idempotency.Enable = true;

            var dto = System.Text.Json.JsonSerializer.Deserialize<OrderCreatedEventDTO>(payload);
            var command = new Payment()
            {
                CustomerName = dto.CustomerName,
                OrderID = dto.OrderID,
                Value = dto.Value
            };
            paymentService.Add(command);
        });
    }
}

When you try to run new with duplicate events you will already notice that it has returned to work.

Idempotency in microservices and REST APIs
Building an idempotent event-driven microservices

Last modified March 13, 2022 (4cc8d5e)