Devprime | Documentation – Idempotency

Examples: Implementing idempotency in microsservices

Mon, 01 Jan 0001 00:00:00 +0000

We’ll implement an example in backend mode where you don’t need to pass a key to consume the microservice API. The process automatically manages the idempotency and the API payload.

The first step to enabling idempotency is to modify the “Idempotency” setting in Idempotency in the “Devprime_App”. Later, if you haven’t done so, you need to add a second state for the persistence of the idempotency.

Parameters used in configuration

Alias indicates the State used in the persistence of Idempotency
Duration is used to set the time of the command duplicate control register.
Flow indicates type of Idempotency and can be set to “backend” or “frontend”
Key is used to set a key required in the head in the Flow frontend
Scope limits the performance of ideopontence and uses the “all|web|stream” parameters
Action defines whether ideopontence will be automatic or manual with the “auto|manual” parameters

Creating a microservice to use in the example

We will use a microservice based on the Devprime platform

Use an active account on Devprime and activate the Devprime CLI.
Start the containers from docker (MongoDB, RabbitMQ and Redis)
Create the orderevents queue in RabbitMQ.
Create a new microservice “order”.

Starting the idempotency configuration

Setting the idempotency in the “Devprime_App” key
Change the Idempotency setting to “Enabled” to “true”
code src/app/appsettings.json

    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "auto"
    }

Include a second persistence in the “Devprime_State” key
In this example we will use a local Redis with a default password
code .\src\app\appsettings.json

{
  "enable": "true",
  "alias": "State2",
  "dbtype": "redis",
  "connection": "127.0.0.1:6379,password=LltF8Nx*yo",
  "timeout": "5",
  "retry": "2",
  "durationofbreak": "45"
}

Starting automatic idempotency in the microservice

Now it’s time to test the setup. Before you start the microservice, make sure that you have activated the containers and created the queue in RabbitMQ.

Run the application and make a post via the Order API
In the example below we are using a standard json for the post.

{
  "customerName": "Bill",
  "customerTaxID": "string",
  "itens": [
    {
      "description": "short",
      "amount": 1,
      "sku": "XNY1AC",
      "price": 100
    }
  ],
  "total": 100
}

Observe in the Log the information of the idempotency
The idempotency process automatically came in to secure this HTTP request without the need for any information from the API consumer side.

Make a second identical post via the Order API
By observing this second scenario, it is already possible to see that idempotency protected the microservice by preventing a duplicate inclusion.

API has already received a return rejecting the duplicate POST

By analyzing the Log, we can follow the entire idempotency process

Initiating manual idempotency in the microservice

Devprime allows you to manually enter a specific method that will support idempotency. The first step is to change the main setting and then go directly to the method to add the setting.

** Changing the “Action” key to “Manual” **

    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "manual"
    }

When using the “Action” option as “manual” if you run the microservice again and make a duplicate post you will notice that it will not reject, because the automatic process is turned off.

** Enabling idempotency in a method **
Since the process is now manual, we will enable idempotency on each method to be protected. In the example below we demonstrate this possibility directly in the Web Adapter.
code src\Adapters\Web\Order.cs

Locate the “Order” API in the Web Adapter and make the modification as shown.

app.MapPost("/v1/order", async (HttpContext http, IOrderService Service, Devprime.Web.ViewModels.Order.Order command) => await Dp(http).Pipeline(() =>
        {
            Dp(http).Idempotency.Enable = true;
            Service.Add(command.ToApplication());
        }));

Run it again and you will notice that the idempotency feature is back to work, but only for POST in the “/v1/order” API.

Examples: Idempotency in microsservices and REST APIs

Mon, 01 Jan 0001 00:00:00 +0000

Applications based on the Devprime platform already have an automatic idempotency feature, protecting services from duplicate inclusions.

In this scenario we will use a mandatory key sent by the application that is using the API. To use the feature, simply change the settings in the Devprime-based microservice.

Creating a microservice to use in the example

We will use a microservice based on the Devprime platform

Use an active account on Devprime and activate the Devprime CLI.
Start the containers from docker (MongoDB, RabbitMQ and Redis)
Create the orderevents queue in RabbitMQ.
Create a new microservice “order”.

Starting the idempotency configuration

Setting the idempotency in the “Devprime_App” key
Change the Idempotency configuration to “Enabled” to “true” and “Flow” to “frontend”. We changed the “key” to “my-idempotency-key”. With this configuration, the API will require the idempotency key in the Headar.
code .\src\app\appsettings.json

    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "frontend",
      "key": "my-idempotency-key",
      "Scope": "all",
      "Action": "auto"
    }

Include a second persistence in the “Devprime_State” key
In this example we will use a local Redis with a default password
code .\src\app\appsettings.json

{
  "enable": "true",
  "alias": "State2",
  "dbtype": "redis",
  "connection": "127.0.0.1:6379,password=LltF8Nx*yo",
  "timeout": "5",
  "retry": "2",
  "durationofbreak": "45"
}

Starting automatic idempotency in the microservice

Now it’s time to test the setup. Before you start the microservice, make sure that you have activated the containers and created the queue in RabbitMQ.

Run the application and make a post via the Order API
We need to pass the idempotency key through Head in this scenario so you need to consume the API using curl, podtman, insomnia and many others.

Starting the microservice configured to use an idempotency key
`

Sample json code for use in the API. Modify it in your tests.

{
  "customerName": "Bill",
  "customerTaxID": "string",
  "itens": [
    {
      "description": "short",
      "amount": 1,
      "sku": "XNY1AC",
      "price": 100
    }
  ],
  "total": 100
}

In the first test we will not pass the idempotency key in POST as per example performed in Postman.

When looking at the API return, we will have a denial of the request due to the absence of the key, which in the configuration with the “Flow” of the type “Backend” is mandatory and must be passed through the Header.

Now let’s add our key “my-idempotency-key” and a guid value.

After confirming the post in the previous step, the Log already registers success in the idempotency process.

When trying to make a new Post with the same previous key, the duplicate command is already automatically denied.

In this article, you noted how easy it is to implement the idempotency feature in microservices development with the Devprime platform.

You have at your disposal additional settings such as “Action” that configured as “manual” requires you to activate in a specific method and/or “Scope” that can limit the performance on an Adapter such as “Web” or “Stream”

Examples: Building an idempotent event-driven microservices

Mon, 01 Jan 0001 00:00:00 +0000

Creating a microservice to use in the example

We will use a microservice based on the Devprime platform

Use an active account on Devprime and activate the Devprime CLI.
Start the containers from docker (MongoDB, RabbitMQ and Redis)
Create the orderevents and paymentevents queue in RabbitMQ.
Create a new microservice following the example of “payment” presented.
This microservice will listen to the “orderevents” queue/topic

Starting the idempotency configuration

    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "auto"
    }

Include a second persistence in the “Devprime_State” key
In this example we will use a local Redis with a default password
code .\src\app\appsettings.json

{
  "enable": "true",
  "alias": "State2",
  "dbtype": "redis",
  "connection": "127.0.0.1:6379,password=LltF8Nx*yo",
  "timeout": "5",
  "retry": "2",
  "durationofbreak": "45"
}

For this scenario we will add the event directly to RabbitMQ to simulate a duplicate event.

Run the dp-stream microservice and sign in to the Rabbitmq portal at
http://localhost:15672 with the guest/guest user. Locate the Queues menu > orderevents and locate the “Publish message” item

Locate the payload field and paste the json below and then go to “Publish message” to queue that event.

{
   "Headers":{
      "uber-trace-id":"126e838875d34dd3:5644a477603d0a55:bc20635b06c79bc2:1"
   },
   "Id":"36dbd9d0-7b8b-41fb-bdaf-2ff6a0672bd4",
   "CorrelationId":"18c71a48-0253-4247-9c8b-d576c4f8394e",
   "AppId":"21cef89c-229e-4291-8d6a-1ccf5c401484",
   "AppName":"ms-idempotency",
   "TenantID":"",
   "TenantUserID":"",
   "Version":1,
   "Name":"OrderCreated",
   "CreationDate":"2022-03-13T11:14:01.0284863-03:00",
   "Payload":{
      "ID":"ccfc8248-2ab5-4423-92fc-b16c4555add2",
      "CustomerName":"Bill",
      "CustomerTaxID":"string",
      "Total":100
   }
}

The microservice is listening to this queue and will already react to the event and process successfully.

Now return to RabbitMQ and send the same event again and you will notice that when it reaches the microservice it is automatically rejected.

To repeat the test, change a value in the json payload by putting another guid in the ID, for example.

Starting manual idempotency

Devprime allows you to manually enter a specific method that will support idempotency. The first step is to change the main setting and then go directly to the method to add the setting.

** Changing the “Action” key to “Manual” **

    "Idempotency": {
      "Enable": "true",
      "Alias": "State2",
      "Duration": "86400",
      "Flow": "backend",
      "key": "idempotency-key",
      "Scope": "all",
      "Action": "manual"
    }

In manual mode, when performing a test by sending duplicate events, you will observe that they will be accepted. You need to visit each desired method and add “Dp.Idempotency.Enable = true;” to enable the idempotency control.

** Modifying the Stream to have idempotency**
Open the Event Hub and locate the subscribe of the desired event which in this context is the “OrderCreated” and add the “Dp.Idempotency.Enable = true;”.
code src\Adapters\Stream\EventStream.cs

namespace Devprime.Stream;
public class EventStream : EventStreamBase, IEventStream
{
    public override void StreamEvents()
    {
        Subscribe<IPaymentService>("Stream1", "OrderCreated", (payload, paymentService, Dp) =>
        {
            Dp.Idempotency.Enable = true;

            var dto = System.Text.Json.JsonSerializer.Deserialize<OrderCreatedEventDTO>(payload);
            var command = new Payment()
            {
                CustomerName = dto.CustomerName,
                OrderID = dto.OrderID,
                Value = dto.Value
            };
            paymentService.Add(command);
        });
    }
}

When you try to run again with duplicate events, you will already notice that it is working again.

Devprime | Documentation – Idempotency

Examples: Implementing idempotency in microsservices

Parameters used in configuration

Creating a microservice to use in the example

Starting the idempotency configuration

Starting automatic idempotency in the microservice

Initiating manual idempotency in the microservice

More articles on idempotency

Examples: Idempotency in microsservices and REST APIs

Creating a microservice to use in the example

Starting the idempotency configuration

Starting automatic idempotency in the microservice

More articles on idempotency

Examples: Building an idempotent event-driven microservices

Creating a microservice to use in the example

Starting the idempotency configuration

Starting manual idempotency

More articles on idempotency