Stream

The Stream Adapter offers native connectivity with the main Stream services on the market, allowing you to use platforms such as RabbitMQ, Kafka and many other services just by exchanging the information in the configuration, enabling a rich experience in the development of microservices.

The Stream Adapter settings are performed in the “DevPrime_Stream” key within the “src/App/appsettings.json” file in the development environment. In the production environment, we recommend that configurations be passed through the DevOps process, with credentials protected through the use of a vault.

Opening Settings from Visual Studio Code
code src/App/appsettings.json

The main configuration parameters are the items “Alias” which acts as an identifier for the service, Enable which enables the service on the adapter, and StreamType which defines the type of service to be used. On each platform, we will have customized configuration items according to the context.

Setting Up the Stream Adapter with RabbitMQ

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

  "DevPrime_Stream": [
    {
      "Enable": "true",
      "Alias": "Stream1",      
      "Default": "true",
      "StreamType": "RabbitMQ",
      "HostName": "Localhost",
      "User": "guest",
      "Password": "guest",
      "Port": "5672",
      "Exchange": "devprime",
      "ExchangeType": "direct",
      "Retry": "3",
      "Fallback": "State1",
      "Threads": "30",
      "Buffer": "1",
      "Subscribe": []
    }
  ],

In the example below, we’re detailing the default Adapter settings and the RabbitMQ-specific settings.

General —
Enable Enable the Stream adapter (True/False)
Alias Defines a unique name to identify the service
Default Sets as default to the centary of having more than one Stream (True/False)
StreamType Sets the Stream type (RabbitMQ/Kafka)
HostName Configure the url of RabbitMQ
User Configures the user to sign in
Password Configures the user’s password
Port Configure the port
Exchange Sets up the default RabbitMQ
ExchangeType Configures the type of Exchange
Retry Configure the number of retries
Fallback Configures a State for automatic resitence in case of failure
Threads Sets the number of concurrent connections processing events
Buffer Defines the number of events obtained per connection
Subscribe Configures RabbitMQ Queues for Read

Stream Adapter supports multiple Stream at the same time

The Devprime Platform Stream Adapter supports more than one streaming service simultaneously, to cater for specific scenarios where you need to have both RabbitMQ and Kafka on the same microservice.

In the example below, both RabbitMQ and Kafka were added at the same time. It’s important to note that each item has an alias (Stream1/Stream2), and only Stream1 is set to “Default”.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

  "DevPrime_Stream": [
    {
      "Enable": "true",
      "Alias": "Stream1",      
      "Default": "true",
      "StreamType": "RabbitMQ",
      "HostName": "Localhost",
      "User": "guest",
      "Password": "guest",
      "Port": "5672",
      "Exchange": "devprime",
      "ExchangeType": "direct",
      "Retry": "3",
      "Fallback": "State1",
      "Threads": "30",
      "Buffer": "1",
      "Subscribe": []
    },
    {
    "Alias": "Stream2",
    "Enable": "true",
    "Default": "false",
    "StreamType": "Kafka",
    "HostName": "Localhost",
    "User": "",
    "Password": "",
    "Port": "9092",
    "Retry": "3",
    "Fallback": "State1",
    "Subscribe": []
  }
  ],

Example in the application log with this configuration:

1
2

[INF][Stream][Type "RabbitMQ"][Alias "Stream1"]["Enable"]
[INF][Stream][Type "Kafka"][Alias "Stream2"]["Enable"]

YAML example with application settings:
To export the configurations as an environment variable and use in Kubernetes, run the command in the Devprime CLI:
dp export kubernetes

1
2
3
4
5
6
7
8

- name: devprime_stream1
  value: "alias=Stream1|||enable=true|||default=true|||streamtype=RabbitMQ|||
  hostname=rabbitmq.default.svc|||user=guest|||password=guest|||port=5672|||
  exchange=devprime|||exchangetype=direct|||retry=3|||fallback=State1|||
  threads=30|||buffer=1" 
- name: devprime_stream2
  value: "alias=Stream2|||enable=true|||default=false|||streamtype=Kafka|||
  hostname=kafka.default.svc|||port=9092|||retry=3|||fallback=State1"

Explore an example of asynchronous communication between microservices'

For detailed information on the settings for each platform, please refer to the specific settings.

RabbitMQ

The Devprime platform natively supports RabbitMQ, an open source messaging software that operates on the Advanced Message Queuing Protocol (AMQP) and Streaming Text Oriented Messaging Protocol (MQTT) protocols.

Kafka

The Devprime platform natively supports Apache Kafka, a streaming platform developed by the Apache Software Foundation. The project aims to provide a unified, high-capacity, low-latency platform for real-time data processing.

CloudAMQP

The Devprime platform natively supports CloudAMQP, which provides a SaaS approach to RabbitMQ, an open-source messaging software that operates on the Advanced Message Queuing Protocol (AMQP) and Streaming Text Oriented Messaging Protocol (MQTT) protocols.

Azure Service Bus

You can work with normal queues, fanout

Google Pub/Sub

You can work with normal queues, fanout

AWS SQS

You can work with normal queues, fanout

Multistream

The Devprime platform, through the Stream Adapter, offers the Multistream approach, allowing the integration of more than one Stream service into a single microservice.

Last modified April 11, 2024 (cc33f7e6)