Stream

The Stream Adapter offers native connectivity with the main Stream services on the market, allowing you to use platforms such as RabbitMQ, Kafka, Google PubSub, Azure Service Bus, Azure EventHub, and many other services just by exchanging the information in the configuration, enabling a rich experience in microservices development.

The Stream Adapter settings are performed in the “DevPrime_Stream” key within the “src/App/appsettings.json” file in the development environment. In the production environment, we recommend that configurations be passed through the DevOps process, with credential protection through the use of a Vault.

Opening Settings via Visual Studio Code

1

code src/App/appsettings.json

The main configuration parameters are the “Alias” items which acts as an identifier for the service, Enable which enables the service on the adapter and StreamType which defines the type of service to be used. On each platform we will have custom configuration items according to the context.

Configuring the Stream Adapter with RabbitMQ

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


  "DevPrime_Stream": [
    {
      "Enable": "true",
      "Alias": "Stream1",      
      "Default": "true",
      "StreamType": "RabbitMQ",
      "HostName": "Localhost",
      "User": "guest",
      "Password": "guest",
      "Port": "5672",
      "Exchange": "devprime",
      "ExchangeType": "direct",
      "Retry": "3",
      "Fallback": "State1",
      "Threads": "30",
      "Buffer": "1",
      "Subscribe": []
    }
  ],

In the example below, we’re detailing the default Adapter settings and the RabbitMQ-specific settings.

General	—
Enable	Enable the Stream adapter (True/False)
Alias	Define a unique name to identify the service
Default	Sets the default for the centary to have more than one Stream (True/False)
StreamType	Sets the Stream Type (RabbitMQ/Kafka)
HostName	Configure the RabbitMQ
User	Configures the user to connect
Password	Configures the user’s password
Port	Configure the port
Exchange	Configures the default RabbitMQ
ExchangeType	Configures the Exchange type
Retry	Sets the number of retries
Fallback	Sets a State for automatic persistence in case of failure
Threads	Sets the number of concurrent connections processing events
Buffer	Sets the number of events obtained per connection
Subscribe	Configures RabbitMQ queues for read

Stream Adapter supports multiple Streams at the same time

The Devprime Platform Stream Adapter supports more than one streaming service simultaneously, to meet specific scenarios where you need to have both RabbitMQ and Kafka in the same microservice.

In the example below, both RabbitMQ and Kafka were added at the same time. It is important to note that each item has an alias (Stream1/Stream2), and only Stream1 is set to “Default”.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32


  "DevPrime_Stream": [
    {
      "Enable": "true",
      "Alias": "Stream1",      
      "Default": "true",
      "StreamType": "RabbitMQ",
      "HostName": "Localhost",
      "User": "guest",
      "Password": "guest",
      "Port": "5672",
      "Exchange": "devprime",
      "ExchangeType": "direct",
      "Retry": "3",
      "Fallback": "State1",
      "Threads": "30",
      "Buffer": "1",
      "Subscribe": []
    },
    {
    "Alias": "Stream2",
    "Enable": "true",
    "Default": "false",
    "StreamType": "Kafka",
    "HostName": "Localhost",
    "User": "",
    "Password": "",
    "Port": "9092",
    "Retry": "3",
    "Fallback": "State1",
    "Subscribe": []
  }
  ],

Example in the application log with this configuration:

1
2


[INF][Stream][Type "RabbitMQ"][Alias "Stream1"]["Enable"]
[INF][Stream][Type "Kafka"][Alias "Stream2"]["Enable"]

YAML example with application settings:
To export the configurations as an environment variable and use in Kubernetes, run the command in the Devprime CLI:

1

dp export kubernetes

Example of the yaml generated with the settings for use in the DevOps flow.

1
2
3
4
5
6
7
8


- name: devprime_stream1
  value: "alias=Stream1|||enable=true|||default=true|||streamtype=RabbitMQ|||
  hostname=rabbitmq.default.svc|||user=guest|||password=guest|||port=5672|||
  exchange=devprime|||exchangetype=direct|||retry=3|||fallback=State1|||
  threads=30|||buffer=1" 
- name: devprime_stream2
  value: "alias=Stream2|||enable=true|||default=false|||streamtype=Kafka|||
  hostname=kafka.default.svc|||port=9092|||retry=3|||fallback=State1"

Explore Implementation Examples

Asynchronous communication between microservices with RabbitMQ
Asynchronous communication between microservices with Kafka
Asynchronous communication between microservices with Azure Event Hubs. /../../examples/stream/azure-event-hubs/asynchronous-microservices-communication/)

For detailed information about the settings for each platform, see the specific settings.

Communication and integration protocol

Applications developed with Devprime use an event-driven architecture (EDA) for asynchronous communication and use a standard protocol, ensuring fluid communication and integration between microservices. This approach ensures traceability and improves observability through logs and distributed tracing.

Enabling events on the Steam Adapter

Use the Devprime platform to subscribe to events with queues, topics using the Stream Adapter to receive notifications in the event hub and then distribute either to standard Devprime events and/or non-standard events.

Enabling Multistream Communication

The Devprime platform, through Adapter Stream, offers the Multistream approach, allowing the integration of more than one Stream service into a single microservice.

RabbitMQ

The Devprime platform uses a cloud-native approach, supporting multiple stream platforms and native integration with RabbitMQ via the Stream Adapter. This open source messaging software uses the Advanced Message Queuing Protocol (AMQP), providing rich experience in microservices development and event-driven architecture.

Kafka

The Devprime platform natively supports Apache Kafka, a streaming platform developed by the Apache Software Foundation. The project aims to provide a unified, high-capacity, low-latency platform for real-time data processing.

CloudAMQP

The Devprime platform natively supports CloudAMQP, which provides a SaaS approach to RabbitMQ, an open-source messaging software that operates on the Advanced Message Queuing Protocol (AMQP) and Streaming Text Oriented Messaging Protocol (MQTT) protocols.