Kafka

The Devprime platform natively supports Apache Kafka, a streaming platform developed by the Apache Software Foundation. The project aims to provide a unified, high-capacity, low-latency platform for real-time data processing.

Apache Kafka can be used on-premise, cloud, docker, kubernetes, and managed platforms such as Confluent Cloud, Azure Event Hub, and others.

Setting up the Stream with Kafka

The configuration example below demonstrates the use of Apache Kafka defined with Alias “Stream1” and as the default for sending events. By enabling the “Subscribe” item, we are subscribing to this topic and receiving events in the Stream Adapter Hub.

General	—
Enable	Enable the Stream adapter (True/False)
Alias	Define a unique name to identify the service
Default	Sets the default for the centary to have more than one Stream (True/False)
StreamType	Sets the Stream type (RabbitMQ
HostName	Configure the Kafka
User	Configures the user
Password	Configures the user’s password
Port	Configure the port
Retry	Sets the number of retries
Fallback	Sets a State for automatic persistence in case of failure
Subscribe	Configures Kafka

Standard Kafka Configuration Example

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15


  "DevPrime_Stream": [
    {      
      "Enable": "true",
      "Alias": "Stream1",
      "Default": "true",
      "StreamType": "Kafka",
      "HostName": "Localhost",
      "User": "",
      "Password": "",
      "Port": "9092",
      "Retry": "3",
      "Fallback": "State1",
      "Subscribe": []
    }
  ],

Example of Kafka configuration with subscription in the topics: notifications and paymentevents
When adding more than one topic, it is necessary to separate them by a comma “,” as demonstrated.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


  "DevPrime_Stream": [
    {      
      "Enable": "true",
      "Alias": "Stream1",
      "Default": "true",
      "StreamType": "Kafka",
      "HostName": "Localhost",
      "User": "",
      "Password": "",
      "Port": "9092",
      "Retry": "3",
      "Fallback": "State1",
      "Subscribe": [
        {
          "Topics": "notifications,paymentevents"
        }
      ]
    }
  ],

Example in the application log with this configuration:

1

[INF][Stream][Type "Kafka"][Alias "Stream1"]["Enable"][Subscribe]["orderevents"]

YAML example with application settings:

1
2
3
4


- name: devprime_stream1
    value: "alias=Stream1|||enable=true|||default=true|||streamtype=Kafka|||
    hostname=kafka.default.svc|||port=9092|||retry=3|||fallback=State1|||
    subscribe=[topics=orderevents]" 

Last modified November 20, 2024 (61099f59)