Kafka

The Devprime platform natively supports Apache Kafka, a streaming platform developed by the Apache Software Foundation. The project aims to provide a unified, high-capacity, low-latency platform for real-time data processing.

Apache Kafka can be used on-premise, cloud, docker, kubernetes, and managed platforms such as Confluent Cloud, Azure Event Hub, and others.

Setting Up Stream with Kafka

The configuration example below demonstrates the use of Apache Kafka defined with the Alias “Stream1” and as the default for sending events. By enabling the “Subscribe” item, we are subscribing to this topic and receiving events in the Stream Adapter Hub.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19


  "DevPrime_Stream": [
    {      
      "Enable": "true",
      "Alias": "Stream1",
      "Default": "true",
      "StreamType": "Kafka",
      "HostName": "Localhost",
      "User": "",
      "Password": "",
      "Port": "9092",
      "Retry": "3",
      "Fallback": "State1",
      "Subscribe": [
        {
          "Topics": "orderevents"
        }
      ]
    }
  ],

In the example below, we’re detailing the default Adapter settings and Kafka-specific settings.

General	—
Enable	Enable the Stream adapter (True/False)
Alias	Defines a unique name to identify the service
Default	Sets as default to the centary of having more than one Stream (True/False)
StreamType	Sets the Stream type (RabbitMQ
HostName	Configure the Kafka
User	Configure the user
Password	Configures the user’s password
Port	Configure the port
Retry	Configure the number of retries
Fallback	Configures a State for automatic resitence in case of failure
Subscribe	Set up Kafka

Example in the application log with this configuration:

1

[INF][Stream][Type "Kafka"][Alias "Stream1"]["Enable"][Subscribe]["orderevents"]

YAML example with application settings:

1
2
3
4


- name: devprime_stream1
    value: "alias=Stream1|||enable=true|||default=true|||streamtype=Kafka|||
    hostname=kafka.default.svc|||port=9092|||retry=3|||fallback=State1|||
    subscribe=[topics=orderevents]" 

Last modified January 10, 2024 (967dcac3)