Kubernetes (DOKS)

Introduction

The Devprime platform accelerates software developer productivity by offering a complete software architecture design, components with intelligent behaviors, accelerators for code deployment, and updates with new features.

In this article, we will discuss the use of DigitalOcean’s Kubernetes service (DOKS), a cloud platform that offers several features for software developers to use in their applications.

We will use two microservices with one business rule related to orders and the other to payments using the MongoDB database managed by DigitalOcean. We’ll demonstrate the steps to build the microservice, cloud services on DigitalOcean like Kubernetes (DOKS), Container Registry, MongoDB and deploy RabbitMQ right on Kubernetes, then the initial configurations needed.

Checklist and preparation of the initial environment

• Open an account on Devprime Platform and purchase a Developer/Enterprise) license.
• Install an updated version of .NET (Linux, macOS, and Windows).
• Install and/or update Visual Studio Code and/or Visual Studio 2023 Community / Professional / Enterprise.
• Install and/or update docker (Linux, macOS, and Windows) (For Windows, use WSL2).
• Initialize the [MongoDB and RabbitMQ containers in docker] locally. /../../quick-start/docker/) and add queue ‘orderevents’ and ‘paymentevents’ in RabbitMQ.
• Install and activate the latest version of the Devprime CLI.
• Create a local folder for your projects and set read and write permissions.
• See the article “Creating the first microservice” to explore getting started on the Devprime platform.
• Open a Cloud account at DigitalOcean (Your $200 Credit, You’ve been invited to DigitalOcean! Sign up and get a $200, 60-day credit to try our products. Spend $25 after your credit expires and whoever referred you will get $25 in credit!).

Get started

In this article, we will use a project with two Microservices using asynchronous communication, with the first Order and the second Payment being that you can choose to create from scratch or using the example on GitHub.

a) Run a clone of the project on GitHub:
git clone https://github.com/devprime/devprime-microservices-order-payment.git

b) Access the cloned folder and check the home folder with the items Order and Payment. Each folder in this has a microservice developed with the Devprime platform and ready to use.
cd devprime-microservices-order-payment
git remote remove origin

c) Run the dp stack command to apply update and your Devprime license:
dp stack

d) Run each microservice in a terminal tab to test locally using the .\run.ps1 files on Windows or run.sh on Linux and macOS. Apply chmod permission on Linux, macOS.
chmod +x run.sh

Start the microservice by running one of the scripts below:
.\run.ps1 (Windows) or ./run.sh (Linux, macOS)

• /ms-order
• /ms-payment

With the microservices running, post to the order microservice API and track the processing in the payment microservice, as described in the article on asynchronous communication between microservices.

Creating a Kubernetes (DOKS), Container Registry, and MongoDB cluster on DigitalOcean

Access your portal in DigitalOcean and in the Kubernetes menu, enter the Create Cluster option to start the creation procedure and in node plan choose the appropriate size for your scenario, define the number of instances and proceed with the process. Perform the same procedure with Container Registry and in Database with MongoDB by creating the initial services for use in this article.

The image below shows a view of the first services created MongoDB and Kubernetes.

After creating the MongoDB cluster, it is essential to access the Connection String item and copy it to use in the configuration in the microservices (order/payment). Copy the entire connection string and make sure it has the username and password.

The Kubernetes cluster has been named “do-k8s-nyc1-demo” for use in this article. In the marketplace item you have some interesting features for deployment in the cluster using 1-Click Apps such as NGINX Ingress Controller, Cert-Manager, OpenEBS NFS Provisioner, Kubernetes Monitoring Stack.

To publish containers to Kubernetes, we’ll use the Container Registry which in this typico is named “dp-registry-nyc3”. Copy the url presented at this point, as we will use it later when creating the container image.

Now you need to take an important step by linking your Kubernetes Cluster with Container Registry for easy access to containers within the cluster. In the portal of your Kubernetes Router, go to Settings and then enter the “DigitalOcean Container Registry Integration” option and select the Container Registry.

Installing tools for use in Kubernetes

We will use some tools for integration with Kubernetes such as kubectl, which is the Kubernetes CLI, Helm, tool to install packages on Kubernetes, K9S tool to visualize the cluster, Stern, tool to view logs in Kubernetes, doctl, DigitalOcean’s command-line tool and docker.

After installing the doctl CLI, it is essential to follow the doctl configuration procedures to enable access in DigitalOcean and enable operations such as obtaining Container Registry and Kubernetes credentials.

Getting access to Kubernetes

Access credentials to the Kubernetes Cluster can be obtained through the portal or by using the doctl CLI, as we will do during this article.

• Obtaining the Cluster access credentials when confirming will be saved in the local environment “.kube/config”. Copy this command directly from the portal with your cluster’s Guid.
  doctl kubernetes cluster kubeconfig save a61716c9-c207-4c23-b789-399a811ef050

• Listing all pods in the Cluster
  kubectl get pods --all-namespaces

• Using k9s to view the Cluster
  k9s

In the image, we’re listing all the active containers deployed when the cluster was created. In the next steps, we will start creating the order and payment container.

Creating docker images and publishing to Container Registry

An important step in the Kubernetes landscape is to build the docker image of our microservices so that we can publish them within the Kubernetes Cluster environment.

Authenticating docker to Container Registry
doctl registry login

Building the order microservice image in docker

• Enter the first of the order microservice and run
  /ms-order
docker build -t ms-order .

• Create a tag for this image
  docker tag ms-order registry.digitalocean.com/dp-registry-nyc3/ms-order:latest

• Pushing to docker
  docker push registry.digitalocean.com/dp-registry-nyc3/ms-order:latest

Building the payment microservice image in docker

• Enter the second folder of the payment microservice and run
  /ms-payment
docker build -t ms-payment .

• Create a tag for this image
  docker tag ms-payment registry.digitalocean.com/dp-registry-nyc3/ms-payment:latest

• Pushing to docker
  docker push registry.digitalocean.com/dp-registry-nyc3/ms-payment:latest

At the end of this push procedure on the order and payment images, it will be possible to check them on the Container Registry portal and they are ready
for use in Kubernetes.

Deploying RabbitMQ on Kubernetes

The Devprime platform enables connectivity to several Stream services with RabbitMQ, Kafka and others and in this project we will use helm to deploy RabbitMQ in the cluster. Depending on the scenario in your project, you can choose to use the managed Kafka service offered by Digital Ocean or another alternative for communication between microservices.

Deploying RabbitMQ using Helm. Change the username and password at your discretion.
helm repo add bitnami https://charts.bitnami.com/bitnami
helm repo update
helm install rabbitmq bitnami/rabbitmq --set auth.username=admin --set auth.password=MQ@passX6DP

The RabbitMQ deployment process takes a while to complete, and you can follow along through K9S. After installation, you can map the RabbitMQ ports in the on-premises environment to access. Before performing the mapping, it is critical to stop the local RabbiMQ container to avoid TCP port conflicts.

RabbitMQ Ports

• RabbitMQ AMQP port:
  kubectl port-forward --namespace default svc/rabbitmq 5672:5672
• RabbitMQ Web Management interface:
  kubectl port-forward --namespace default svc/rabbitmq 15672:15672

Configuring the Initial Configuration of RabbitMQ
Map port 15672 and access the RabbitMQ administrative portal from url http://localhost:15672 to configure queues ‘orderevents’ and ‘paymentevents’ in RabbitMQ and link to the exchange as per the RabbitMQ documentation on Devprime for the purpose of enabling this article.

Analyzing MongoDB and RabbitMQ credentials in on-premises microservices environments

In the local environment, you can edit the application settings and credentials in the src/App/appsettings.json file, such as the MongoDB and RabbitMQ access data used in the project and other items available in the Devprime configuration documentation.

To open the configuration for each microservice, enter the related folder
/ms-order
/ms-payment

Open from Visual Studio Code or another editor
code src/App/appsettings.json

View in the RabbitMQ and MongoDB key the credentials used in the local environment and can be modified at any time.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

  "DevPrime_Stream": [
    {
      "Alias": "Stream1",
      "Enable": "true",
      "Default": "true",
      "StreamType": "RabbitMQ",
      "HostName": "Localhost",
      "User": "guest",
      "Password": "guest",
      "Port": "5672",
      "Exchange": "devprime",
      "ExchangeType": "direct",
      "Retry": "3",
      "Fallback": "State1",
      "Threads": "30",
      "Buffer": "1",
      "Subscribe": []
    }
  ],

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14

  "DevPrime_State": [
    {
      "enable": "true",
      "alias": "State1",
      "type": "db",
      "dbtype": "mongodb",
      "connection": "mongodb://mongoadmin:LltF8Nx*yo@localhost:27017",
      "timeout": "5",
      "retry": "2",
      "dbname": "ms-order",
      "isssl": "true",
      "durationofbreak": "45"
    }
  ],

Configuring MongoDB and RabbitMQ credentials on Kubernetes

In our context, we will create a dpeloyment file for each microservice and use Devprime CLI to build the initial file deployment.yml and service.yml by running the command in the folder of each microservice and then we will perform the configuration with the credentials that we will use in the ordeer and payment microservices within the Kubernetes cluster.

Go into each microservice folder and run “dp export kubernetes”
/ms-order
/ms-payment
dp export kubernetes

The result will be similar to the excerpt below with two files created with the initial settings needed for us to publish
on Kubernetes.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

 ____             ____       _
|  _ \  _____   _|  _ \ _ __(_)_ __ ___   ___
| | | |/ _ \ \ / / |_)| '__| | '_ ` _ \ / _ \
| |_| |  __/\ V /|  __/| |  | | | | | | |  __/
|____/ \___| \_/ |_|   |_|  |_|_| |_| |_|\___|
  Empower developers. Accelerate productivity
==============================================
Welcome to the DevPrime CLI v8.0.8

Export configurations kubernetes

.devprime/kubernetes/deployment.yml' created
.devprime/kubernetes/service.yml' created

Now it’s time to edit the “deployment.yml” file and locate
The “image” item in the order folder and in the payment folder.

Open the folder in Visual Studio Code or another editor
code .devprime

File: deployment.yml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17

apiVersion : apps/v1
kind: Deployment
metadata:
  name: ms-order
spec:
  replicas: 1
  selector:
    matchLabels:
      app: ms-order
  template:
    metadata:
      labels:
        app: ms-order
    spec:
      containers:
        - name: ms-order 
          image: ms-order:latest

Perform the procedure for each microservice (order/payment) and change the item image to the path of your image in Container Registry. The url
of the Container Registry is the same one used by the docker push command.

File: order/deployment.yaml

1
2
3

      containers:
        - name: ms-order 
          image: registry.digitalocean.com/dp-registry-nyc3/ms-order:latest

File: payment/deployment.yaml

1
2
3

      containers:
        - name: ms-order 
          image: registry.digitalocean.com/dp-registry-nyc3/ms-payment:latest

Now it’s time to configure the RabbitMQ and RabbitMQ production credentials on each microservice.
MongoDB provided by DigitalOcean and dealt with at the beginning of the file where we talked about Connection String.

File:deployment.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

         env:
           - name: devprime_stream1
             value: "alias=Stream1|||enable=true|||default=true|||streamtype=RabbitMQ
        |||hostname=rabbitmq.default.svc|||user=admin|||password=guest|||port=5672|||
             exchange=devprime|||exchangetype=direct|||retry=3|||fallback=State1|||
             threads=30|||buffer=1"
           - name: devprime_state1
             value: "enable=true|||alias=State1|||type=db|||dbtype=mongodb|||
             connection=mongodb.default.svc|||timeout=5|||retry=2|||
             dbname=ms-order|||isssl=true|||durationofbreak=45"

In item devprime_stream1 change the user to “admin” and the password to “MQ@passX6DP”.
in item devprime_state1 change the connection to the MongoDB Connection String in DigitalOean.

Below you can see an example after the changes that must be made to the microservices deployment files (order/payment). Make the change directly to each file to avoid modifying other parameters in this demo, as the keys have slight differences between microservices.

File:deployment.yaml

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13

         env:
           - name: devprime_stream1
             value: "alias=Stream1|||enable=true|||default=true|||streamtype=RabbitMQ|||
             hostname=rabbitmq.default.svc|||user=admin|||password=MQ@passX6DP|||
             port=5672|||exchange=devprime|||exchangetype=direct|||retry=3
             |||fallback=State1|||threads=30|||buffer=1"
           - name: devprime_state1
             value: "enable=true|||alias=State1|||type=db|||dbtype=mongodb|||
             connection=mongodb+srv://user:password@
             do-mongodb-nyc1-demo-29a3fecf.mongo.ondigitalocean.com/admin?
             tls=true&authSource=admin&replicaSet=do-mongodb-nyc1-demo|||timeout=5|||
             retry=2|||dbname=ms-order|||isssl=true|||durationofbreak=45"
           - name: Devprime_Custom

Deploying microservices to Kubernetes

Now the most awaited moment has arrived, where we will effectively publish the microservices (order and payment) within the Kubernetes cluster and for this we will use the deployment.yml and service.yml files of each microservice.

Enter each folder and run the commands below:

• For the order microservice

  • Navigate to the Kubernetes folder for the order service:
    ms-order/.devprime/kubernetes
  • Apply deployment and service:
    kubectl apply -f deployment.yml
kubectl apply -f service.yml

• For the ayment microservice

  • Navigate to the Kubernetes folder for the payment service:
    ms-payment/.devprime/kubernetes
  • Apply deployment and service:
    kubectl apply -f deployment.yml
kubectl apply -f service.yml

Now that we’ve finished deploying the two microservices, we can view directly on the cluster using the k9s tool or using kubectl.

Performing a test on the microservice

Open a terminal tab and run the port-forward command to map the order microservice on local port 8080 and allow you to perform the first test by accessing the platform directly in the cluster.

• Run the command for the Order
  kubectl port-forward --namespace default svc/ms-order 8080:80

• Go to the URL and POST the order microservice to process and emit an event in the RabbitMQ queue and propagate to the payment microservice
  http://localhost:8080

Assigning a public IP to the order microservice

Open the ms-order/.devprime/kubernetes/service.yml file with Visual Studio Code and change the “ClusterIP” to “LoadBalancer”:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

apiVersion: v1
kind: Service
metadata:  
  name: ms-order
spec:
  selector:    
    app: ms-order
  type: LoadBalancer
  ports:  
  - name: http
    port: 80
    targetPort: 80

Deleting the ms-order service:
kubectl delete svc ms-order

Recreating the ms-order service:
kubectl apply -f service.yml

Now refer to the update until you receive a public IP in the order microservice:
kubectl get svc ms-order -w

NAME       TYPE           CLUSTER-IP     EXTERNAL-IP     PORT(S)        AGE
ms-order   LoadBalancer   10.245.38.41   157.x.x.x   80:31256/TCP   2m32s

Viewing logs in Kubernetes

Microservices based on the Devprime platform provide an [automatic observability of distributed logs] approach. /../../features/observability/) that can be viewed by kubectl, k9s, stern and indexed in tools such as SEQ, Elastic Search and many others to track the behavior of applications in the production environment.

In this example, we’ll use the stern tool to view the log for each deployment:

• Open a tab for the ‘order’ microservice in the terminal and run:
  stern ms-order
• Open a tab for the ‘payment’ microservice in the terminal and run:
  stern ms-payment
• Now, use the port or public IP mapping and post to the ‘order’ microservice to check a result. In the log below, you’ll see a summary of the ‘order’ as an example.

[INF][ms-order][Web]["http://*:80"][Host]["Production"][Parameters]["Environment"][RID 30c1c7a9-7799-43e6-a1b2-5d4007b3edb8][TID 30c1c7a9-7799-43e6-a1b2-5d4007b3edb8]
[INF][ms-order][Stream][Type "RabbitMQ"][Alias "Stream1"]["Enable"][RID 30c1c7a9-7799-43e6-a1b2-5d4007b3edb8][TID 30c1c7a9-7799-43e6-a1b2-5d4007b3edb8]
[INF][ms-order][Web]["HTTP"][Order][POST /v1/order][Origin "http://localhost/swagger/index.html"][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][Application][OrderService][Add][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][Domain][Order][Add][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][Domain][Order][ProcessEvent]["CreateOrder"][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][Application][EventHandler]["CreateOrderEventHandler"][Event]["CreateOrder"][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][State][Type "MongoDB"][Alias "State1"][Initialize][OrderRepository][Add][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][State][Type "MongoDB"][Alias "State1"][Complete][OrderRepository][Add][Duration 1192.1472ms][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][Domain][Order][ProcessEvent]["OrderCreated"][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][Application][EventHandler]["OrderCreatedEventHandler"][Event]["OrderCreated"][RID c1461cc3-7000-473d-a62c-915867954ced][TID c1461cc3-7000-473d-a62c-915867954ced]
[INF][ms-order][Stream][Type "RabbitMQ"][Alias "Stream1"][Out][Event]["OrderCreated"]["Delivered"]["orderevents"]
["{\""Payload\":{\"ID\":\"ca9983\",\"CustomerName\":\"Ramon Duraes\",\"CustomerTaxID\":\"AC875512\",\"Total\":1200}}"]

Final thoughts

During this article, we demonstrated the publication of two microservices (order and payment) developed using Devprime platform technology on DigitalOcean’s Kubernetes Cluster (DOKS), in conjunction with the managed Container Registry and MongoDB services and RabbitMQ that we deployed on the cluster using Helm. You also followed how to access the service using an internal IP and an External IP and view the logs.

In a productive environment, it is essential to add an nginx ingress, an internet domain and Let’s Encrypt to generate an SSL certificate, use a DevOps service to automate publishing, digital vault for credentials, access protection with an API Gateway, identity provider and Web Application Firewall.

Find out more

• Deploying a Devprime microservice on a DigitalOcean Droplet