DevOps Archives - gopaddle blog

Release notes gopaddle v4.1

Posted on June 8, 2021November 25, 2021 | Read time: 3 min. read | Announcements

gopaddle v4.1 is a minor update with a few bug fixes and minor enhancements.

In this minor release we have introduced a gopaddle command line utility – gpctl to scaffold applications and import existing Kubernetes workloads to gopaddle. We have added support for Huawei Cloud and features that help in advanced configurations for service deployments and Docker builds.

gpctl import

Imports a pre-existing Kubernetes project with YAML files to gopaddle.

Use cases:
- Migrate from GitOps to GUI based Cloud Native Governance platform
- Migrate from one cloud platform to another by building a reusable gopaddle template.
Support Matrix:
- Supported on Ubuntu 20.04
- Supports github, bitbucket and gitlab source control repositories
- If the Kubernetes YAML files in the Kubernetes project have references to docker images, then they can be linked during the gpctl import. gptcl supports Docker public & private hub, AWS ECR, Google GCR, Azure ACR, and any Quay based private repositories.
- Here is a quick reference to the import capability – https://help.gopaddle.io/en/articles/5019209-import-an-existing-kubernetes-project-to-gopaddle

gpctl init (Alpha Version)

Initialize a microservice from source control repository and deploy to kubernetes in minutes.

Use cases:
- Code to Kubernetes using a single command
- Automatically create a Docker file by profiling the microservice
- Automatically generate Kubernetes YAML files
- Deploy the service on Kubernetes and get the end point to access the service
Support Matrix:
- Supported on Ubuntu 18.04 or later
- Only github based projects are supported
- Java, NodeJS, Python and any type of linux based workloads that do not bring up a terminal
- Here is a quick reference to initialize a microservice from source control repository – http://help.gopaddle.io/en/articles/5056807-initializing-a-microservice-from-scratch

Huawei Cloud Support

Use a pre-existing Huawei Cloud as an external cluster by registering it via Bastion Host or directly using its Private IP address

Use case:
- Use Huawei Cloud for building docker containers
- Use Huawei Cloud for deploying applications
- Launch Stateful services on Huawei Cloud using Elastic Volume Service (EVS)
- Use Huawei LoadBalancer to access services via Domain Name and Ingress rules
- Launch Services on Huawei Cloud and access them using Public Elastic IP
Open Issues
- Container Terminal doesn’t work when the Cluster is registered via Bastion Host
- Here is a quick start reference to use Huawei cloud with gopaddle – https://help.gopaddle.io/en/articles/5088697-registering-and-using-huawei-cloud-container-engine-via-bastion-host

Shared Persistent Volumes

Use Shared persistent volumes across services within an application.

Use case:
- Share data across services within an application. Say Share Consul TLS Certificates across other Services within the same application.
- Here is a quick reference to how to create shared volumes across services – http://help.gopaddle.io/en/articles/5183639-creating-shared-volumes-across-services

Dockerfile Custom Path and Build Arguments

Use Dockerfile custom paths and build ARG while creating a Docker Project in gopaddle.

Use cases:
- If the Docker project uses Dockerfile in a location other than the project root, the custom docker file path can be provided at the time of adding the build scripts to a container
- If the Container build requires certain environment variables to be set, it can be provided using the Build arguments at the time of adding the build scripts to a container
- Here is a quick reference on how to use custom Dockerfile path and build arguments –http://help.gopaddle.io/en/articles/3570925-adding-build-scripts

Version Control ConfigMaps and Secrets

Configurator integration: Use configurator to update and maintain configMaps and secrets in gopaddle

Use cases:
- Keep services and configMaps/secrets in sync
- Version control configMaps/secrets
- Perform rolling updates or rollbacks on configMaps/secrets along with Deployments and Stateful sets

Custom Ingress

Use Ingress Controller for deployments. Define custom ingress controller like any other service in gopaddle and add it to the deployment templates.

Use Case:
- Use custom ingress controller other than the default controller provided by gopaddle
- Custom Ingress Controllers can be on-boarded like any other service. Here is a quick reference to enable custom ingress at the time of creating a service – http://help.gopaddle.io/en/articles/3645657-creating-a-service

Azure Autoscaling

Define autoscaling triggers for Azure node pools.

Use Case:
- Set autoscaling rules to scale the AKS nodes based on various factors like CPU / Disk Utilization
- Here is a quick reference on setting the AKS rules. http://help.gopaddle.io/en/articles/5316151-azure-autoscaling-rules

Node and Service Affinity Rules

Use Case:
- Schedule and Co-locate Services based on Node and Service Affinity
- Here is a quick reference on setting the Node and Service Affinity – http://help.gopaddle.io/en/articles/5316070-service-and-node-affinity

About The Author

UI Developer & QA @ gopaddle

← Back to blog home

Strange things you never knew about Kubernetes ConfigMaps on day one

Posted on April 1, 2021August 16, 2021 | Read time: 7 min. read | configurator

Deploying on Kubernetes is one complex task, but dealing with surprises during maintenance is another. I can talk from my own experience of maintaining our gopaddle platform deployments on Kubernetes for more than a year now. Even with careful planning and collective knowledge from within the team, there are still hidden challenges in keeping the deployments intact. We get to learn those hidden challenges only by running the production systems on kubernetes for a while. This blog is one such wisdom we learnt. I would like to share our experience with ConfigMaps and what solution we built (open source) to overcome some trouble with ConfigMaps. For the rest of the blog, I am going to be focusing on ConfigMaps, but secrets also have the same set of challenges. Though I have referenced deployments through out the blog, the given scenario and solution exists for kubernetes secrets as well.

ConfigMaps and Secrets are often overlooked topics when it comes to Cloud Native Deployments. But they can add unforeseen challenges during application maintenance. Let me first introduce you to what ConfigMaps are.

ConfigMaps are Kubernetes resources that are used to store application configurations. It enables build time and run time attribute segregation in a cloud native deployment. Say by using ConfigMaps, you do not have to package application configurations along with your container images. Thus changing application configurations does not require the entire application to be rebuilt. We leverage ConfigMaps to keep our applications 12-factor compatible. In a nutshell, ConfigMaps are:

Collection of regular files or key/value pairs
Can be used to set Environment variables inside a container (using ValueFrom: ConfigMapRef to refer to values defined in ConfigMaps)
Can be mounted as directories inside containers (using VolumeMounts/MountPath keywords) and all the files within ConfigMaps get mounted inside the container on the mountPath provided.
Shared across deployments/replicas
Confined to a namespace
Created from files, literals, kustomize configMapGenerator
Replaces all files within the mount path : Since ConfigMaps are mounted inside the container on a given mount path, any existing files and folders within the container will not be available.

Here is an example of how ConfigMaps are defined and referenced inside a deployment specification.

Example of defining and using ConfigMaps inside deployment specifications

Hidden challenges of ConfigMaps

Some of the challenges with ConfigMaps are realized as soon as they are mounted inside the container. But some are unearthered only during maintenance. Following are some challenges we have observed :

Can’t execute files in ConfigMaps : Starting from K8s 1.9.6, ConfigMaps get mounted as read-only files by default. Hence you may not be able to execute or run these files. Say, if you are planning on executing these files as container EntryPoint or CMD ARGs, then container may crash during startup as it cannot execute these read-only files. If you have recently upgraded the cluster version, then your deployments may break due this. Please check this K8s issue for information on how to configure ReadOnlyAPIDataVolumes to mount ConfigMaps as ReadWrite files.
Deployments & ConfigMaps are loosely coupled, ie., they follow different lifecycle, but updating the contents of a ConfigMap automatically reflects inside the Pods. More often, applications running inside the containers need a restart to pick the new changes. But, applications are clueless of the changes. These changes are noticed during a scale up/down or a Pod restart event when the application inside the container gets restarted.
No Versioning/No Rollbacks of ConfigMaps: ie when deployments are rolled back, it does not roll back the contents of the ConfigMaps.

The last two issues are the resultant of the mutable nature of ConfigMaps.

ConfigMaps are mutable

ConfigMaps are mutable ie., they can be edited. Every time a change is made, it is the same ConfigMap that gets updated. There are no revisions. Let me illustrate this with an example.

We have a ConfigMap, which is referenced in two different deployments. When you change the ConfigMap, the contents of the ConfigMap changes inside the deployments. When the deployments are rolled back, they still point to the current content of the ConfigMap. This can cause a problem when your application is expecting something but it actually sees something else. Deployments do not maintain any state regarding the ConfigMap changes.

Workarounds

Smart apps: Applications can be designed in such a way that they constantly poll for changes in the ConfigMaps. This approach still cannot address the roll back issue.
Induced Rolling Update: Another common approach is to hash the contents of the ConfigMap in to the deployment. When the ConfigMap changes, the hash changes and that automatically triggers a rolling update on the deployment. But even in this case, rolling back a deployment does not roll back to the previous content of the ConfigMap. Here is a reference to how ConfigMap hash can be used.
Immutable ConfigMaps: The next option is to use ConfigMap as an immutable content. This feature was introduced in kubernetes 1.19. When this feature is turned on, you cannot update a ConfigMap and thus you can avoid all the associated problems.

Versioning the ConfigMaps

The ideal solution to keep the deployments and the ConfigMaps in sync is to version control the ConfigMaps and reference them in the deployments.

In the above example, when the contents of the ConfigMap version 1 is updated, it creates a new ConfigMap version 2. When the deployment specification is updated with ConfigMap version 2, it automatically triggers a rolling update and creates a new deployment version. When the deployment is rolled back, the rolled back version will reference ConfigMap v1. Thus ConfigMaps and deployments go hand in hand.
To make this work, we need to :

Version ConfigMaps whenever a change is committed to a ConfigMap
Automatically update ConfigMap versions in Deployment Specifications where ever it being referenced
Purge unused ConfigMaps periodically – Since ConfigMaps are shared resources across deployments and since each deployment may have a different revision history limits, we must consider checking all the revisions of all the deployments within the namespace to know if a ConfigMap is being used and purge accordingly.

Introducing Configurator

Configurator is an open source solution from gopaddle that makes use of Custom Resource Definitions (CRDs) for ConfigMaps/Secrets and an operator to automate the above mentioned steps. ConfigMaps and Secrets are now defined as CustomConfigMaps and CustomSecrets which are custom resources constantly monitored by the Configurator.

How it solves the problem

When a new CustomConfigMap or a CustomSecret resource is created, it generates a ConfigMap or a Secret with a postfix. This ConfigMap along with the post fix need to be added to the deployments/statefulsets initially.

From then on, if any change to the CustomConfigMap or CustomSecret is detected, configurator automatically updates all the deployments/statefulsets referencing the specific ConfigMap/Secret with a postfix. Configurator heavily depends on the configMapName in the CustomConfigMap and the labels in the deployment/statefulset specifications.

In the above example you can see that the example-customConfigMap.yaml creates a CustomConfigMap with the configMapName as testconfig. As soon as the CustomConfigMap is created, it automatically creates a ConfigMap testconfig-sn8ya. We need to manually add this ConfigMapName testconfig and the postfix sn8ya in the deployment’s metadata.labels as testconfig: sn8ya and also use the ConfigMapName testconfig-sn8ya in the volumes section.

From now on, user does not have to manage ConfigMaps directly.

Any change required in the ConfigMap or Secret needs to be done through the CustomConfigMap or CustomSecret.

When the CustomConfigMap is updated with the new content in the data section, it automatically generates a new ConfigMap testconfig-10jov and updates the deployment with the new ConfigMap name under the volumes and the metadata.label section.

Configurator purges unused ConfigMaps and Secret every 5 mins. It scans the replicaset or controllerRevision of all the deployments and statefulsets in the namespace and checks if the metadata.label exists for the ConfigMap. If there are no references, it purges the ConfigMap version.

How to use it ?

Download run the YAML files from our repository here and install them in your cluster.

kubectl apply -f deploy/crd-customConfigMap.yaml
kubectl apply -f deploy/crd-customSecret.yaml
kubectl create ns configurator
kubectl apply -f deploy/configurator-clusterrole.yaml
kubectl apply -f deploy/configurator-clusterrolebinding.yaml
kubectl apply -f deploy/configurator-serviceaccount.yaml
kubectl apply -f deploy/configurator-deployment.yaml

Once the configurator is deployed into the cluster, start creating CustomConfigMap or CustomSecret.

example-customConfigMap.yaml

apiVersion: "configurator.gopaddle.io/v1alpha1"
kind: CustomConfigMap
metadata:
 name: configtest
 namespace: test
spec:
  configMapName: testconfig
  data:
   application.properties: |
    FOO=Bar

Create CustomConfigMap in cluster

Kubectl apply -f example-customConfigMap.yaml

List the ConfigMaps

kubectl get configMap -n test
NAME               DATA   AGE
testconfig-sn8ya   1      7s

Copy the ConfigMap name and add the ConfigMap name in the deployment.yaml file at the volume level and metadata label level. In metadata level split the ConfigMap name and postfix separately and add that in the label.

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deployment
  labels:
   testconfig: sn8ya
    app: busybox
spec:
  replicas: 1
  revisionHistoryLimit: 1
  strategy: 
    type: RollingUpdate
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      containers:
      - name: busybox
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ['sh', '-c', 'echo Container 1 is Running ; sleep 3600']
        volumeMounts:
        - mountPath: /test
          name: test-config
      volumes:
      - name: test-config
        configMap:
          name: testconfig-sn8ya

Edit the CustomConfigMap and list the ConfigMap. You can see new ConfigMap name with a postfix.

Kubectl edit ccm configtest  -n test
Kubectl list cm -n test
NAME               DATA   AGE
testconfig-10jov   1      10s
testconfig-sn8ya   1      111s

Now check the deployment. You can see that it is updated with new ConfigMap and metadata label.

kubectl get deployment busybox-deployment -n test -o yaml 

apiVersion: apps/v1
kind: Deployment
metadata:
  name: busybox-deployment
  labels:
  testconfig: 10jov
    app: busybox
spec:
  replicas: 1
  revisionHistoryLimit: 1
  strategy: 
    type: RollingUpdate
  selector:
    matchLabels:
      app: busybox
  template:
    metadata:
      labels:
        app: busybox
    spec:
      containers:
      - name: busybox
        image: busybox
        imagePullPolicy: IfNotPresent
        command: ['sh', '-c', 'echo Container 1 is Running ; sleep 3600']
        volumeMounts:
        - mountPath: /test
          name: test-config
      volumes:
      - name: test-config
        configMap:
          name: testconfig-10jov

Give configurator a try and share your feedback with us. If you are interested in contributing to the project, you can reach out to us. The project can be cloned from https://github.com/gopaddle-io/configurator

About The Author

Cloud Native Developer @gopaddle.io

← Back to blog home

Integrating github Actions with gopaddle for a seamless CI/CD on Kubernetes

Posted on February 18, 2021February 18, 2021 | Read time: 7 min. read | DevOps

Continuous Integration (CI) and Rolling Update to a kubernetes service can be achieved seamlessly when using gopaddle Deck and Propeller together. gopaddle Deck does an automatic source to image conversion and uses Kubernetes based build environments to build Docker images. gopaddle Propeller helps to create services and application templates that can be deployed across different kubernetes environments. You can enable Continuous Integration in Deck, such that a Docker build is triggered as soon as a code commit or a code merge happens in github. When Deck and Propeller are used together, a seamless CI/CD can be achieved with a robust version controlling across the kubernetes resources. However, in some scenarios, you may choose to use a different tool/platform in place of Deck to build the docker images and use Propeller deploying and managing the applications. For example, you may choose to use github Actions and then integrate with Propeller for rolling updates. Here is a comparison of github Actions vs Deck and when one can be used instead of the other. Irrespective of the build tool used, gopaddle exposes APIs that can be integrated with any third party tools.

	github Actions	gopaddle Deck
Dockerfile	Must pre-exist in the github repo.	Auto-generate Dockerfile or use an existing Dockerfile
Build Environment	* On-demand Virtual Hosted environments or Pre-provisioned self-hosted environments. Self hosted environments can be physical or virtual machines or containers. * Requires configuration effort to setup self-hosted environments * Runs on VMs or on Containers. Scaling up/down the environments requires requires automation.	* On-demand: Runs inside your kubernetes cluster as isolated build jobs with a specified CPU/Mem capacity. * Requires no additional effort. * Uses shared kubernetes environment and can save cost as the environments scales up/down automatically based on build jobs.
Shared Environments	Can be dedicated for a single repo or shared within/ across organizations	Can be run on a shared kubernetes cluster, but new jobs are created for every build.
OS support for build environments	Linux, Windows and MacOS. For complete list of OS supported check here.	Bring your own OS image. Any Linux Operating system.
Architectures	`x64` – Linux, macOS, Windows. `ARM64` – Linux only. `ARM32` – Linux only.	`Intel x86_64` – Linux only. `AMD64` – Linux only. `ARM64` – Linux only. `Multi-arch` – Linux only.
Subscription	Hosted environments has free minutes beyond which per-minute rates are applied. Self-hosted environments are maintained by end-users on their local infrastructure.	Billed based on build concurrency and total number of builds.

github Actions vs gopaddle Deck

If you choose to use github actions for building the containers, then this blog covers the method of integrating github Actions with gopaddle propeller.

Here are some simple steps to follow:

Create Github Actions
Register Docker Registry in gopaddle
Create an Image based container in gopaddle
Create necessary artifacts in gopaddle and deploy the service on a Kubernetes environment
Generate gopaddle API token
Update github actions to invoke gopaddle API

Let us explore each of these steps in detail.

Trigger Rolling Update directly from github actions

Create Github Actions

As the first step, we will set the github actions to build a docker container. You can use github secrets to store sensitive information like credentials and tokens. Then create a github workflow using those secrets. We will be using Azure Docker Registry to push the docker image. Hence we will use the azure/docker-login@v1 plugin in the workflow.

Let us create the secrets in github repository. Under settings option in github respository, edit Secrets and Add a New respository secret. We will add the Azure credentials in the github secret for the time being. But as we progress through the remaining steps, we will need to add the secrets for gopaddle API token as well. You can follow the steps documented under Azure ACR Registry section here to create an Azure Registry. Here is the list of secrets we need in order to use Azure Registry. You will need these information while registering the Azure Registry in gopaddle as well.
- REGISTRY_LOGIN_SERVER
- REGISTRY_USERNAME
- REGISTRY_PASSWORD

2. Now let us create the .github/workflows folder in the repository. Go to the workflow tab and create the yaml file to build your source code. Your github actions task would look like this to build and push the image to the docker registry. petclinic is the name of the docker image and we will use the github commit ID as the docker image tag. By using the commit ID as the tag, we can relate the docker image to the commit that triggered the build for future debugging.

    - name: 'Build and push image'
      uses: azure/docker-login@v1
      with:
        login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
        username: ${{ secrets.REGISTRY_USERNAME }}
        password: ${{ secrets.REGISTRY_PASSWORD }}
    - run: |
            docker build . -t ${{ secrets.REGISTRY_LOGIN_SERVER }}/petclinic:${{ github.sha }}
            docker push ${{ secrets.REGISTRY_LOGIN_SERVER }}/petclinic:${{ github.sha }}

3. Execute github actions once, so that a docker image is created and pushed to docker registry. We will be using this image in the next steps.

Register Docker Registry in gopaddle

We will be using the image created in earlier step to deploy an application in gopaddle. Once an application is deployed, we can perform further rolling updates on that application. We can automate this initial deployment process as well using gopaddle APIs. But for the time being, we will manually deploy an application using gopaddle UI. To create the gopaddle artifacts, Docker Registry has to be registered with gopaddle, so that gopaddle knows the source from which a docker image can be pulled and deployed.

4. Register the Azure Registry in gopaddle as described here.

Create necessary artifacts in gopaddle and deploy the service on a Kubernetes environment

5. We need to create 3 resources in gopaddle.

(a) Container : Create an image based container in gopaddle using the image created in the earlier step as described here. Once the container is created, note down the container ID by viewing the container and extracting the ID from the browser URL.

(b) Service : Create a Service and add the container to the service as described here. Once the service is created, note down the service ID by viewing the service and extracting the ID from the browser URL.

6. Create a Kubernetes cluster as described here and deploy the template on the Kubernetes cluster to create a running application.

7. Once the application is launched, view the application and gather the application ID from the browser URL.

Using the container ID, service ID and the application ID, we can trigger the rolling update on the specific container within an application as soon as the build is complete.

Generate gopaddle API token

gopaddle APIs can be securely invoked by using an API token. A user can generate one or more API tokens. A role defines the permissions for a user. A role can be assigned to a user to restrict actions within gopaddle. Let us first create a role, a user and then assign the role to the user. Once the user is created, we will generate an API token.

8. In the gopaddle portal, under user profile, navigate to Teams -> Access Control Lists and create a new role with access to All Services and All Permissions for the time being.

gopaddle – create a role with All Permissions

12. Under the Users tab, create a user and assign the role.

13. Click on User actions and Add a token

Update github actions to invoke gopaddle API

Now we have the necessary details to invoke gopaddle APIs.

Container ID
Service ID
Application ID
API Token

14. Add this API token in the github secrets.

15. Update the github actions code to invoke gopaddle API as below.

Your github actions YAML would look like this: We will be using fjogeleit/http-request-action@master github plugin to trigger an API call.

# This workflow will build a Java project with Maven
# For more information see: https://help.github.com/actions/language-and-framework-guides/building-and-testing-java-with-maven

name: Java CI with Maven

on:
  push:
    branches: [main]
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Set up JDK 1.8
        uses: actions/setup-java@v1
        with:
          java-version: 1.8
      - name: Build with Maven
        run: mvn -B package --file pom.xml
      - name: "Build and push image"
        uses: azure/docker-login@v1
        with:
          login-server: ${{ secrets.REGISTRY_LOGIN_SERVER }}
          username: ${{ secrets.REGISTRY_USERNAME }}
          password: ${{ secrets.REGISTRY_PASSWORD }}
      - run: |
          docker build . -t ${{ secrets.REGISTRY_LOGIN_SERVER }}/petclinic:${{ github.sha }}
          docker push ${{ secrets.REGISTRY_LOGIN_SERVER }}/petclinic:${{ github.sha }}
      - run: echo "DESC=$(echo -n ${{ github.event.head_commit.message }} | base64)" >> $GITHUB_ENV
      - name: Deploy Stage
        uses: fjogeleit/http-request-action@master
        with:
          url: "https://portal.gopaddle.io/gateway/v1/application/appsa8218974c6043c4fd2cbd9ac6a71035fcbcf"
          method: "PUT"
          bearerToken: ${{ secrets.BEARER_TOKEN }}
          data: '{"updateType":"buildUpdate","deploymentTemplateVersion":"draft","serviceGroups":[{"name":"spring-petclinic","id":"sg63e2f795sg62e6sg4cd7sgbaefsg4e572ff17787","version":"draft","services":[{"id":"svcea1ce6dcdccaccc4c64cabbec84f5b0f495ee","serviceVersion":"draft","releaseConfig":{"image":"${{ secrets.REGISTRY_LOGIN_SERVER }}/petclinic:${{ github.sha }}"}}],"description":"${{ env.DESC }}"}]}'

Note that, in the above YAML, BEARER_TOKEN is the API token, the URL contains the application ID, id under services array is the container ID, id under serviceGroups is the service ID. Also note that the commit description is encoded in base64 format and then used as the description in rolling update.

CI/CD pipeline is now ready.

Let us now commit a few changes to the github code and this triggers the workflow.

In the gopaddle portal, you can check the version history under the application. We can see that a new rolling update is triggered.

We can also extend this pipeline by triggering a Jenkins pipeline instead of triggering gopaddle APIs, so that a few integration tests can be run before performing a rolling update. We can use the payload described in the YAML file to trigger gopaddle API from Jenkins pipeline as well. Hope you find this blog useful.

About The Author

← Back to blog home

Announcing gopaddle v4.0 – No Code Cloud Native Platform

Posted on January 28, 2021April 1, 2021 | Read time: 3 min. read | Announcements

This new year brings a great relief to the Cloud Native Developers. Today we are pleased to announce gopaddle v4.0 release. This is a major release from gopaddle as we have extended the platform to address the developer needs right from the microservice transition to maintenance, in a simple, yet powerful fashion.

Challenges today

Refactoring Applications: Most often the barrier to modernization is refactoring the existing infrastructure oriented and monolithic application to Cloud Native architecture. Every application is unique and the team dynamics in every organizaton is unique. Thus the strategy to refactor the application has to be crafted to suite the application and the organization.
Cultural Shift : Once the strategy is streamlined, developers need to acquire new skills to build & package their modules as Containers & deploy their modules on Kubernetes.
Maintaining YAML files & Separation of Concern : One of the biggest challenges in modernization today is to write and maintain thousands of lines of YAML code.When it comes to maintaining these YAML files, GitOps is fairly a popular methodology to centrally manage these Kubernetes YAML files, but is inadequate for large distributed teams where DevOps responsibilities are shared across the teams.
Dependency Management : When the number services increase, and when the deployments need to be maintained for a longer period of time, dependency management and change tracking becomes even more critical. Making a change to a service can result in a ripple effect and can lead to application downtime.
Service/Cluster Sprawl : Similarly, when the number of services and environments increase, it can lead to service or a cluster sprawl.

How gopaddle helps ?

gopaddle is a Do-It-Yourself (DIY) platform to build, deploy and maintain Cloud Native Applications. With this new release, we have introduced tools and features that not only reduce the barriers to entry, but also ease the Day-2 operation and maintenance.

Sub-Products

Sail: Migration Service (Alpha Version)
1. Migrate Linux Processes to Docker Containers
2. Available as Open Source https://github.com/gopaddle-io/sail
Configurator : ConfigMap Management Controller
1. Version Control ConfigMaps
2. Keep ConfigMaps and Secrets in Sync with Deployments and Stateful Sets for rollouts and rollbacks
3. Available as Open Source https://github.com/gopaddle-io/configurator
Deck : Container Build Service
1. Use Kubernetes environments to build Docker Images
2. Convert Source Code to Docker Images without writing Dockerfiles
Propeller : Kubernetes Deployment Service
1. No Code/YAML Free Service to build and deploy complex Kubernetes Deployments
2. Create reusable Application templates
3. Centrally Manage Applications across different Kubernetes Environments, ideal for Edge deployments
4. Simplifies Day 2 Operations with the production grade capabilities built in to the system – like Ingress, LoadBalancer, Domain & TLS support, Network Access control , Logging, Monitoring, Alerting etc.
Gear : Multi/Hybrid-Cloud Kubernetes Management Service
1. Provision managed Kubernetes environments across popular Cloud Platforms like AWS, Google & Azure
2. Use pre-existing on-premise Kubernetes Clusters for build and deployment

Capabilities & Use cases

With this new release, you can perform :

Speed up the moderization effort as the Docker and Kubernetes artifacts are automatically generated.
Automate end to end Build and Release automation including CI/CD automation
Multi-Cloud and Edge management from a central dashboard.
Reduce the risk of misconfiguration and/or overlooking YAML changes with the help of granular version control for Kubernetes artifacts.

We are committed to making this platform Developer friendly and to removing any frictions or obstacles in modernizing the applications.

Apart from the product capabilities, we offer Cloud Native Transformation Consulting to analyse your application architecture and suggest a transformation strategy.

Do you want to schedule a call with us to see if gopaddle works for your application stack ? Contact us today !

About The Author

Founder & CEO @ gopaddleio. Talks about distributed systems and composite architectures.

← Back to blog home

Application Log Monitoring using Fluentd as a Daemon Set in gopaddle

Posted on August 17, 2020August 18, 2020 | Read time: 3 min. read | DevOps

Introduction

In order to comply with the Payment Card Industry Data Security Standard (PCI DSS), application access, transaction and system logs need to be monitored. Fluentd is an open source, distributed logging and aggregation framework. It offers a unified logging layer to collate & filter logs of different nature like access logs, application logs and system logs. Fluentd also helps to aggregate, analyse and archive these logs. Kubernetes uses the glog library module for logging Fluentd for application monitoring.

For application (microservice) log monitoring, a sidecar container is suggested pattern for deploying Fluentd within the same pod where the application lives. A shared volume between the application container and the Fluentd container holds the log files for the containers. Fluentd uses the shared volume as the data input and the Google Cloud Logging Service as the data output, provided the cluster is on Google Compute and the VM has cloud-logging.write scope.

The Daemon Set feature, which went GA in Kubernetes 1.2, allows you to deploy Fluentd as a pod across all the nodes in your cluster, instead of deploying them specific to an application.

Technical Implementation

Daemon Set is an extended feature available in Kubernetes 1.2 beta. Some of the typical use cases of Daemon Sets are log aggregation, monitoring services that need to be run on a selected set of nodes within a cluster. Daemon Sets are headless services and are very different from replication controllers. Replication controllers are stateless services that can scaled up or down and can be scheduled across different nodes, whereas Daemon Sets help in performing specific tasks within selected nodes.

Benefits of Daemon Sets

Daemon Sets can be deployed the same way as deploying an application on Kubernetes
Daemon Sets can be deployed on a selected set of nodes with specific tags. Whenever a new node is added with a similar tag, Daemons get deployed automatically.
Daemon Sets with different resource requirements like CPU, memory & storage can be grouped and deployed on selected nodes
Running Daemon Sets as separate PODs & containers within a node gives better resource isolation rather than running them as simple daemons within the node

Fluentd Use Case

Let us consider a use case with a 2-tier application deployment using goPaddle. The application design composes of ngnix and tomcat services. Using goPaddle, each of these services can be deployed as scalable micro-services using POD per container architecture shown below. Each service will have its own replication controller and can scale independently.
fluentd

The services are deployed across a 3 node Kubernetes 1.2 cluster.

Fluentd is available as an add-on component in goPaddle designer. It can be configured to accept log folder(s) and write logs to an external MongoDB service. Fluentd can be created as a single service design and can be deployed the same way as any other service. At the time of deployment, goPaddle provides an option to select the nodes where it needs to be deployed.

In all the selected nodes, Fluentd gets deployed as a Daemon Set and gets configured automatically to collect the logs from the specified folders. goPaddle deploys the services and the daemons using the Sidecar Pods pattern (similar to Sidecar containers pattern) where the pods share the volumes for log read/write.

Best practices

As a best practice, if the application loggers are configured to prepend the host IPs, then the aggregated logs would be prepended with the Pod IPs which can later be used to filter the logs from specific service instance.

Benefits:

By deploying Fluentd as a Daemon Set using goPaddle,

It can be customized to monitor specific or all applications running within the same node
It can collect logs from specific log files across all applications within the same node
It can collect system logs across all the containers within the same node
Since they are run as a separate pod, they run outside the application scaling rules
Use custom log data outputs other than Google Cloud Logging
Since it is run as a Daemon Set, it is ensured that is always in running state

Note:
Using goPaddle, it is also possible to deploy a Daemon Set in ambassador pod pattern (similar to ambassador container pattern) where the application Pod can communicate with the Daemon Set pod using localhost reference. A typical use case would be to deploy etcd services within a Pod and the applications can access etcd for key/value store and retrieval.

About The Author

Founder & CEO @ gopaddleio. Talks about distributed systems and composite architectures.

	Azure DevOps (without gopaddle)	Azure DevOps (with gopaddle)
Releases and Distributions	Supports multi-branch triggers with validation.Multi-strategy to build for different OS – supports Linux, MacOS and Windows	gopaddle supports multi-branch, parallel builds across multiple releases and distributions.Customized build strategy for individual release and distribution.
Build Environments	Agent Pools – Microsoft-hosted agents and Self-hosted agents are supported. Agent pools can be Linux, MacOS or Window VMs or Linux/Windows Containers	On-demand: Builds run inside your Kubernetes cluster as isolated build jobs with a specified CPU/Mem capacity.Requires no additional effort. Standardize the environments across build and deployment workflows.Uses a shared Kubernetes environment and can save cost as the environment scales up/down automatically based on build jobs.
Registries	Azure Container Registry, Google Container Registry (via Google Service Accounts), Docker Hub (username/password-based authentication) are supported. For artifacts other than container images, Azure Artifacts or a NuGet repository can be used.	Get additional support for a variety of Docker registries – gopaddle supports Azure Container Registry, AWS Elastic Container Registry, Google Container Registry, Docker Hub private registry, On-premise Container Registry like GitHub Registry.
Source Control Repos	Azure Repo/TFS, Bitbucket cloud, GitHub cloud, GitHub Enterprise, External Git repositories, and Subversion. Check the service connection types here – https://docs.microsoft.com/en-us/azure/devops/pipelines/library/service-endpoints?view=azure-devops&tabs=yaml	Get additional support for a variety of source control repositories – gopaddle supports GitHub Cloud & On-prem, GitLab Cloud & On-prem, BitBucket Cloud
Multi-Arch Build	Multi-Arch build requires Linux agents along with a QEMU emulator.	Get support multi-arch builds natively. gopaddle supports – Intel x86_64 – Linux onlyAMD64 – Linux onlyARM64 – Linux onlyMulti-arch – Linux only
Deploy to K8s	Has integrations with Azure Kubernetes Service. Support for other Kubernetes services through service connections (using username/password or token-based authentication) along with scripts to deploy to Kubernetes clusters using cloud-specific APIs or kubectl or kube APIs. In either scenario, it requires scripting to deploy applications to Kubernetes.	Works seamlessly with Azure AKS, Google GKE, AWS EKS, HPE Ezmeral Runtime Enterprise, Huawei Kubernetes Service, MicroK8s, and any CNCF compatible Kubernetes environment.

Category: DevOps

Using Azure DevOps Pipeline With Gopaddle for No-Code Kubernetes Deployments

Introducing gopaddle

No-Code and Multi-Cloud Capabilities with Azure DevOps

Workflow

Creating Azure DevOps Pipeline

A Closer Look at the Pipeline Script

Demo

About The Author

Release notes gopaddle v4.1

gpctl import

gpctl init (Alpha Version)

Huawei Cloud Support

Shared Persistent Volumes

Dockerfile Custom Path and Build Arguments

Version Control ConfigMaps and Secrets

Custom Ingress

Azure Autoscaling

Node and Service Affinity Rules

About The Author

About The Author

Integrating github Actions with gopaddle for a seamless CI/CD on Kubernetes

Register Docker Registry in gopaddle

Create necessary artifacts in gopaddle and deploy the service on a Kubernetes environment

Generate gopaddle API token

Update github actions to invoke gopaddle API

About The Author

Announcing gopaddle v4.0 – No Code Cloud Native Platform

Challenges today

How gopaddle helps ?

Sub-Products

Capabilities & Use cases

About The Author

About The Author

Ready To Modernize Applications The Easy Way ?