Kubernetes CI/CD Pipelines with Jenkins and ArgoCD

In the modern software industry where developers perform lightning-fast deployments to their production servers, Continuous Integration (CI) and Continuous Deployment (CD) have played pivotal roles in maintaining the pace of innovation and stability. CI/CD is one of the best practices from the DevOps ecosystem which is integrated to automate the build, test, and deployment process of software. Where CI is a process that is responsible for testing code, bug detection, and taking builds which results in early bug detection and a shortened feedback loop for developers. On the other hand, CD is a process that automates the deployment of tested and verified code to target environments, while ensuring seamless and reliable software deployment. The CI/CD pipelines help teams automate repetitive tasks and accelerate the whole process of software development. In this blog, we will be looking at the tools and techniques that can help us execute seamless and reliable CI/CD pipelines. Specifically, we will be discussing how tools like Jenkins and ArgoCD can help us build a reliable CI/CD infrastructure.

Continuous Integration

Continuous Integration (CI) is a DevOps practice that streamlines development by automating code integration, building, and testing. It allows developers to frequently commit changes to a shared repository, triggering automatic builds and tests. This process ensures code quality, and smooth integration, and accelerates software delivery.

What is Jenkins?

Jenkins is an open-source Continuous Integration (CI) server widely used to automate the CI process of the software development journey. It provides a flexible framework for automating various stages of the development lifecycle, which includes builds, running tests, security scans, static code analysis, and the creation of docker images. The complete automation of Jenkins at its core is handled by “Jenkins Pipelines" typically written in Groovy scripts, these pipelines define multi-stage automation workflows for executing defined tasks. To provide flexibility to developers Jenkins provides robust support of plugins, these plugins help developers to integrate external tools for their CI workflows. Moreover, Jenkins also provides and supports various version control systems, build tools, and cloud platforms, making it adaptable to diverse development environments. The web-based interface provides easy access to build statuses, test results, and other important metrics, facilitating transparency and collaboration within multiple teams.

Why Jenkins?

There are several tools available that provide continuous integration capabilities, some of them are GitHub Actions, Circle CI, Gitlab, and Bitbucket Pipelines. Let’s discuss some of the capabilities of Jenkins making it a popular choice among CI/CD tools and whether is it still relevant to use Jenkins for continuous deployment in the modern cloud-native ecosystem. .

Flexibility and Customization

With Jenkins, we get extensive customization options through its vast plugin ecosystem. These Jenkins plugins allow users to tailor their CI/CD operations according to their specific needs and integrate them with various tools and technologies. This flexibility allows teams to create highly customized workflows and adapt Jenkins to different project requirements.

Open-Source and Community-Driven

Being open-source, Jenkins benefits from a large and active community of developers and users. This community contributes to its continuous improvement, bug fixes, and the development of new plugins. The open-source nature also means that organizations can modify Jenkins to suit their needs if required.

Integration Capabilities

Jenkins can integrate with a wide range of development, testing, and deployment tools. Its extensive plugin ecosystem allows it to work seamlessly with version control systems, build tools, testing frameworks, container platforms, cloud services, and more. This broad integration support makes Jenkins a versatile choice for diverse technology stacks. On the other hand, there are frequent zero-day vulnerabilities with these plugins, so users need to make sure they constantly update these plugins.

Declarative Pipelines

Jenkins has a concept of Declarative Pipelines, which allows users to define their CI/CD pipelines using a structured, easy-to-read syntax. This feature simplifies pipeline creation making it more accessible to team members. 

On-Premises Deployment

Unlike many modern CI/CD tools, Jenkins gets deployed on on-premises environments. Giving teams more control over their CI/CD operations. In many cases where organizations have to follow strict security practices to keep their data and build processes within their control, Jenkins being an on-premises tool provides them with the capability they require.

Mature and Battle-Tested

Jenkins has been around since 2011. Its long history means it has been thoroughly tested in various production environments and has proven its reliability and scalability. Back in 2011, cloud-native was not popular enough and Jenkins was primarily used for on-premises development, but now with the advancement in modern technologies, questions arise, Is Jenkins still relevant for continuous deployment (CD) for cloud-native environments?

To deepen your understanding of using Jenkins for CI in Kubernetes environments, we recommend exploring our comprehensive guide: What is Jenkins? A Quick Guide for CI/CD with Jenkins

Continuous Deployment

Continuous Deployment (CD) automatically deploys code changes to various target environments. It enables rapid delivery of features and fixes to users, the automated process reduces human error, and improves production stability. To facilitate reliable deployments tools like ArgoCD implement GitOps principles, using Git repositories as the source of truth for declarative infrastructure and application configurations. This approach ensures consistent, version-controlled deployments, streamlining the CD process and enhancing system reliability.

Why ArgoCD

Argo CD stands out as the popular GitOps tool for managing Kubernetes deployments, particularly for those who seek a user-friendly interface. Argo CD as a declarative tool offers a great user-friendly dashboard that enables users to perform various tasks from application onboarding to deployment in Kubernetes clusters, without the need to access CLI.

Seamless Integration with Kubernetes

Argo CD integrates seamlessly with the ecosystem of Kubernetes by providing support for the Kubernetes deployment using Helm charts, Kustomize applications, JSONNET files, and Plain directory of YAML/JSON manifests. This flexibility caters to various deployment preferences.

Support for Advanced Deployment Strategies

Beyond the basic deployments, Argo CD supports advanced deployment strategies like Blue/Green deployments and Canary deployments, which enable control rollouts and minimize downtime during the release of newer versions of applications to production environments. This makes Argo CD and GitOps a valuable tool for production-critical deployments.

Deployment to Kubernetes

In the above sections of CI with Jenkins, the pipeline was configured to build the docker image and update the deployment.yml file with the latest image for deployment. Now going forward with the deployment we will use ArgoCD a GitOps tool for deployment to the Kubernetes cluster and maintain the history of our deployments to the Git repository. As ArgoCD treats Git repositories as a single source of truth it continuously syncs the changes from the Git repository to the target deployment environment. As ArgoCD also requires a deployment.yml file to execute the deployments to the Kubernetes cluster, we need to provide the deployment.yml file stored in the Git repository.

Let’s configure ArgoCD for CD, for installation of ArgoCD you need a Kubernetes cluster. For experimentation purposes, you can spin a Minikube cluster and proceed with the ArgoCD installation.

Step 1: Setting up the Environment

Spin up a Kubernetes cluster, for starting up you can opt for the Minikube cluster, refer to this documentation for firing up a Minikube cluster.

For ArgoCD installation there are two methods, you can install ArgoCD by Helm chart or by using the Controller and Operator of ArgoCD. For installation through Controller and Operator navigate to  Operatorhub, click Install, and execute the given commands for installation of ArgoCD Controller. Once the Controller is installed, proceed with the installation of the Operator. Once all pods of ArgoCD are up and running the dashboard of ArgoCD can be accessed.

Step 2: Accessing the ArgoCD Dashboard

ArgoCD provides its dashboard where you can configure and manage the deployments to your Kubernetes clusters. Once the pods of ArgoCD are running, you need to change the type of example-argocd-server to access the dashboard from ClusterIP to NodePort. Now, try to access the service through the minikube service example-argocd-serve command. As the command is executed you will be redirected to the dashboard of ArgoCD.

Step 3: ArgoCD Credentials

The ArgoCD dashboard requires a username and password to access and execute the deployments. The initial Username for the ArgoCD dashboard is admin.

The password for the ArgoCD dashboard is stored in one of the secrets created at the time of ArgoCD installation.

• To get the password navigate to the terminal and execute 

  argocd admin initial-password -n argocd

• The password needs to be decoded as it's base64 encoded, To decode the password execute

echo <base64-encoded-password> | base64 -d 

• Paste the output password on the ArgoCD dashboard with the username as admin and you can access the ArgoCD dashboard now.

Step 4: Deploying Application Using ArgoCD

Let’s deploy the application to the Kubernetes cluster using the ArgoCD dashboard.

• On the dashboard navigate to + New App.
• Provide an application name argo-demo.
• Choose the project default.
• Choose the SYNC POLICY as Automatic.

• Under the SOURCE section provide the URL of the application repository in the Repository URL field.
• Under the Revision field choose the branch of Git Repository.
• At the Path, field the Path to the deployment.yml file which we updated through the Jenkins pipeline.
• Under the DESTINATION section, we need to provide the details of where we have to deploy the application. Provide a Cluster URL and the Namespace where the application should be deployed.

Once all required fields are filled, click CREATE. The application will be created and deployed to the target environment.

With this, your application has been deployed to the target Kubernetes cluster and environment. Now ArgoCD will be treating the Git repository as the single source of truth and, as soon as it detects any change to the deployment.yml file from the Git repository, the current state of the application will be automatically updated. In case someone changes the state of the application directly at the Kubernetes cluster, the ArgoCD will bring back the application to the state defined in the Git repository.

You can also automate the entire process by creating an ArgoCD application object and pushing it to the git repository using Jenkins. This will create a continuous deployment flow and streamline the GitOps-based deployments using ArgoCD and Jenkins.

ArgoCD Falling Short for CD 

ArgoCD appears simple and handy for managing GitOps-based deployments to Kubernetes clusters, handling deployments effectively. However, at scale, ArgoCD alone can become cumbersome and tedious. The challenges faced when executing deployments through ArgoCD depend on the chosen architecture. Deployments to Kubernetes clusters through ArgoCD can be done using two main approaches.

Single Instance Architecture

• One ArgoCD instance manages deployments to all Kubernetes clusters.
• Major setback: Scaling becomes difficult as each component of every cluster needs tuning accordingly.

ArgoCD Instance per-cluster Architecture

• Each Kubernetes cluster has its own ArgoCD instance, this instance manages deployments for that specific cluster.
• Challenge: Managing multiple instances of ArgoCD leads to complex context switching.

The challenges for managing deployments at multiple Kubernetes clusters do not end here, some major areas of Kubernetes deployments where ArgoCD falls short are:

Fined-Grained Access Control

ArgoCD lacks the ability to provide granular access for deployments, you will not be able to set any approval workflows for executing critical deployments natively with ArgoCD. An escalated access for execution of deployments to critical servers can cost downtime due to wrong deployment.

Multi-Cluster/Environment Visibility

ArgoCD does a great job of providing the visibility of applications deployed in one environment with its application set. When it comes to multi-cluster workflows, it falls short. You need to manually check the application for a specific environment. Additionally, ArgoCD doesn’t provide any insights into the micro-services upstream and downstream dependencies making it difficult to release packages when there are microservices dependencies.

Advance Rollback Strategies

ArgoCD lacks the capabilities of easy rollback strategies, the rollbacks need to be executed through the CLI again which becomes tedious.  Also, it lacks the support for advanced automated rollback strategies like SLO-based rollbacks.

Configuration Diff

ArgoCD doesn't provide an intuitive, visual way to compare configurations across different environments or versions. This makes it difficult to identify changes between deployments quickly, understand the impact of updates, or troubleshoot configuration-related issues efficiently. There are hacks that can help you track the changes since it treats git as a single source of truth, you can check the diff, but it can be difficult to manage complex application configurations at scale.

Advanced Deployment Strategies

Out of the box, ArgoCD doesn’t support some advanced deployment strategies, like blue-green or canary deployments. While it can be integrated with Argo Rollouts to handle these scenarios, this requires additional configuration and maintenance.

Lack of Security Details

ArgoCD doesn’t provide insights on any security vulnerabilities for the images and microservices deployed.

Resource Intensive

As ArgoCD scales to manage more applications and clusters, it can become resource-intensive. Large-scale environments may require more resources for ArgoCD controllers and the GitOps reconciliation process, potentially slowing down deployments. Since ArgoCD continuously reconciles the desired and actual states of the cluster, which can be resource-heavy in large clusters or for applications with frequent changes.

While ArgoCD excels in GitOps-based deployments for Kubernetes clusters, it falls short in providing a comprehensive deployment experience, especially for complex, multi-cluster environments. ArgoCD lacks areas such as pre and post-deployment operations, multi-cluster and multi-environment visibility, advanced rollback strategies, and integrated continuous deployment CD experiences. In terms of security with ArgoCD, we do not get support for vulnerability scanning of container images.  

To address these challenges and achieve a more comprehensive deployment experience, Devtron joins hands with ArgoCD. As an open-source software delivery and Kubernetes management platform, Devtron offers features like executing pre and post-deployment operations, vulnerability scanning integrated into CD pipelines, visibility across multiple Kubernetes clusters from a single dashboard, advanced deployment and rollback strategies, and debugging capabilities. Moreover, Devtron provides audit logs for the executed deployments and their configurations, where we can compare the configurations across environments and versions of applications.

Conclusion

CI/CD practices have become fundamental to modern software development, they have accelerated the delivery pipelines and seamless team collaboration. Jenkins has emerged as a leader in CI tools, known for its adaptability, wide-ranging plugin ecosystem, and active user community. Complementing Jenkins, ArgoCD has revolutionized Kubernetes deployments by embracing GitOps methodologies, ensuring dependable and uniform application updates across environments.

This combination allows teams to focus on innovation while maintaining production stability. It reduces manual errors, improves deployment frequency, and enhances overall software quality. Moreover, organizations can achieve faster time-to-market and higher customer satisfaction using automated CI/CD with a reliable GitOps approach.

However, it's important to acknowledge that both Jenkins and ArgoCD have limitations. Jenkins can be resource-intensive, requires extensive scripting, and is not a Kubernetes native CI solution. While ArgoCD also has some limitations as we discussed above. Moreover, managing multiple tools can become tedious and error-prone, potentially introducing new challenges in the CI/CD workflow.

As a comprehensive solution to these challenges, platforms like Devtron offer an integrated GitOps approach to CI/CD. Devtron combines the functionalities of various tools into a single, streamlined interface, simplifying the entire process of application deployment and management over Kubernetes. This all-in-one solution can reduce the complexity of tool management, minimize potential errors, and provide a more cohesive experience for development teams.

To learn about CI/CD implementation within Devtron, we recommend the following articles:
Simplifying the Kubernetes CI/CD Pipeline
Kubernetes CI/CD Pipeline with Jenkins and Devtron
Kubernetes CI/CD Pipelines: GitHub Actions with Devtron