What is Jenkins? A Quick Guide for CI/CD with Jenkins

In today fast paced world of software development, many organizations have been adopting DevOps practices by which they can innovate faster and also maintain the stability of their services. One of the practices of DevOps is continuous integration and continuous deployment (CI/CD) where multiple teams can collaborate to manage the life cycle of services. In these CI/CD pipelines teams can automate the repetitive tasks and accelerate the process at each stage like auto build of code, running test cases, scanning for code vulnerabilities, building docker images, and deploying to target environments.

In this blog post, we will look at how to set up production-grade CI/CD pipelines with Jenkins and understand if Jenkins is the right tool for setting up continuous deployment (CD) pipelines for Kubernetes.

Continuous Integration and Continous Deployment (CI/CD)

Jenkins has been used for setting up CI/CD pipelines for a long time. Originally, Jenkins was developed for on-premises environments, but over time, with an increase in cloud-native adoption, it has continued to be one of the most popular tools used for CI/CD in cloud-native environments. Is it the right tool for continuous deployment in cloud-native? In this blog, we'll further answer the question and set up the CI/CD pipelines, but before moving there, let's take a quick re-cap on continuous integration (CI) and continuous deployment (CD).

Continuous Integration (CI)

Continuous Integration (CI) is the DevOps practice streamlining the development process, enabling developers to collaborate effectively and accelerate software delivery using automation at multiple stages. At the CI phase, multiple developers frequently keep committing changes to the central repository, once the code is committed the build and test stage get triggered automatically. The automated process ensures that best practices are followed, maintains the quality of code, and ensures that the code changes are integrated smoothly with existing code.

Continuous Deployment (CD)

The continuous deployment (CD) stage of the software lifecycle enables rapid and reliable deployment of the software changes to target environments. At the Continuous Deployment stage once the code changes are successfully passed the Continuous Integration phase is automatically or with manual approvals get deployed. The CD allows teams to rapidly deliver features and bug fixes to the end users, reduce the risk of human errors in the deployment process, and maintain the stability of production environments.

What is Jenkins?

Jenkins is an open-source tool from the CNCF landscape for managing continuous integration (CI) in the application development lifecycle. By using Jenkins you automate the stages like build, test, security scans, docker builds, etc. All these stages get triggered as the developer commits the code changes to the central code repository i.e. GitHub, Bitbucket, and GitLab. The core of Jenkins which executes all these automations is the Jenkins pipeline which can be defined in two ways: Jenkins file or Groovy script. We will discuss the Jenkins file and Groovy scripts further in this blog in detail. Jenkins also provides you with a user-friendly interface and robust plugin support, using them you will be able to connect multiple tools with Jenkins.

Jenkins Components

Jenkins consists of several key components that work together to provide a comprehensive solution for automating the build, test, and deployment processes. Let's explore the main components of Jenkins:

Jenkins Controller

Jenkins Controller is the central Jenkins server that manages CI builds for distributed systems. In distributed systems, the Controller acts as the brain, determining which builds are executed on which Jenkins Agent. It also handles crucial tasks such as user authentication, authorization, and build distribution. 

For instance, in a case with four Jenkins Agents and one Controller, the Controller will be deciding which of the four Jenkins Agents will execute the build, the actions taken by Jenkins Controller are based on the provided configurations and Jenkins pipeline.

Jenkins Agent

Jenkins Agent formerly known as Jenkins Slave is a components that execute actions on behalf of the Jenkins Controller. Jenkins Agents take instructions from the Jenkins Controller and execute them accordingly. We can run these Jenkins Agents over local systems, docker containers, and on Kubernetes using containers the only prerequisite for Jenkins Agents is the system should have Java installed on it.

Jenkins Project (Jenkins Jobs)

Jenkins Projects previously known as Jenkins Jobs allow users to execute different tasks. It's a type of automation work that the user wants to get done using Jenkins, in case the user wants to set a Multibranch Pipeline for CI they can opt out of the Multibranch Pipeline project. 

Jenkins Plugins 

Jenkins has a robust extension and plugin system, enabling integration with numerous external tools like SonarQube for code scanning. These are community-developed modules that seamlessly integrate with Jenkins. By leveraging these plugins, developers can effortlessly incorporate external tools into their CI pipelines, enhancing functionality and streamlining workflows.

Jenkins Pipeline

Jenkins Pipelines are declarative workflows for CI processes, allowing developers to define multiple stages for automated CI execution. Each stage performs specific tasks such as code building, security, code scanning, or creating Docker images. Jenkins Pipelines can be defined in two ways:

1. Jenkinsfile: A file containing the declarative Jenkins pipeline, stored in a version control system like GitHub, Bitbucket, or GitLab.
2. Jenkins Dashboard: Directly writing declarative pipelines on the Jenkins dashboard interface.

How Jenkins Work

1. The developer commits the code to the central repository which triggers the Jenkins build.
2. Jenkins Controller fetches the Jenkins file from version control and starts executing the stages accordingly.
3. According to the Jenkins file, the Jenkins Controller launches the Jenkins Agent which will be executing the CI build.
4. Once the Jenkins Agent is started, it fetches the code from a central repository.
5. The code gets built as defined in the Jenkins file.
6. Once the code gets built, for instance, once the .jar file is created the next stage is executed i.e. Docker build. On completion of this Docker build a Docker image is generated which can be pushed on container registries.
7. A stage can be defined to update the deployment of the application to the Kubernetes environment.yaml file with the latest container image.
8. Once the deployment.yaml is updated with the latest version of the container image it's ready for getting deployed on target environments.

How to Create a CI/CD Pipeline with Jenkins

Now, let's get our hands dirty and set up the Jenkins CI/CD pipeline to deploy an application onto Kubernetes

Step 1: Install Jenkins

Before diving into writing our Jenkins pipeline, we need to set up an environment to support our CI/CD process. This setup will include static code analysis with SonarQube and Docker for Jenkins Agent. 

Spin Up a Cloud VM: First, create a virtual machine in your preferred cloud platform. This VM will host our Jenkins server.

Install Java: Once the VM is up and running, install Java on the VM as Java is required for running Jenkins.

Install Jenkins: Once Java is installed, proceed with the installation of Jenkins. Refer to Jenkins's installation documentation.

Install SonarQube: As we will be running a static code analysis over code, install SonarQube on our virtual machine. The SonarQube dashboard can be accessed over http://<virtual machine ip address>:9000

The dashboard can be accessed with default credentials,

username: admin
password: admin

Install Docker: For our application build, we will be using Docker agent, for which we need to install Docker.

Step 2: Configuring Jenkins

Accessing the Jenkins dashboard. Once Jenkins is installed the dashboard can be accessed over port no. 8080. To access the Jenkins dashboard navigate to http://<server ip address>:8080

Enter the admin password which can be found in the Jenkins logs at the time of installation of Jenkins. Either you can navigate to the initialAdminPassword file where the password is stored, which can be accessed by sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Once we enter the password of admin, we need to install some suggested plugins or move by selecting the required plugin installation.

Create an Admin user, Before accessing the actual Jenkins dashboard we need to create a new user with a password and email of our choice (This step can be skipped).

Now we are all set to access our Jenkins dashboard and execute our build pipeline.

Now, let's install some more plugins to execute our Jenkins pipeline smoothly. (e.g., Docker Pipeline, GitHub, SonarQube Scanner)

On Jenkins dashboard navigate to Manage Jenkins > Plugins > Available Plugins, and install the following plugins. Docker Pipeline, Github, SonarQube Scanner.

Note: Post installation of the plugins, restart of the Jenkins server is recommended.

As we will be using external tools with Jenkins we need to provide credentials so that Jenkins can access them or the required files. Here, we will provide credentials for Docker (for Jenkins Agent), GitHub (to access the application code), and SonarQube (to perform static code analysis).

Navigate to Manage Jenkins > Credentials > Add credentials. Choose the scope of credentials, and paste the token/password in place of the password. Set an ID for unique identification of credentials which can be used as an environment variable.

Step 3: Setting Jenkins Pipeline

Jenkins pipeline is a declarative approach to defining continuous integration and delivery processes. These pipelines are generally written in Groovy scripts, these pipelines consist of multiple stages that execute sequentially. Common stages include building the application, performing static code analysis, creating Docker images, and pushing those images to repositories like Docker Hub. The declarative nature of the Jenkins pipeline provides us flexibility and capabilities to integrate multiple tools in a single workflow It also provides us with easily maintainable CI/CD workflows.

Jenkins offers two methods for configuring pipelines: you can either create the pipeline directly using Pipeline script on the Jenkins dashboard or store the pipeline script in a Jenkinsfile within a Git repository i.e. Pipeline script from SCM. For this blog, we will be using the Pipeline script from SCM.

• For configuring the Jenkins pipeline navigate to Dashboard > New Item > Enter an item name > Pipeline > Ok.
• Under Advance Project Options, choose Pipeline as Pipeline script from SCM > Choose SCM > Under Repositories Provide a URL of Git Repository > Credentials if the repository is private > Branches to build > Script Path (provide the path of JenkinsFile from the repository).

Step 4: JenkinsFile

Let’s take a look at each stage of this Jenkins pipeline, and understand it:

Agent Configuration:

agent {
  docker {
    image 'base-docker-image'
    args '--user root -v /var/run/docker.sock:/var/run/docker.sock'
  }
}

A Jenkins agent is an environment that runs build jobs for the Jenkins server. These agents are used to offload the execution of the build job allowing parallel execution, efficient resource use, and scalability. In the above section, we are using Docker as an agent to execute the Jenkins builds, the builds will be executed on Docker containers, and once the build is complete the Docker container will be deleted.

Build and Test Stage:

stage('Build and Test') {
      steps {
        sh 'ls -ltr'
        // build the project and create a JAR file
        sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn clean package'
      }
    }

This stage of the Jenkins pipeline, execute the build for our application which generates a .jar file, and run the test cases.

Static Code Analysis:

stage('Static Code Analysis') {
      environment {
        SONAR_URL = "http://<ip-address-of-server>:9000/"
      }
      steps {
        withCredentials([string(credentialsId: 'sonarqube', variable: 'SONAR_AUTH_TOKEN')]) {
          sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && mvn sonar:sonar -Dsonar.login=$SONAR_AUTH_TOKEN -Dsonar.host.url=${SONAR_URL}'
        }
      }
    }

At this stage, we will perform static code analysis using SonarQube which we have already configured on a virtual machine.

Provide the URL of the SonarQube server at SONAR_URL and provide the credential ID which is already configured on Jenkins.

By performing the static code analysis you can detect the vulnerabilities and bugs from the code and make necessary changes to the code to deploy secure applications.

Build and Push Docker Image:

stage('Build and Push Docker Image') {
  environment {
    DOCKER_IMAGE = "<docker-hub-repository>:${BUILD_NUMBER}"
    REGISTRY_CREDENTIALS = credentials('dockerhub')
  }
  steps {
    script {
        sh 'cd java-maven-sonar-argocd-helm-k8s/spring-boot-app && docker build -t ${DOCKER_IMAGE} .'
        def dockerImage = docker.image("${DOCKER_IMAGE}")
        docker.withRegistry('https://index.docker.io/v1/', "dockerhub") {
            dockerImage.push()
        }
    }
  }
}

At the Build and Push stage, the docker image will be built and pushed to the Docker hub.

For accessing the Docker we need to provide the Docker hub credentials Id from Jenkins and the Docker hub repository name.

Once this stage is successfully executed you will be able to use the application image to deploy in any environment.

Updating the Deployment Template for K8s:

stage('Update Deployment File') {
        environment {
            GIT_REPO_NAME = "<github-repository-name>"
            GIT_USER_NAME = "<github-username>"
        }
        steps {
            withCredentials([string(credentialsId: 'github', variable: 'GITHUB_TOKEN')]) {
                sh '''
                    git config user.email "<github-account-email>"
                    git config user.name "<github-username>"
                    BUILD_NUMBER=${BUILD_NUMBER}
                    sed -i "s|image: .*|image: <docker-hub-repository>:${BUILD_NUMBER}|g" java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml
                    git add java-maven-sonar-argocd-helm-k8s/spring-boot-app-manifests/deployment.yml
                    git commit -m "Update deployment image to version ${BUILD_NUMBER}"
                    git push https://${GITHUB_TOKEN}@github.com/${GIT_USER_NAME}/${GIT_REPO_NAME} HEAD:main
                '''
            }
        }
    }
  }
}

Now, that you have a Docker image of the application, for the deployment of the application to Kubernetes you need to update the deployment.yml file with the latest version of the application image. 

In the above stage, you need to provide the credentials of GitHub and Docker hub, for fetching the latest version of the image and updating it to the deployment.yml file which is stored in the Git repository.

Once the deployment.yml file is updated you can configure ArgoCD for performing the GitOps deployment to Kubernetes.

Step 5: Triggering Jenkins CI pipeline

Now we are ready to trigger the Jenkins pipeline click Build Now on the Jenkins dashboard, which will execute the following steps.

• Creating a Docker Agent, to execute builds in a Docker container.
• Building an Application and generating the .jar file as the application is in Java.
• Performing the static code analysis using Sonarqube.
• Building a Docker Image and pushing the image to the Docker hub.
• The last step will update the deployment.yml file, which is stored in a git repository.

Jenkins Pros and Cons

Recommendation: Don’t Use Jenkins for CD 

Jenkins has primarily designed and developed as a Continuous Integration (CI) tool, which it excels by automating build, test, and integration processes. However, Jenkins has its limitations for Continuous Deployment (CD) operations, preventing it from being a complete CI/CD solution. For setting robust CI/CD pipelines for Kubernetes you can couple up Jenkins with any tool that supports the Continuous Deployment (CD), i.e. Argocd, FluxCD, or Devtron. Now let's take a sneak peek at how you can manage the continuous deployment (CD) flow with Devtron.

Manage your CD Pipelines with Devtron 

Devtron, an open-source project, addresses the limitations of ArgoCD and FluxCD, providing robust support for executing Continuous Deployment (CD) pipelines. It offers multiple advanced capabilities to manage our Kubernetes ecosystem, making it an essential tool for handling CD and post-deployment operations. Some of the key features that Devtron offers when it comes to managing CD operations are:

1. Approval-Based Deployments: Devtron enables us to configure approval-based deployment workflows for critical environments like production, which helps us ensure that production servers are stable.
2. Modern GitOps Practises: With Devtron, you are day-1 GitOps enabled, making sure the best practices are followed along for deploying your application without worrying about setting up and managing tools like ArgoCD. Devtron internally automates the entire process, making it a single pane of glass for you to interact with.
3. Advance Deployment Strategies: With Devtron we can configure and execute advanced deployments like canary, blue-green, and rolling updates.

1. Pre/Post Stage: Devtron offers a wide range of plugins which are can be used to integrate with your existing checklist and tools in the ecosystem such as JIRA ticket validations, GitHub PR updator, Bitbucket runner trigger, customer email notifier, running migrations before deployments, JIRA approval check, and lot more.
2. Easy Configuration Management: Devtron comes with pre-defined configuration for your Kubernetes applications which makes it a lot easier to configure applications and check configuration drifts for your helm releases or microservices deployment.
3. Automated Rollback: Rollback is one of the most crucial aspects of any deployment pipeline. With Devtron you get one-click rollbacks as well as advanced automated SLO-based rollbacks making it a production-ready deployment.
4. Policy and Governance: Devtron allows you to configure policies that can be implemented for your production CI/CD pipelines and deployment configurations making it a comprehensive CD tool. Additionally, with its robust RBAC, you can control the access level of developers and devops, and provide a guardrail for your developers to play around without worrying about the stability of production applications. You also get advanced CEL-based policies that can be implemented at continuous deployment (CD) pipelines.

To leverage Devtron's Continuous Deployment (CD) capabilities alongside your Jenkins CI pipelines explore our in-depth blog on Kubernetes CI/CD Pipeline with Jenkins and Devtron.

The features of Devtron we discussed above are just some of the highlights - Devtron has much more to offer. In addition to its core functionality, Devtron allows you to configure and execute robust CI pipelines directly within the platform. Check out this blog to simplify the CI/CD pipelines for Kubernetes. Alternatively, you can integrate any continuous integration (CI) tools like Jenkins and GitHub Actions. Moreover, Devtron provides visibility across multiple Kubernetes clusters, offering granular insights.

If you have any queries, don't hesitate to connect with us or ask your questions in our Discord Community.

Bhushan Nemade

Bhushan is an OSS Evangelist at Devtron with experience in promoting open-source adoption. He is an expert in DevOps ecosystems who actively contributes to and writes about open-source innovations.