Using DevOps practices, you no longer need to worry about manually testing and finding bugs in your code, which saves a lot of time and allows developers to focus on writing code. However, one aspect that might get overlooked in a DevOps lifecycle is security, and it might often be overlooked during the entire SDLC and is checked only after the product has been built.
Normal DevOps Workflow
Before we look into why traditional security fails with DevOps, let's look at a normal DevOps lifecycle.
In a normal DevOps workflow, you would have a ton of automation for building, testing, and deploying your applications. Around 60% of your work would be automated and you can focus on writing more code and improving your applications.
Developers will write and push their code to GitHub, then a CI pipeline would build and run unit tests on your code to make sure everything is working as expected. If there are some bugs, you would get notified about them and will have to fix the bugs.
Once all the tests have passed, the pipeline would re-deploying the application with your changes in it and update the existing container image.
Now you can configure and monitor your application once it's on the cloud, look out for any potential bugs, and cases that might cause downtimes, etc.
These practices only apply to the application features. At this point, we don't have any automated security tests for our application. The security aspects are being handled manually by a security team.
Traditional Security Downsides
Many things can cause security issues in your applications which can make them vulnerable to attacks. This is a bigger problem if your application has sensitive information about a lot of users. Below are some examples of security issues:
- Outdated libraries: Using outdated frameworks or libraries can make it easier for hackers to steal sensitive data. To combat this, you should use up-to-date versions of frameworks and ensure that they are being maintained.
- Exposed secrets: Secrets refer to passwords, Auth tokens, SSH keys, and other keys which are used to authenticate your application with third-party services. These keys can get exposed if you accidentally push them to your code repository.
- K8s misconfigurations: Misconfiguring your Kubernetes clusters can also lead to security risks. Some examples of Kubernetes misconfigurations include using wrong namespaces, allowing privilege escalation, giving elevated privileges to containers, and more.
- Container Image vulnerability: Your container images themself could have some kind of vulnerabilities. This can be because of using too many unnecessary dependencies which creates bloated images. This can have both security and performance implications.
- Code Vulnerability: When talking about Security, we certainly cannot exclude the vulnerability that the code might have. These can be some logical errors, using incorrect access modifiers, or using an outdated version of the language.
Security Bottlenecks
As we mentioned above, these processes are not automated and are done by a team of security experts. Since this is a manual task, it can take days or even weeks to run security checks and make sure the application is as secure as possible.
This delays application releases, and since it is a manual process, there is a high chance of human error where some vulnerability is overlooked.
Applications are under greater security risk
Today a lot of organizations run their applications using Kubernetes since it has several useful features. Kubernetes uses a microservice architecture in which your application is made up of hundreds or even more APIs.
On top of these APIs, we use many third-party integrations along with Kubernetes to increase its functionalities. If these integrations are outdated, they can be exploited to gain access to your cluster.
This ultimately increases the attack surface where hackers can find and exploit vulnerabilities in your clusters.
Outdated Security Tools
The cloud-native ecosystem is rapidly evolving, with new tools, technologies, and features being released frequently. Kubernetes is also rapidly evolving.
This ultimately makes it difficult for security tools to keep up with the rapidly evolving ecosystem. To develop security tools, you would need to first find encounter a possible vulnerability, run multiple tests then find out how to secure the vulnerability.
DevSecOps Benefits
This is where DevSecOps comes into the picture. With DevSecOps, security is no longer outside the DevOps scope. Security scannings are included in the CI/CD pipeline which ultimately creates faster and more efficient security scanning.
There are a ton of tools as well which can scan your container images, check for vulnerabilities, and show you exactly what is causing vulnerabilities and potential fixes.
Using DevSecOps also lets you write secure code from the get-go, instead of having to clean it up later which can take a lot of time. This also reduces the overall security risks that are present in a production environment.
How DevSecOps Works
As mentioned above, security scanning is included in your CI/CD pipelines, but there are also a few other steps to take into account. In your pipeline, you would be using a security policy such as the NSA or you could build your custom policy.
We also use an external secret manager. This means that you would no longer be pushing your secrets into your code repository. The secrets will be stored in a secure external environment, which can be accessed in your production and testing environments.
DevSecOps with Devtron
Now that we've understood why you should use DevSecOps, let's take a look at how you can use implement it using Devtron. As of right now, Devtron uses Clair for security scanning.
Setting up Devtron for security scanning
The first thing to do would be to spin up a Kubernetes cluster if you have not already. Now that you have a cluster to work with, use the below command to install Devtron along with its CI/CD features. Alternatively, you could use the stack manager to update it later on
helm repo add devtron https://helm.devtron.ai
helm install devtron devtron/devtron-operator \
--create-namespace --namespace devtroncd \
--set installer.modules={cicd}
Since the CI/CD functionalities install quite a few tools such as Argo, it would take around 10-12 minutes to install completely. So you can go and grab a cup of coffee.
You can use kubectl logs -f -l app=inception -n devtroncd to check the installation status.
Now you want to head over to your Devtron dashboard and log in. To get the dashboard address, you can use this command, or check the service lists from your cloud dashboard
kubectl get svc -n devtroncd devtron-service \
-o jsonpath='{.status.loadBalancer.ingress}'
Use this command to get the password
kubectl -n devtroncd get secret devtron-secret -o jsonpath='{.data.ACD_PASSWORD}' | base64 -d
Now you need to set up your application and implement a CI/CD pipeline. First head to global settings, and set your container registry. Now when you return to the applications page you should see a create button. Click on it and create a custom app.
Give your application a name and select your project. Since we are using a new application, we want to set up a blank app.
You should now see a page like this where you will need to configure your application. Go ahead and configure the Git repository, Docker build config, and the deployment template.
To enable Devtron's inbuilt security scanning features, create a new Continous Integration pipeline and click on advanced options. Then you want to enable the security scanning features under the build stage tab and create your pipeline. After this, you can add deployment to the pipeline.
Now the next step is to select a material and run the security scans. While the tests and scans are running, you can click on details to see everything that is happening. You can also use it to troubleshoot if the scans fail.
You can also configure your security policies. Head over to the security policies, and you can configure which level of vulnerabilities you want to allow. Of course, your critical vulnerabilities should generally be blocked.
Once your scans are completed, you can go to the security section and you can see a list of all your scans. If there are any vulnerabilities your security scan status would show failed. You can click on the scan to see what scans failed and what you can do to fix them.
You can also configure which policies you want to allow or block. You can also add a custom CVE policy. You can browse the CVE list and find the CVE ID which you want to block or allow.
Hierarchical Security Policies
You can also set cluster-specific, environment-specific, or application-specific policies.
Note that these policies work in a hierarchical order. This means the global policies would be checked first, then the cluster, then environment-specific policies, and finally the application-specific policies.
Cluster Policies
Under global policies, you have two options for configuring policies, Allow and Block.
However, for cluster-specific policies, you have three options, Allow, Block and Inherit. Just as it sounds, the inherit options will use the same settings as its upper-level policy.
For example, if you have blocked critical level policies in your global policies it will be blocked in Cluster as well. However, if you want to allow them in the cluster policies, you can select allow in the cluster policies.
Enviornment Policies
Similar to cluster policies, the environment policies also have the three options of Allow, Block, and Inherit.
You can select an environment here which will then inherit the cluster policies.
For example, if you have blocked critical level vulnerabilities in Global Security Policy but allowed them in Cluster Security Policy, then Environment Security Policy will inherit the policy of the cluster not global, Hence critical level vulnerabilities will be allowed in the Environment Security Policy.
Application Policies
The same rules apply to the application policies as well. However, the application policies are a combination of environment policies and application policies.
For example, let us say you have two environments, testing, and production. Your policies for the testing environment are all set to allow, whereas in the production environment all are set to inherit.
If you deploy your application in the testing environment, the application will inherit the policies of the testing environment and not the production environment.
In this way you can include security scanning as a part of your CI pipeline using Devtron. This will allow you to have secure releases and avoid having to do manual security tests which can delay the releases by weeks.
