Canary Deployment with Flagger and Istio on Devtron

In the world of modern software development and continuous delivery, efficient deployment strategies are paramount to ensure smooth releases and minimize disruptions. Canary deployment has emerged as a popular approach for gradually rolling out new features or updates while reducing the risk of potential features.

In this blog, we will dive into canary deployments and explore how Flagger and Istio can help organizations achieve reliable and controlled releases through Devtron.

We will explore the core components of Flagger, an open-source Kubernetes operator, and its integration with Istio, a powerful service mesh platform. By leveraging Flagger and Istio, organizations can automate the canary release process, ensuring seamless traffic shifting and real-time application health monitoring.

Understanding Flagger and Istio

Flagger is a progressive delivery tool that automates the release process for applications running on Kubernetes. It reduces the risk of introducing a new software version in production by gradually shifting traffic to the new version while measuring metrics and running conformance tests.

We can use Flagger to implement deployment strategies like Canary, A/B testing, Blue/Green Mirroring. Flagger can be used with a service mesh or ingress controller to automate the deployment strategies for traffic routing. Flagger also analyzes the releases, Flagger can query Prometheus, DataDog, CloudWatch, etc, to analyze the new deployment.  

Istio is an open-source service mesh platform that provides a way to control how microservices communicate and share data with each other. It is designed to run on Kubernetes, on-premises, cloud, or virtual machines.

Istio provides the feature of traffic management. Istio’s traffic routing rules lets you easily control the flow of traffic and API calls between services. Istio’s traffic management model relies on the Envoy proxies that are deployed along with your services. All traffic that your mesh services send and receive (data plane traffic) is proxied through Envoy, making it easy to direct and control traffic around your mesh without making any changes to your services.

Pre-requisites for Performing Canary Deployment

Before we kick-start the deployment process, here are few pre-requisites that needs to be taken care of.

Installing Istio and Flagger

To begin the journey towards controlled and reliable releases, we first install the Istio service mesh and enable it in a test namespace using `kubectl`. Additionally, we deploy Prometheus to collect telemetry and application metrics, which Flagger utilizes for analysis.

Furthermore, we deploy a load generator to simulate realistic traffic patterns, enabling Flagger to make informed decisions based on generated data.

The installation process for Istio involves using the default profile, and detailed instructions can be found in the installation guide. Subsequently, we follow the documentation to install Flagger, which includes setting up the flagger controller.

After installing the flagger and Istio controller, we can start using the flagger for Canary releases.

Starting with Canary release

With the Devtron base deployment chart of version 1.1 onwards, we have introduced support for the Canary release with Flagger and Istio. This integration empowers organizations to seamlessly implement canary deployments within the Devtron ecosystem, ensuring controlled and reliable software releases.

By embracing the power of Flagger and Istio in conjunction with Devtron, organizations can elevate their deployment strategies to a new level of efficiency, enabling them to achieve smooth, controlled, and reliable releases while mitigating risks.

To enable canary release with Flagger and Istio in Devtron, simply set the value of flaggerCanary.enabled to true. Flagger will create a custom resource object called CANARY, which automates the release process.

When deploying a newer application version, Flagger gradually shifts traffic to the canary while measuring the success rate of requests and average response duration.

For better control over the canary release, you can configure the analysis values in flaggerCanary.analysis. This allows you to specify the step weight for traffic shifting and the maximum weight until the analysis should continue.

During the canary release process, you can enable load testing to generate traffic, and Flagger will measure the success rate of requests accordingly.

If you need to specify headers, corsPolicy rules, or mention gateways, Flagger will create and add them to the desired virtualService and destinationRules.

Once the basic configuration is complete, we can save the configuration and proceed over to building & deploying the application.

By default, the flagger will generate the following Kubernetes objects:

• deployment/<targetRef.name>-primary
• hpa/<autoscalerRef.name>-primary
• <service.name>.<namespace>.svc.cluster.local selector app=<name>-primary
• <service.name>-primary.<namespace>.svc.cluster.local selector app=<name>-primary
• <service.name>-canary.<namespace>.svc.cluster.local selector app=<name>
• Virtual-service
• destination rules
  

A canary deployment is triggered by changes in any of the following objects:

• Deployment PodSpec (container image, command, ports, env, resources, etc)
• ConfigMaps mounted as volumes or mapped to environment variables
• Secrets mounted as volumes or mapped to environment variables

Flagger detects that the deployment revision changed and starts a new rollout.

The canary analysis runs periodically until it reaches the maximum traffic weight or the number of iterations. On each run, Flagger calls the webhooks, checks the metrics, and if the failed checks threshold is reached, stops the analysis and rolls back the canary.

Automated canary analysis

Flagger implements a control loop that gradually shifts the traffic to canary while measuring the HTTP requests success rate, requests average duration and pod health.

To gain a better understanding of how the canary release works with flagger, please refer to the flagger documentation.

Once the canary object is created, you can initiate the canary release process and enjoy the benefits of controlled and reliable deployments.

Canary release process

Once the canary object is created, Flagger's automated system detects the presence of a new application version and initiates the canary release process.

Canary detects the newer version of the application and starts the canary process for the application by controlling the traffic for the target application.

The canary has created the newer pods and analyzing the request on the newer version

Under the hood, flagger is using the Istio virtualService for shifting traffic

Once the canary release is successful we can see the following events of the canary object

Summary

The Flagger has impeccably automated the canary release process, bringing forth a seamless experience. With the inclusion of flaggerCanary in the devtron base deployment chart support, developers now enjoy remarkable convenience in achieving deployments through the canary strategy. By providing a comprehensive view of all objects and events associated with the application, devtron simplifies the deployment process to an extraordinary degree.