Blinkit’s tryst with Kubernetes
Synopsis of the talk presented by Vaidik Kapoor,
Tech Consultant, Fractional CTO/VPE,
Ex-BlinkIt (formally Grofers)
In this microblog, we will talk about the journey of BlinkIt (formerly Grofers) of moving to Kubernetes and important considerations of the same.
The journey started with the philosophy -
You Build it, You Run it
“We moved to Kubernetes around 3.5 years back. Presently, we moved the 90% of the targeted stateless workloads to Kubernetes and the remaining 10% of the stateful workloads are running on EC2 instances.”
With the movement to cloud, there are some significant gains that we had noticed
- Developers now develop in the cloud on top of Kubernetes. We were able to create and destroy on demand dev environments.
Illusion of Agility or the objective for moving to K8s
The journey of transition wasn’t smooth, though. There were certain challenges that we faced during this transition -
During these activities, very soon, we ended up with the proliferation of microservices and our development speed slowed down. Engineers were burning out in managing those microservices. The lessons from this was that before moving to Kubernetes, one should do a quick evaluation of ‘Do they really need Kubernetes and do you need it now?’ Don’t do it just because it’s good and offers many out of the box benefits. Using Kubernetes should match your use case.
During the process, we also launched an initiative called, - The Project ShipIt, March 2019. Now we started containerizing applications, moving towards Kubernetes adoption, but the major challenge was the disparity in dev and prod environments. In production, we were running using some Ansible scripts but in dev environment docker and docker-compose was used. With some time, in late Jan 2019, we moved our prod to Kubernetes and started running all tests and dev environments in Kubernetes.
The major objective to move into Kubernetes was for better dev experience and solving our CI-CD problems.
Change of mindset is important before moving to Kubernetes
If your team is accustomed to working in the Dev & Ops fashion separately, then adopting Kubernetes might not be the best for your organization. It may make things worse.
Managing Kubernetes itself is a hard task because there are lots of moving parts that you need to take care of. Learning Kubernetes requires a steep learning curve. The organization should spend a good amount of time in making the team learn about Kubernetes with training sessions & certifications. It’s not only for DevOps teams & SRE, developers should also learn about Kubernetes in the organization.
Kubernetes standalone is not a PaaS solution. Even after bootstrapping Kubernetes, you need to configure it a lot and make changes as per your requirements. So, before moving to Kubernetes, you have to make a lot of decisions in your head in order to make Kubernetes like a PaaS solution on which you can deploy and manage your containerized workloads. Here is what blinkit used back in the past.
Day 2 operations on top of Kubernetes is hard to set up. You need to set up autoscaling, logging, monitoring etc. on top of Kubernetes so that it can scale your containerized workloads depending on the load and as you get more visibility of your applications in Kubernetes. There can be challenges like pods not scaling properly and high DNS latency issues. There can be networking issues which can be hard to debug and manage, so it’s recommended to go for managed Kubernetes services. Even with managed services, You have to worry about upgrades, rollbacks, cost optimization, security, governance etc.
You can use Kubernetes CRDs, Operators, Controllers, webhooks to enable more of your workloads to be managed declaratively. You can abstract away your day-to-day operations using CRDs.
If you are sure that you want to move to Kubernetes, then doing it all by yourself is not recommended. Rely on external experts who’re working in some particular niche and let them do the heavy lifting for your organization. Consider the hidden cost of spending extra hours in building tools all by yourself rather than having a managed offering.
Use Devtron to manage your pipelines effectively and get deeper insight into your CI-CD, debugging, deployments frequencies and much more. Checkout Devtron today.