Managing Kubernetes Resources Across Multiple Clusters

Bhushan Nemade -

As we delve deeper into the Kubernetes world, we need to manage multiple clusters and resources deployed in it. That's where things start to get more complex. Using traditional command line utilities like kubectl can become complex and error-prone especially when dealing at scale. Efficiently managing Kubernetes resources is crucial for optimizing performance, ensuring reliability, and debugging the clusters. In this blog, we will take a look at the challenges traditional tools brings in when managing Kubernetes resources across multiple clusters. We will discuss some modern tools and understand what these tools bring to us.

Imagine you are handling 50 Kubernetes clusters. Accessing each cluster through command-line tools like kubectl. One day, you were facing some downtime or some of your services were not working as expected. For troubleshooting, you will be using command-line tools that lack visibility and require complex commands for operations like troubleshooting. These complex commands and lack of visibility make things error-prone, which can escalate the issues further

Issues with Traditional Tools

Command line tools like kubectl are powerful for interacting with the Kubernetes cluster. However, they present several challenges when managing Kubernetes resources across multiple clusters.

Complex Commands

Managing Kubernetes resources through command-line tools like kubectl often involves executing multiple complex commands. These powerful commands can quickly become complex, making them error-prone and challenging to remember. Dealing with a vast array of Kubernetes resources, each with its configurations and requirements, becomes increasingly complex when relying solely on command-line tools. The manual nature of these operations increases the risk of introducing unintended errors or omissions.

Limited Visualization

Tools like kubectl and Kubernetes API are CLI-based, which makes it hard to understand the state of resources across multiple clusters. CLI-based outputs are difficult to understand and can take time to analyze. This can result in delayed and inaccurate decision-making.

Now take a scenario where your organization has 100, 500, or 1000 clusters spinning across multiple regions across the globe. Is it feasible to manage these many clusters through kubectl? To manage them, you need something that can allow you to visualize what's happening. A tool or a platform that will allow you to easily navigate in these clusters without hassle and manage the realm of Kubernetes.

Modern-Era Kubernetes Dashboards

Let's explore the set of Kubernetes dashboards designed for managing Kubernetes and what they have for us when it comes to managing the resources in Kubernetes clusters across multiple clusters.

Some modern-era Kubernetes dashboards, heavily adopted by communities and organizations are:

  • Lens
  • Devtron

Let's understand each one of them in detail.

Lens

A tool designed to manage Kubernetes across multiple clusters. The lens is an Integrated Development Environment (IDE) for Kubernetes, which provides a user-friendly graphical interface, helping the developers and DevOps teams manage their Kubernetes infrastructure. Lens comes as a standalone application, providing you with increased visibility, real-time statistics, and logs of cluster resources.

Cluster management

As we are talking about managing resources across clusters we need to start with managing the cluster itself. Lens provides support for multi-cloud and multiple clusters. That implies you can onboard multiple clusters of various flavors (EKS, GKE, AKS, and OpenShift). Lens provides you with a dashboard where you can execute multiple tasks over your clusters, like onboarding a cluster, editing the configuration, and switching between clusters without complex commands.

Resources management in Lens

Once your cluster is onboarded, Lens provides a comprehensive graphical user interface (GUI) solution to streamline the administration of Kubernetes clusters. With Lens, you can easily:

  • Add clusters by browsing through their kubeconfig files or quickly locate kubeconfig files on your local system.
  • Organize clusters into intuitive workgroups based on how you interact with them.
  • Visualize the state of objects in your clusters, such as pods, deployments, namespaces, network, storage, and even custom resources, enabling you to detect and debug cluster issues effortlessly.

If you are an organization with several clusters that are handled by multiple teams, then you have to use the team version of Lens. Lens does come with a trial version which is free of cost but with limited features and accessibilities. For any organization using Lens, you will have to buy a Pro or Enterprise Licence to work in collaboration with the team.  That implies you can’t handle multiple clusters along with your team members for free by using Lens.

Resource Base Access Control (RBAC)

The lens also equipped you with RBAC. Allowing you to set proper permissions for users to access the resources. In team spaces, Lens provides you with permissions like Owner, Admin, and Member. Where the owner and admin have root permissions and members have permission to read the clusters added in team space. The members can also be assigned Role, ClusterRole, RoleBinging, and ClusterRoleBinding. The RBAC in Lens doesn't come preconfigured; you need to configure it manually. RBAC again is a paid feature in Lens that comes with Pro & Enterprise Licensing.

Devtron

Devtron is an open-source platform, designed to manage your applications on Kubernetes.  It comes with a Kubernetes-native dashboard that helps you build your CI/CD pipelines, achieve GitOps, and have DecSecOps practices, governance, and many other integrations under a single pane of glass. The integrations of Devtron facilitate an efficient way for you to manage your Kubernetes clusters and resources. With features like Kubernetes Resource Browser,  it gives you detailed insights into your Kubernetes clusters, where you can perform multiple operations over clusters and resources deployed in them.

Let's delve deeper into Devtron.

Cluster Management

Devtron provides you with two options for onboarding the Kubernetes cluster. The first is to use the cluster URL and bearer token; another option is to use a kubeconfig file to onboard your already-running cluster. After the addition of clusters, you can navigate to the Resource Browser of Devtron.

  • The Resource Browser allows you to navigate across multiple clusters through its dashboard—no need for complex context switching.
  • With Devtron, you can manage the air-gapped clusters through a proxy or SSH. Another way to manage the resources of the air-gapped cluster is by running Devtron itself over that cluster.
  • Devtron provides Single Sign-On (SSO) to access all your clusters. That means you and your team members can use an already existing single set of credentials to access Devtron and your clusters. Devtron understands that the preferences and needs of every organization can be unique. For that, Devtron offers an arsenal of SSO integrations like Google, GitHub, Microsoft, Gitlab, OIDC, LDAP, and Openshift.

So now, if you are someone from an organization that handles multiple clusters or an individual who needs to manage a single cluster, Devtron gives you full control over your clusters and their resources. So now that we have seen what Devtron offers for the management of clusters, Let's see what operations you can execute with Devtron on multiple cluster resources.

Resource Management in Devtron

Managing Nodes:

Your teams can dig deep into each cluster with the Resource Browser of Devtron. Access each node of the cluster, and they can execute some operations on it. Teams can visualize the status of nodes. Your SRE teams can look at the utilization of resources like CPU and Memory usage. Your teams can carry out operations on nodes like edit YAML, debug, delete, drain, taint, edit configurations, or cordon. All these through UI, no use of kubectl, and complex commands. Devtron also allows you to manage the namespaces of your clusters. You can check all the namespaces that exist in your cluster through the dashboard of Devtron.

Workloads:

Devtron allows you to manage workloads like Cronjob, DeamonSet, Deployment, Job, Pod, ReplicaSet, and StatefulSet. Your teams can manage the workloads of your multiple clusters as per requirements. Let's see an example of it:

As we mentioned above, if your organization has 50 Kubernetes clusters and you are experiencing downtime. In this case, Your teams do not need to struggle with a lack of visibility and complex commands. With Devtron’s dashboard, your teams can now:

  • Check for the manifest of the pod, and if needed, they can edit the live manifest.
  • They can look for pod events.
  • For troubleshooting, teams can check the logs of the pod. If needed, Devtron allows the download of pod logs.
  • Devtron also provides a terminal dedicated to that pod in the Resource Browser.

Similarly, your teams can manage the workloads of multiple clusters.

Configs:

In this section, you can manage the config maps, secrets, and persistence volume for your clusters, where you can look for the manifest of the resources and edit them if required.

Along with all these resources, Devtron allows you to manage resources related to Networking, RBAC, Administration, and other resources.

RBAC

When it comes to security, Devtron follows ‘Zero Trust Security’ which means verifying and validating each user before accessing your Kubernetes environment. For this, Devtron allows you to set an SSO for the authentication of the user, and authorization, Devtron equips you with a fine-grained RBAC. Devtron provides RBAC, which goes down to the level of each resource in each cluster. You can authorize users to access specific resources in a specific cluster. A User Access Management feature of Devtron allows you to manage the access of your Devtron for users.

For example, assume that you are facing issues with your clusters and their resources. Where to troubleshoot you need a support engineer or any other stakeholder who can help you in troubleshooting. You can easily onboard the user using SSO, and along with it, you can set RBAC for the specific user, granting access to faulty clusters and resources.

The User Access Management, from where you can mark the user as ' Active’, ‘Inactive’, or ‘Keep active until', is time-based access (TTL). This feature gives you control over how other qualified users access your Devtron dashboard.

Note: Lens does have a free version known as OpenLens, to see a detailed comparison between Devtron and OpenLens, refer to this blog.

Conclusion

In this blog, we talked about the ways of managing Kubernetes resources across multiple clusters. We have discussed the difficulties of managing these resources using command-line tools like kubectl. Later we have seen how two modern-era tools provide ease managing Kubernetes resources. Devtron, while providing the ease of navigation to multiple clusters and managing all resources through a single dashboard, comes with robust authentication and authorization for Kubernetes. With its SSO and fine-grained RBAC Devtron provides full control over your Kubernetes world.

If you have any queries, don't hesitate to connect with us. Join the lively discussions and shared knowledge in our actively growing Discord Community.