Helm is the most widely used application package manager for Kubernetes. It allows describing the structure of application through helm-charts and lets one easily manage the installed applications with simple commands. Since Helm works on the server-client model, it is always recommended to encrypt the connection using TLS keys for security purposes.
Sometimes, these TLS keys need to be rotated as a part of enterprise policy, in case of expiring keys or in the event of a compromised key. Here’s an easy tutorial on how to rotate the Helm TLS keys on a Kubernetes cluster.
Step 1
Generate a set of TLS keys for Helm and Tiller along with a CA for signing the certificates (If required). CA certificates generally have longer validity and don’t necessarily need to be changed unless the CA certificate key was compromised. Here’s a fantastic article on how to generate TLS keys
Step 2
Base64 encode tiller certificate and tiller key certificates along with CA certificate (If it needs to be changed) and remove the new lines. You can easily do this using sed on any Linux machine or VM. Copy the outputs in a file (this will be updated in tiller secret).
cat ca.cert.pem | base64 | sed -z 's/\n//g'
cat tiller.cert.pem | base64 | sed -z 's/\n//g'
cat tiller.key.pem | base64 | sed -z 's/\n//g'
We will update these certificates in Kubernetes secrets, from where tiller reads it to verify the connection. Kubernetes secrets are Base64 encoded and then saved which allows you to provide binary data (certificates etc.) as secret, and also escape any tricky characters such as ” ‘ \ etc.
Step 3 – Update the certificates in tiller Secrets
Update the tiller certificates, keys and ca certificate in the tiller Secrets in cluster.
kubectl edit secret tiller-secret -n tillernamespace (kube-system by default)
- ca.crt – CA certificate ( ca.cert.pem generated in steps above )
- tls.crt – Tiller TLS key ( tiller.cert.pem generated in steps above )
- tls.key – Tiller key ( tiller.key.pem generated in steps above )
Replace the corresponding entries with the base64 encoded keys that were generated in the steps above
Step 4 – Verify the check the connection using new Keys
Use the helm keys to connect to the tiller and verify if you’re able to establish the connection.
helm ls --tiller-namespace=tillernamespace --tls --tls-ca-cert /path-to-key/ca.cert.pem --tls-cert /path-to-key/helm.cert.pem --tls-key /path-to-key/helm.key.pem
If all the steps were followed correctly, helm ls should display the list of all the charts installed in your kubernetes cluster. If the TLS keys were not properly configured, the above command would give you an error.
PS: If you’re using the same CA certificate as old one, you might still be able to connect using your old Helm TLS keys if they have not expired.
FAQ
What is Helm?
Helm is a package manager for Kubernetes, used for managing applications and services with simple commands.
How Do You Generate TLS Keys for Helm and Tiller?
To generate TLS keys for Helm and Tiller, create a certificate authority (CA) to sign the certificates. Then, base64 encode the generated certificates and keys before updating them in Kubernetes secrets for Helm to use.
How Do You Update Tiller Secrets with New TLS Certificates?
After generating new TLS certificates and base64 encoding them, update the Tiller secrets in Kubernetes by editing the tiller-secret in the respective namespace (typically kube-system) and replacing the old certificates with the new ones.
How Can You Verify the TLS Key Rotation in Helm?
To verify the TLS key rotation, use the helm ls command with the new certificates, ensuring a successful connection to Tiller. If properly configured, Helm will list all charts installed in the Kubernetes cluster.
