Helm is the most widely used application package manager for Kubernetes. It allows describing the structure of application through helm-charts and lets one easily manage the installed applications with simple commands. Since Helm works on the server-client model, it is always recommended to encrypt the connection using TLS keys for security purposes.
Sometimes, these TLS keys need to be rotated as a part of enterprise policy, in case of expiring keys or in the event of a compromised key. Here’s an easy tutorial on how to rotate the Helm TLS keys on a Kubernetes cluster.
Step 1
Generate a set of TLS keys for Helm and Tiller along with a CA for signing the certificates (If required). CA certificates generally have longer validity and don’t necessarily need to be changed unless the CA certificate key was compromised. Here’s a fantastic article on how to generate TLS keys
Step 2
Base64 encode tiller certificate and tiller key certificates along with CA certificate (If it needs to be changed) and remove the new lines. You can easily do this using sed on any Linux machine or VM. Copy the outputs in a file (this will be updated in tiller secret).
cat ca.cert.pem | base64 | sed -z 's/\n//g'
cat tiller.cert.pem | base64 | sed -z 's/\n//g'
cat tiller.key.pem | base64 | sed -z 's/\n//g'
We will update these certificates in Kubernetes secrets, from where tiller reads it to verify the connection. Kubernetes secrets are Base64 encoded and then saved which allows you to provide binary data (certificates etc.) as secret, and also escape any tricky characters such as ” ‘ \ etc.
Step 3 – Update the certificates in tiller Secrets
Update the tiller certificates, keys and ca certificate in the tiller Secrets in cluster.
kubectl edit secret tiller-secret -n tillernamespace (kube-system by default)
- ca.crt – CA certificate ( ca.cert.pem generated in steps above )
- tls.crt – Tiller TLS key ( tiller.cert.pem generated in steps above )
- tls.key – Tiller key ( tiller.key.pem generated in steps above )
Replace the corresponding entries with the base64 encoded keys that were generated in the steps above
Step 4 – Verify the check the connection using new Keys
Use the helm keys to connect to the tiller and verify if you’re able to establish the connection.
helm ls --tiller-namespace=tillernamespace --tls --tls-ca-cert /path-to-key/ca.cert.pem --tls-cert /path-to-key/helm.cert.pem --tls-key /path-to-key/helm.key.pem
If all the steps were followed correctly, helm ls should display the list of all the charts installed in your kubernetes cluster. If the TLS keys were not properly configured, the above command would give you an error.
PS: If you’re using the same CA certificate as old one, you might still be able to connect using your old Helm TLS keys if they have not expired.
