DayOne
Journal
Today I Learned

Day 6 - How to upgrade Kubernetes cluster using kubeadm

In fact, you can find the official guideline to upgrade your k8s cluster from Kubernetes's official documentation here.

Without further a do, just straight to guideline

1 - Update your OS

I'm using debian OS and hence here's the command:

sudo apt-get update

2 - Change package repository

Based on the docs here, you can change the repository by this command:

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.28/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

This command ensure that you will use community-owned repository

Additionally, you can download public signing key (which is not mandatory since it has been downloaded to your cluster already), but just in case:

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.28/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

Finally, you must run apt update

sudo apt-get update

3 - Check the version of kubernetes

To choose specific version you want to upgrade to, you can check the list of version by running this command:

sudo apt-cache madison kubeadm

4 - Upgrade your master node (controlplane) FIRST!

You must make the node unscheduleable, so that no application will be scheduled during upgrade process

To make the node unscheduleable, run this command

kubectl drain node <node-name> --ignore-daemonsets

That command will make sure the node is unscheduleable and also move current pods to other nodes so the current application is still running properly without having a downtime.

Next, tun this command to upgrade latest kubeadm version

sudo apt-mark unhold kubeadm && \
sudo apt-get update && sudo apt-get install -y kubeadm='1.32.0-1.1' && \
sudo apt-mark hold kubeadm

Then you run upgrade plan command:

kubeadm upgrade plan

Make sure that your kubeadm, kubectl and other package that is going to be installed are in the right version, including kubelet. For kubelet, you must upgrade it manually. After that, you run upgrade apply command with the version you get from the list:

kubeadm upgrade apply v1.32.0

Next, upgrade kubelet manually by running this command

sudo apt-mark unhold kubelet kubectl && \
sudo apt-get update && sudo apt-get install -y kubelet='1.32.0-1.1' kubectl='1.32.0-1.1' && \
sudo apt-mark hold kubelet kubectl

Next, reload the kubelet service

sudo systemctl daemon-reload
sudo systemctl restart kubelet

Next, you can verify that the kubelet version is upgraded by looking at the version tab in nodes information

kubectl get nodes

Last but not least, you must make the node is scheduleable again by running this command:

kubectl uncordon <node-name>

5 - Upgrade worker nodes

You must make the node unscheduleable, so that no application will be scheduled during upgrade process

To make the node unscheduleable, run this command

kubectl drain node <node-name> --ignore-daemonsets

To upgrade worker nodes, you don't have to run kubeadm upgrade plan and kubeadm upgrade apply. You simply run this command:

kubeadm upgrade node

Then, continue with the same command to upgrade kubelet in the controlplane node

sudo apt-mark unhold kubelet kubectl && \
sudo apt-get update && sudo apt-get install -y kubelet='1.32.0-1.1' kubectl='1.32.0-1.1' && \
sudo apt-mark hold kubelet kubectl

sudo systemctl daemon-reload
sudo systemctl restart kubelet

Then, uncordon the nodes

kubectl uncordon <node-name>

Happy learning!