Switch microk8s to k3s
I recently switched my HP Proliant Microserver from mircok8s to k3s.
Why did I do it
First of all I’m always interested into new stuff and somehow I found k3s mentioned on twitter and it sounded quite interesting. Second the performance of microk8s on my server was not that good e.g. sometimes when trying to access logs I got timeouts. So when I read about k3s and that it is designed to even run on Raspberries etc. that sounded like a good choice.
How did I do it
At the beginnging I just followed the instructions on https://dev.to/niveditacoder/steps-for-installing-k3s-on-ubuntu-2576. It’s basically the standard way which is also mentioned on https://rancher.com/docs/k3s/latest/en/installation/install-options/. It just shows in addition how to be able to connect to the new cluster.
1
2
3
4
curl -sfL https://get.k3s.io | sh -
mkdir -p $HOME/.kube
sudo cp /etc/rancher/k3s/k3s.* $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
That’s it. Cluster is up an running. I executed my helmfile and most of the stuff was just working fine. But I had a few small issues:
-
k3s uses flannel and traefik both are maybe a good choice but I’m not familar with them
-
I have a daemonset which allows me to access my nextcloud instance via HostIP has on mobile phones (regardless if Android or Iphone) it’s not easy to manipulate /etc/hosts. But the hostPort wasn’t mapped with flannel and I didn’t find out how to solve it.
Traefik I already deleted directly after the fist installation and switched to nginx ingress. As my installation was already based on it. So with the issues I had I also decided to switch flannel with Calico. The same as when setting up Calico on AWS EKS I found the documentation for Calico pretty good: https://projectcalico.docs.tigera.io/getting-started/kubernetes/k3s/quickstart
Note
|
You must uninstall (guess at least you need to restart it) k3s before doing so (as always when switching CNI as that manipulates the whole IP stack of the nodes). |
1
2
3
4
5
6
curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--flannel-backend=none --cluster-cidr=192.168.0.0/16 --disable-network-policy --disable=traefik" sh -
kubectl create -f https://projectcalico.docs.tigera.io/manifests/tigera-operator.yaml
kubectl create -f https://projectcalico.docs.tigera.io/manifests/custom-resources.yaml
mkdir -p $HOME/.kube
sudo cp /etc/rancher/k3s/k3s.* $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
I connected to the cluster, but the svclb-ingress-nginx-controller
didn’t came up. I just asked Google and it came out that doesn’t allow container ip forwarding which is necessary to run the klipper-lb
. So I hat to manipulate the custom-resource.yaml
applied above:
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# This section includes base Calico installation configuration.
# For more information, see: https://projectcalico.docs.tigera.io/v3.22/reference/installation/api#operator.tigera.io/v1.Installation
apiVersion: operator.tigera.io/v1
kind: Installation
metadata:
name: default
spec:
# Configures Calico networking.
calicoNetwork:
containerIPForwarding: Enabled
# Note: The ipPools section cannot be modified post-install.
ipPools:
- blockSize: 26
cidr: 192.168.3.0/20
encapsulation: VXLANCrossSubnet
natOutgoing: Enabled
nodeSelector: all()
---
# This section configures the Calico API server.
# For more information, see: https://projectcalico.docs.tigera.io/v3.22/reference/installation/api#operator.tigera.io/v1.APIServer
apiVersion: operator.tigera.io/v1
kind: APIServer
metadata:
name: default
spec: {}
I added containerIPForwarding: Enabled
and changed cidr: 192.168.0.0/16
to cidr: 192.168.3.0/20
. I applied this objects and everythings working fine. All mentioned issues are solved and the cluster is running fine.
Conclusion
For my server (and I guess that’s basically true for all small servers) k3s offers a much better performance then microk8s. The installation is pretty easy and if you allready have a more or less 'automated' installation of your applications then migration is done with one-three hours (depending if you have speciall needs or not).