Storage in Kubernetes is hard, complicated, and messy. Configuring volumes, mounts, and persistent volumes claims and getting it right can be a challenge. It’s also challenging to manage that storage and replicate it across all your Kubernetes clusters. It’s also been very challenging to do this on bare metal, outside of a cloud provider. That’s where Longhorn comes. Longhorn is an open source, a CNCF distributed block storage system for Kubernetes. It comes with a UI, backups, snapshots, cluster disaster recovery, and it does all this with or without Rancher. Rancher is NOT a requirement.
Installation
Additional Dependencies
There are some additional dependencies you might want to install on target nodes prior to configuring
1
2
3
4
sudo apt update
sudo apt install nfs-common open-iscsi
#start the service now and on reboot
sudo systemctl enable open-iscsi --now
Install Methods
Rancher app catalog
See the app catalog within Rancher
Kubectl
1
kubectl apply -f https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
1
2
3
kubectl get pods \
--namespace longhorn-system \
--watch
See more at https://longhorn.io/docs/1.0.0/deploy/install/install-with-kubectl
Helm
helm3
1
2
kubectl create namespace longhorn-system
helm install longhorn ./longhorn/chart/ --namespace longhorn-system
1
kubectl -n longhorn-system get pod
Taints
This is not required, nor do I taint nodes anymore. I allow Longhorn storage to use any available space on any node that is not running
etcd/ control plane. You can simply skip this step and it will work like this. If you’re still convinced you need dedicated nodes, it’s much easier doing it in the Longhorn UI after a node joins the cluster than with taints.
I ended up tainting my storage nodes using this command
1
2
kubectl taint nodes luna-01 luna-02 luna-03 luna-04 CriticalAddonsOnly=true:NoExecute
kubectl taint nodes luna-01 luna-02 luna-03 luna-04 StorageOnly=true:NoExecute
Then applying that toleration to Longhorn in settings
StorageOnly=true:NoExecute;CriticalAddonsOnly=true:NoExecute
This ensures that the storage nodes won’t take on any general workloads and still allow Lonhorn to use these as storage.
Links
⚙️ See all the hardware I recommend at https://l.technotim.live/gear
🚀 Don’t forget to check out the 🚀Launchpad repo with all of the quick start source files