A 'small' vanilla Kubernetes install on NixOS
5 days ago
- #Networking
- #NixOS
- #Kubernetes
- Kubernetes abstracts system administration tasks and automates processes like blue-green deployments.
- The article explores setting up a self-managed, small-but-multi-node Kubernetes installation.
- The setup avoids complexities like Public-Key Infrastructure (PKI), NixOS Kubernetes modules, and Kubernetes distributions like K3s or K0s.
- NixOS is used for the Kubernetes package and systemd parts, despite some configuration interweaving issues.
- The setup includes a QEMU environment for testing, available at a provided link.
- Basic NixOS configuration includes enabling nftables and systemd-networkd for networking.
- Kubernetes version 1.34 is targeted, with careful version upgrade considerations.
- Networking setup involves container subnets, Wireguard for encrypted links, and firewall/NAT rules.
- Hostnames are configured with a domain name (k8s.internal) for Kubernetes services.
- The API server setup includes etcd for storage, encryption configuration, and static token file authentication.
- CoreDNS is configured for DNS resolution, running outside Kubernetes but managed by NixOS.
- Kube-proxy and kubelet are set up for service IP management and container runtime, respectively.
- Testing confirms functionality with kubectl commands and container internet connectivity.
- The article suggests next steps like adding an Ingress controller, storage provisioner, and database management.
- Takeaways include the complexity of Kubernetes for small setups and potential simplifications like SQLite support.