This checklist includes the setup instructions for a turbolift-compatible pi cluster, and includes additional configuration and tooling (marked with [optional]) that I find useful. It is available as a manual installation checklist, with explanations for each command, as well as in script format.
In short, this setup is a microk8s cluster running on ubuntu with the default microk8s ingress. Turbolift does not rely on any microk8s-specific APIs, and this is not the only turbolift-compatible cluster configuration. This setup requires a 64-bit pi and was built/tested using a pi4 with 4GB of RAM. If you are building a cluster using these instructions, I would recommend going through this installation once and storing the image after completion to be reused without having to re-install for each pi.
sudo apt-get update
sudo apt-get install network-manager -y
(I prefer using nmcli, iwlist etc. to iw directly, though
if you don't have any wired connection, you can activate the wifi card using
sudo ip link set wlan0 up
and then use wpa_supplicant to connect to your network using this example).
nmcli d wifi
to see available networks.nmcli d wifi connect <YOUR SSID HERE> password <YOUR PASSWORD HERE>
nmcli connection modify <YOUR SSID HERE> connection.autoconnect yes
sudo apt-get install bison build-essential golang-go -y
bash <(curl -s -S -L 'https://raw.githubusercontent.com/moovweb/gvm/master/binscripts/gvm-installer' )
bash -c "source ~/.gvm/scripts/gvm
gvm install go1.17
gvm use go1.17 --default
cd ~ && git clone 'https://github.com/derailed/k9s'
cd ~/k9s &&
git checkout e5759218e6cf70767399c465891dba5161f8ffea
go mod tidy
make build
echo 'PATH=$PATH:~/k9s/execs' >> ~/.zshrc
"; ls ~/k9s/execs/k9s || exit 1
curl --proto '=https' --tlsv1.2 -sSf 'https://sh.rustup.rs' | sh -s -- -y --default-toolchain nightly-2020-09-28
echo 'source ~/.cargo/env' >> ~/.zshrc
sudo apt-get update
sudo apt-get install apt-transport-https ca-certificates curl gnupg lsb-release -y
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg
echo "deb [arch=arm64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io -y
echo '{"insecure-registries": ["localhost:32000"]}' | sudo tee /etc/docker/daemon.json
sudo usermod -aG docker $USER
echo 'docker system prune -f' | sudo tee /etc/cron.daily/docker-system-prune
cat /boot/firmware/cmdline.txt | tr '\n' ' '| xargs -i% echo "% cgroup_enable=memory cgroup_memory=1" | sudo tee /boot/firmware/cmdline.txt
sudo reboot
sudo apt-get install iptables-persistent -y
sudo snap install microk8s --classic
sudo usermod -aG microk8s $USER
/usr/bin/newgrp microk8s <<EOT
microk8s start & microk8s status --wait-ready
sudo snap install kubectl --classic
microk8s enable ingress
cd ~/.kube && microk8s config > config
EOT
sudo apt install zsh -y
chsh -s /bin/zsh
and reboot.
cat <<EOT>> ~/.zshrc
HISTFILE="$HOME/.zsh_history"
HISTSIZE=1000000
SAVEHIST=1000000
setopt SHARE_HISTORY
EOT
git clone --depth=1 'https://github.com/romkatv/powerlevel10k.git' ~/powerlevel10k
echo 'source ~/powerlevel10k/powerlevel10k.zsh-theme' >> ~/.zshrc
sudo apt install fzf -y
cat <<EOT>> ~/.zshrc
source /usr/share/doc/fzf/examples/key-bindings.zsh
source /usr/share/doc/fzf/examples/completion.zsh
EOT
sudo dd if=/dev/zero of=/swap0 bs=1K count=10M
sudo chmod 0600 /swap0
sudo mkswap /swap0
sudo swapon /swap0
echo '/swap0 swap swap defaults 0 0' | sudo tee -a /etc/fstab
sudo sysctl vm.swappiness=0
cat <<EOT>> ~/.zshrc
screen
EOT
cat <<EOT>> ~/.screenrc
defscrollback 20000
startup_message off
EOT
sudo reboot
microk8s join
and follow the instructions to start clustering the pis.
The script completes the above instructions, excluding attempting to connect to a wifi network, enabling cgroups in the firmware, changing the user's shell, or running the microk8s join command. The script triggers a reboot on completion.
To run the script, prepare your
freshly imaged pi by enabling cgroups (used by microk8s) and rebooting, using cat /boot/firmware/cmdline.txt | tr '\n' ' '| xargs -i% echo "% cgroup_enable=memory cgroup_memory=1" | sudo tee
/boot/firmware/cmdline.txt
&& sudo
reboot
After logging back in,
you're ready to run the script. It's available on a designated page
and can be applied by running
bash <(curl --proto '=https' --tlsv1.2 -sSf 'https://dominic.computer/code?page=/blog/2021/turbolift_pi_k8s_install&lang=Bash' )
.
Then, run chsh -s /bin/zsh
, exit
and on login you should be done with installation!