8.4 KiB
From a machine with nothing on it to an environment you can work in
The README says the prerequisite is Docker and nothing else. This is what that actually looks like end to end: a bare Linux box, and an overlay running under Tilt at the end of it.
Read it once before running anything. Three of the steps below need root and one needs a logout, so knowing about them in advance is cheaper than meeting them halfway through.
Docker, and the two sysctls Tilt depends on
rig installs a toolchain; it does not install Docker. That line is not modesty — Docker is a daemon, a group membership and usually a logout, and a script that did it would have to be trusted with root on a machine it knows nothing about.
sudo apt-get install -y docker.io && sudo usermod -aG docker "$USER"
Then log out and back in, and check docker info answers. Until it does, nothing
below works and everything below reports the same failure.
While you have root, raise the inotify limits:
echo -e 'fs.inotify.max_user_watches=524288\nfs.inotify.max_user_instances=512' \
| sudo tee /etc/sysctl.d/99-rig.conf
sudo sysctl --system
kind and Tilt both watch large trees, and WSL ships 8192 watches and 128 instances — far too low. The failure mode is the reason this is here at step zero rather than mentioned later: Tilt does not error, it simply stops noticing that files changed, and you lose an afternoon to a hot reload that silently isn't.
Read the docs before installing anything
cd rig
make docs
ctrl/docs.sh runs a throwaway nginx:alpine over a read-only bind mount of
docs/ and prints the URL. That is deliberate: the docs are the instructions for
building everything else, so they cannot live in the cluster and cannot need
python3 -m http.server either — a minimal Debian has no python. What it has,
by definition, is Docker.
The port is this environment's HTTP_PORT + 4. Nothing is installed and nothing
persists; ctrl-c ends it.
Ask what is wrong with this machine
make check
cp ctrl/.env.example ctrl/.env
check.sh reports and instructs, and fixes nothing. It runs bare rather than
in a container because host detection only ever reads /proc and /etc — no
dependency beyond coreutils.
Read the whole output, but the ports block is the one to read carefully. Every
port rig binds derives from this directory's name, so the answer is specific to
this copy, and a clash here surfaces as an opaque failed to bind host port in
the middle of cluster creation if you skip it.
Copy the .env even though the check only warns about it. It is gitignored, it is
where a machine-local override goes, and ports.sh persist expects it to exist.
Install the toolchain — through the container
This is the step where "nothing installed" stops being rhetorical.
make deps runs ctrl/deps.sh install directly on the host, and the installer
fetches with curl. A stock debian:trixie-slim has no curl — detection runs
fine, then the first download dies with curl: command not found and an exit
code of 127. That is the bootstrap paradox ctrl/Dockerfile.deps exists
to kill — the installer carries its own toolchain so the host needs only Docker —
but building the image and running it are two different things, and only the
build has a Makefile target today. On a genuinely bare machine, run it by
hand:
make deps image # builds rig-deps:deps
mkdir -p ~/.local/bin
docker run --rm \
-v /:/host:ro \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$HOME/.local/bin:/out/bin" \
-e HOST_UID="$(id -u)" -e HOST_GID="$(id -g)" \
rig-deps:deps install dev
The image name follows the directory, like everything else here: in rig
it is rig-deps, in a copy called other-rig it is other-rig-deps. The
tag is deps (or full, below), not latest.
None of the four arguments are guessable, so:
/:/host:ro— the installer reads the host's/etc/os-releaseand/etc/wsl.conf, not the container's.HOST_ROOT=/hostis already baked into the image; this is what it points at. Read-only, and it is the only reason detection inside a container tells you anything about the machine.- the docker socket — how detection reaches the daemon it is reporting on, and how it counts kind clusters already running.
/out/bin— the image'sOUT_BIN. Whatever you mount here is where the four binaries land.HOST_UID/HOST_GID— the installer runs as root so it can reach that socket, which means everything it writes into a mounted volume is root-owned and useless to you. These drive thechownback. Omit them and the install looks like it worked.
dev is kubectl, jq, kind and tilt. core is kubectl and jq alone — no cluster
tooling — which is the right answer on a managed or corporate-issued machine and
is why the split exists.
Then put them on PATH, which the installer will remind you about because it cannot edit your shell for you:
export PATH="$HOME/.local/bin:$PATH" # and add the same line to ~/.bashrc
If something else on this machine already provides kubectl, the installer says so
by name rather than shadowing it quietly. OUT_BIN=$PWD/def/bin installs
somewhere private instead.
Two variants worth knowing before you need them. make deps image full bakes
every pinned binary into the image at build time (DEPS_SOURCE=baked), so
docker save gives you the entire installer as one file to carry into an
air-gapped network. And DEPS_SOURCE=artifactory with DEPS_ARTIFACTORY_URL
pulls from a generic internal repo, which is usually the only thing a locked-down
client allows.
From here on this machine has curl, so make deps is the short form for
every later run and every later copy of this directory. The container path is
the first-time path.
How the installer itself is tested — machine fixtures, clean containers, a
Workspace snapshot, a throwaway WSL distro — is in
docs/notes/installer-testing.md.
Prove the machine before blaming the project
make check
make cluster up
kubectl get nodes
make check re-runs every check — host, docker, toolchain, memory, ports — and
changes nothing. Run now, it should end with nothing left to do by hand, and that
is the point: it is the scoreboard, not the installer.
make cluster up builds rig's built-in defaults — one node, no addons,
boots fast. You do not need it to develop anything, but you do want to know that
kind, the kubeconfig context and the derived port block work before a new
project has any problems of its own to confuse them with. make cluster down
when you are done looking.
Before starting a second cluster, and it will not be long:
make cluster list
Available memory, per-cluster usage and each cluster's port block. On a 16 GiB
box four single-node clusters are comfortable and six push into swap, so this is
worth reading before rather than after. make cluster free <names> stops
clusters without deleting them; docker start brings them back untouched.
Start an overlay
What runs lives outside rig, in an overlay — see
docs/notes/overlay.md. Start from rig's own:
cp -r examples/starter local/myenv # local/ is gitignored by rig
echo 'OVERLAY=local/myenv' >> ctrl/.env
make check # shows the overlay, its cluster and ports
The cluster takes the overlay's folder name and the context becomes
kind-<name>, so there is nothing to edit for either. Replace the two example
components under k8s/base/, and add your images and resources to the
overlay's Tiltfile. Check the manifests before kind spends minutes on anything
— this renders the whole tree without a cluster and catches a broken patch
immediately:
kubectl kustomize local/myenv/k8s/overlays/dev
If the overlay is to be versioned, make local/myenv a repository of its own (rig
never tracks it), or keep it anywhere else and name it by path.
Run it
make kind-up # idempotent create, then selects the context
make tilt-up # context + your assigned port
tilt-up passes --context kind-<name> every time, which is the point of going
through make at all: tilt cannot deploy into whichever cluster you last looked
at.
make tilt-down and make kind-down close the loop, and make kind-reset is
delete-and-recreate for when a cluster wedges.