Files
soleprint/rig/BOOTSTRAP.md
2026-09-17 01:12:15 -03:00

286 lines
12 KiB
Markdown
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# From a machine with nothing on it to a project you can work in
The README says the prerequisite is Docker and nothing else. This is what that
actually looks like end to end: a bare Linux box, and a new project running under
Tilt at the end of it.
A copy of this directory is a sibling of it, named after the environment it
models (`acme-rig`). Paths below are relative to the parent checkout.
It spans three repos because the work does. **rig** prepares the machine — the
pinned toolchain, the cluster, the port arithmetic. **all** owns the shape a
project takes, in `all/projects/templates/conventions.md` and the `broad`
scaffold beside it. **ppl** owns everything after local, and is where this
document stops.
Read it once before running anything. Three of the steps below need root and one
needs a logout, so knowing about them in advance is cheaper than meeting them
halfway through.
## Docker, and the two sysctls Tilt depends on
rig installs a toolchain; it does not install Docker. That line is not modesty —
Docker is a daemon, a group membership and usually a logout, and a script that
did it would have to be trusted with root on a machine it knows nothing about.
```bash
sudo apt-get install -y docker.io && sudo usermod -aG docker "$USER"
```
Then log out and back in, and check `docker info` answers. Until it does, nothing
below works and everything below reports the same failure.
While you have root, raise the inotify limits:
```bash
echo -e 'fs.inotify.max_user_watches=524288\nfs.inotify.max_user_instances=512' \
| sudo tee /etc/sysctl.d/99-rig.conf
sudo sysctl --system
```
kind and Tilt both watch large trees, and WSL ships 8192 watches and 128
instances — far too low. The failure mode is the reason this is here at step
zero rather than mentioned later: Tilt does not error, it simply stops noticing
that files changed, and you lose an afternoon to a hot reload that silently
isn't.
## Read the docs before installing anything
```bash
cd rig
make docs
```
`ctrl/docs.sh` runs a throwaway `nginx:alpine` over a read-only bind mount of
`docs/` and prints the URL. That is deliberate: the docs are the instructions for
building everything else, so they cannot live in the cluster and cannot need
`python3 -m http.server` either — a minimal Debian has no python. What it has,
by definition, is Docker.
The port is this environment's `HTTP_PORT + 4`. Nothing is installed and nothing
persists; ctrl-c ends it.
## Ask what is wrong with this machine
```bash
make check
cp ctrl/.env.example ctrl/.env
```
`check.sh` reports and instructs, and fixes nothing. It runs bare rather than
in a container because host detection only ever reads `/proc` and `/etc` — no
dependency beyond coreutils.
Read the whole output, but the `ports` block is the one to read carefully. Every
port rig binds derives from this directory's name, so the answer is specific to
this copy, and a clash here surfaces as an opaque `failed to bind host port` in
the middle of cluster creation if you skip it.
Copy the `.env` even though the check only warns about it. It is gitignored, it is
where a machine-local override goes, and `ports.sh persist` expects it to exist.
## Install the toolchain — through the container
This is the step where "nothing installed" stops being rhetorical.
`make deps` runs `ctrl/deps.sh install` directly on the host, and the installer
fetches with `curl`. A stock `debian:trixie-slim` has no curl — detection runs
fine, then the first download dies with `curl: command not found` and an exit
code of 127. That is the bootstrap paradox `ctrl/Dockerfile.deps` exists
to kill — the installer carries its own toolchain so the host needs only Docker —
but building the image and running it are two different things, and only the
build has a Makefile target today. **On a genuinely bare machine, run it by
hand:**
```bash
make deps image # builds rig-deps:deps
mkdir -p ~/.local/bin
docker run --rm \
-v /:/host:ro \
-v /var/run/docker.sock:/var/run/docker.sock \
-v "$HOME/.local/bin:/out/bin" \
-e HOST_UID="$(id -u)" -e HOST_GID="$(id -g)" \
rig-deps:deps install dev
```
The image name follows the directory, like everything else here: in `rig`
it is `rig-deps`, in a copy called `acme-rig` it is `acme-rig-deps`. The
tag is `deps` (or `full`, below), not `latest`.
None of the four arguments are guessable, so:
- **`/:/host:ro`** — the installer reads the *host's* `/etc/os-release` and
`/etc/wsl.conf`, not the container's. `HOST_ROOT=/host` is already baked into
the image; this is what it points at. Read-only, and it is the only reason
detection inside a container tells you anything about the machine.
- **the docker socket** — how detection reaches the daemon it is reporting on,
and how it counts kind clusters already running.
- **`/out/bin`** — the image's `OUT_BIN`. Whatever you mount here is where the
four binaries land.
- **`HOST_UID` / `HOST_GID`** — the installer runs as root so it can reach that
socket, which means everything it writes into a mounted volume is root-owned
and useless to you. These drive the `chown` back. Omit them and the install
looks like it worked.
`dev` is kubectl, jq, kind and tilt. `core` is kubectl and jq alone — no cluster
tooling — which is the right answer on a managed or corporate-issued machine and
is why the split exists.
Then put them on PATH, which the installer will remind you about because it cannot
edit your shell for you:
```bash
export PATH="$HOME/.local/bin:$PATH" # and add the same line to ~/.bashrc
```
If something else on this machine already provides `kubectl`, the installer says so
by name rather than shadowing it quietly. `OUT_BIN=$PWD/def/bin` installs
somewhere private instead.
**Two variants worth knowing before you need them.** `make deps image full` bakes
every pinned binary into the image at build time (`DEPS_SOURCE=baked`), so
`docker save` gives you the entire installer as one file to carry into an
air-gapped network. And `DEPS_SOURCE=artifactory` with `DEPS_ARTIFACTORY_URL`
pulls from a generic internal repo, which is usually the only thing a locked-down
client allows.
From here on this machine has curl, so **`make deps` is the short form** for
every later run and every later copy of this directory. The container path is
the first-time path.
## Prove the machine before blaming the project
```bash
make check
make cluster up
kubectl get nodes
```
`make check` re-runs every check — host, docker, toolchain, memory, ports — and
changes nothing. Run now, it should end with nothing left to do by hand, and that
is the point: it is the scoreboard, not the installer.
`make cluster up` builds rig's built-in defaults — one node, no addons,
boots fast. You do not need it to develop anything, but you do want to know that
kind, the kubeconfig context and the derived port block work *before* a new
project has any problems of its own to confuse them with. `make cluster down`
when you are done looking.
Before starting a second cluster, and it will not be long:
```bash
make cluster list
```
Available memory, per-cluster usage and each cluster's port block. On a 16 GiB
box four single-node clusters are comfortable and six push into swap, so this is
worth reading before rather than after. `make cluster free <names>` stops
clusters without deleting them; `docker start` brings them back untouched.
## Scaffold the project
The canonical layout is [`all/projects/templates/conventions.md`](../all/projects/templates/conventions.md).
Read it — it is short, opinionated, and exists precisely so nobody
reverse-engineers a layout from whichever repo they happened to open. What
follows is only the mechanical part.
```bash
SLUG=<slug> # short, lowercase, no separators
cp -r ~/wdir/semester/all/projects/templates/broad ~/wdir/semester/"$SLUG"
cd ~/wdir/semester/"$SLUG"
grep -rl '<slug>' ctrl | xargs sed -i "s/<slug>/$SLUG/g"
cp ctrl/k8s/.env.example ctrl/k8s/.env
git init && git add -A && git commit -m "scaffold $SLUG from broad"
```
`<slug>` is the only placeholder and it lives only under `ctrl/` — cluster name,
namespace, ConfigMap name, and the `NAME=` in `kind-up.sh` / `kind-down.sh`. One
sed does all of it.
The slug is the folder name, lowercase and short — `mpr`, `unt`, `nvi`. The
cluster takes that name and the context becomes `kind-<slug>`, derived by the
scaffold's Makefile from the directory, so there is nothing to edit for either.
**Pick the Tilt port deliberately.** `ctrl/k8s/.env.example` ships a value that
is already in use, so copying it unchanged puts two projects on one port:
```bash
grep -h '^TILT_PORT=' ~/wdir/semester/*/ctrl/k8s/.env 2>/dev/null | sort
```
Choose a free one in `1030010399` — the range ALL reserves in
`projects/index.json` under `policy` — avoiding `10350`, which is Tilt's own
default. Currently taken: `nvi` 10330, `unt` 10340, `mpr` 10360, `mlv` 10370,
`eth` 10380, `lng` 10390. This is the Tilt *web UI* port, not a service port;
each project owns its own service ports separately. The scaffold ships it blank
on purpose, so there is nothing to collide with until you choose.
The scaffold's `ctrl/k8s/` is the same shape as every other project here, and it
builds as shipped:
```
kind-config.yaml one node; gateway NodePort 30080 -> hostPort 8080
base/ namespace, configmap, app (Deployment + Service)
overlays/dev/ promotes the app Service to NodePort 30080
```
Check it before `kind` spends minutes on anything — this renders the whole tree
without a cluster and catches a broken patch immediately:
```bash
kubectl kustomize ctrl/k8s/overlays/dev
```
The workload is an nginx placeholder so a fresh copy reaches something that
answers; replace it. Keep `30080` in step between the overlay patch and
`kind-config.yaml`'s `containerPort` — the hostPort is this project's to pick.
Reachability is a plain kind port mapping: no ingress controller and no MetalLB.
Caddy maps `<slug>.local.ar` onto the host port (`~/wdir/semester/ppl/local/Caddyfile`),
with `*.local.ar` resolving to 127.0.0.1 through dnsmasq. That is the whole chain.
**For `ctrl/Tiltfile`, copy rig's** rather than a live project's. rig ships one
that derives its cluster, context, ports and manifest directory from
`ctrl/ports.sh active` instead of hardcoding a slug, and carries a catalogue of
the blocks every project here ends up needing. Copying from `unt` or `nvi` is
what the estate did until now, and it is why the same Tiltfile preamble exists
in six places with the slug typed in by hand five times each.
> **Two things in this document disagree with rig and are not settled.** It
> mandates Tilt ports in `1030010399`, while rig derives a block from the
> directory name at `20000+` so copies cannot collide — a rig-managed project
> takes rig's. And it names `ctrl/k8s/.env.example`, which is the `broad`
> scaffold's layout; rig's is `ctrl/.env.example`. Both are this document
> describing the house scaffold from inside rig's tree.
## Run it
```bash
make kind-up # idempotent create, then selects the context
make tilt-up # context + your assigned port
```
`tilt-up` passes `--context kind-<slug>` every time, which is the point of going
through `make` at all: tilt cannot deploy into whichever cluster you last looked
at.
`make tilt-down` and `make kind-down` close the loop, and `make kind-reset` is
delete-and-recreate for when a cluster wedges.
## Register it
The project exists; now it is findable. Add an entry to
`~/wdir/semester/all/projects/index.json` and write its `projects/<slug>.md` beside the
others. Structured fields in the index, prose in the markdown.
Putting it on the CI server and deploying it is `ppl`'s half, and it starts at
`~/wdir/semester/ppl/ctrl/init-repo.sh` — gitea remote, then Woodpecker. That is a
different document.