199 lines
8.2 KiB
Markdown
199 lines
8.2 KiB
Markdown
# rig
|
|
|
|
A runnable local model of a large, regulated estate — legacy and new side by
|
|
side. Its job is onboarding and exploration, not a production replica: most
|
|
services are deliberately mocked, because what has to be faithful is the
|
|
topology, not the workloads.
|
|
|
|
## Prerequisite
|
|
|
|
**Docker.** Nothing else — no curl, no jq, no python, no apt repositories.
|
|
|
|
### Starting from plain Windows
|
|
|
|
Everything here is bash and runs *inside* a Linux shell, so on a Windows machine
|
|
that means WSL. Nothing in rig installs WSL, and nothing will: `wsl --install`
|
|
enables Windows features and requires a reboot, which is not something a script
|
|
should do to a machine on your behalf — and there is no tested undo for it.
|
|
|
|
From an elevated PowerShell or Command Prompt, once:
|
|
|
|
```powershell
|
|
wsl --install
|
|
```
|
|
|
|
Then reboot and open the Linux shell it installed.
|
|
|
|
**If you cloned this on the Windows side, copy it into WSL before carrying on.**
|
|
WSL can reach the Windows drives at `/mnt/c`, and working from there mostly
|
|
functions — slowly — but file watching does not: that filesystem raises no
|
|
inotify events, so anything watching for edits silently stops seeing them.
|
|
|
|
```bash
|
|
cp -r /mnt/c/Users/<you>/rig ~/rig
|
|
cd ~/rig
|
|
```
|
|
|
|
`make deps` reports it if you are running from `/mnt/...`. Then carry on below.
|
|
|
|
If it fails, the usual causes give unhelpful messages:
|
|
|
|
| symptom | cause |
|
|
| --- | --- |
|
|
| "the virtual machine could not be started" | virtualization disabled in BIOS/UEFI |
|
|
| the command is not recognised | Windows build too old — needs 2004 or later |
|
|
| the install starts, then nothing works | a reboot is still pending |
|
|
|
|
Running the scripts from **Git Bash, MSYS or Cygwin does not work** — those look
|
|
close enough to a Linux shell to get started and then fail without `/proc` or a
|
|
docker socket. `ctrl/deps.sh` detects that and says so rather than letting you
|
|
find out the slow way.
|
|
|
|
## Read the docs first
|
|
|
|
```bash
|
|
make docs # serves on localhost, prints the URL
|
|
```
|
|
|
|
They run before anything is installed, which matters because they are the
|
|
instructions for everything else. No cluster and no toolchain required.
|
|
|
|
Why the code is the way it is — the reasoning, measurements and gotchas — lives
|
|
in [`docs/notes/`](docs/notes/), one file per script, so the code keeps short comments.
|
|
|
|
## Then
|
|
|
|
```bash
|
|
make check # is this machine ready? short; `make check all` for every detail
|
|
make deps # install the toolchain (add `core` on a managed machine)
|
|
make cluster up # cluster + registry + the profile's addons
|
|
```
|
|
|
|
That is the whole setup. `make cluster up` also starts this environment's local
|
|
registry and wires it into the node, so an image built locally is pullable by the
|
|
cluster without going near docker.io. `make check` shows its port, among
|
|
everything else:
|
|
|
|
```bash
|
|
make check # ... registry localhost:<port> (running)
|
|
docker build -t localhost:<port>/app:1 .
|
|
docker push localhost:<port>/app:1
|
|
kubectl --context kind-$(basename $PWD) run app --image=localhost:<port>/app:1
|
|
```
|
|
|
|
The port block is derived from the directory name, so two copies of rig never
|
|
collide — nothing to configure. `make check all` lists it; `bash ctrl/ports.sh persist`
|
|
pins it into `ctrl/.env` if you want it fixed:
|
|
|
|
```bash
|
|
make cluster list # every cluster on this machine, with memory
|
|
make cluster free # stop the others if memory is tight
|
|
make cluster down # remove this cluster and its registry
|
|
```
|
|
|
|
**The verbs are yours to change.** `cluster` is the script — `ctrl/cluster.sh` —
|
|
and every spelling above is a `Makefile` target that calls it. `make kind-up` is
|
|
an alias for `make cluster up`, kept because the other projects on this machine
|
|
answer to that spelling and muscle memory spans repos rather than stopping at
|
|
one. Nothing outside the `Makefile` reads these names, so rename them, drop the
|
|
ones you never type, or add whatever your own projects already say: each alias
|
|
is two lines at the bottom of the file, calling the same script the canonical
|
|
target does.
|
|
|
|
**`make tilt` works on a fresh copy, unedited.** rig's `ctrl/Tiltfile` does
|
|
rig's part — identity, context guard, registry, the manifests — and then includes
|
|
the overlay's own Tiltfile. With no overlay named that is `examples/starter`, so
|
|
the dev loop comes up with its two examples running and nothing to configure.
|
|
|
|
It hardcodes nothing. It asks `ctrl/ports.sh active` for this environment's
|
|
cluster name, kube context, ports and paths — the same values every other rig
|
|
script resolves through `ctrl/lib/config.sh` — so a copied and renamed rig, or a
|
|
moved overlay, deploys into its own cluster with no edits.
|
|
|
|
`make help` lists every target.
|
|
|
|
On a machine where Docker really is the only thing installed, `make deps` has
|
|
nothing to download with — see [BOOTSTRAP.md](BOOTSTRAP.md), which runs the
|
|
toolchain through the installer container and carries on to scaffolding and running
|
|
a new project.
|
|
|
|
## What runs is an overlay
|
|
|
|
rig is the machine: toolchain, cluster, registry, port block, the dev loop's
|
|
plumbing. What runs on it is an **overlay** — one folder, outside rig's version
|
|
control, holding a use case: its settings (`rig.env`), its manifests
|
|
(`k8s/overlays/dev`), its images and Tiltfile, its addons, its kind config if it
|
|
needs its own. rig reads it and never writes into it. See
|
|
[`docs/notes/overlay.md`](docs/notes/overlay.md).
|
|
|
|
```bash
|
|
cp -r examples/starter local/myenv # local/ is gitignored
|
|
OVERLAY=local/myenv make cluster up # or OVERLAY=local/myenv in ctrl/.env
|
|
OVERLAY=local/myenv make tilt
|
|
```
|
|
|
|
An overlay can also be a repo of its own, anywhere, or a project folder that
|
|
carries rig at `<project>/rig/` with a three-line forwarding Makefile.
|
|
|
|
## One environment per folder
|
|
|
|
Cluster name, kubectl context, image tags and the host port block all derive
|
|
from a folder name — the overlay's when one is named, else rig's own — so copies
|
|
never collide and neither one's teardown can touch the other. Two overlays run
|
|
side by side from one rig; two copies of rig do too.
|
|
|
|
## Profiles
|
|
|
|
**rig needs no profile.** With none named it runs on built-in defaults: one node,
|
|
no addons, a local registry, the newest Kubernetes version it pins. A profile is
|
|
an optional file in `ctrl/env.d/`, named by `PROFILE`, that says how this machine
|
|
reaches the world. rig ships two as **examples**; copy one to use it (the copy is
|
|
gitignored):
|
|
|
|
| example | what it changes |
|
|
| --- | --- |
|
|
| `mirror.env.example` | images through a pull-through cache of an internal registry |
|
|
| `offline.env.example` | air-gapped: everything from a preloaded local registry |
|
|
|
|
```bash
|
|
cp ctrl/env.d/mirror.env.example ctrl/env.d/mirror.env
|
|
PROFILE=mirror make cluster up
|
|
```
|
|
|
|
The layers, weakest first: built-in defaults < `ctrl/versions.env` < the profile
|
|
< the overlay's `rig.env` < `ctrl/.env` < your command line.
|
|
|
|
The **cluster itself** is one file: the overlay's `kind-config.yaml.tpl` if it has
|
|
one, else rig's `ctrl/k8s/kind-config.yaml.tpl` (one node). To change it — more
|
|
nodes, other port mappings, mounts — edit it and `make cluster reset`. The node
|
|
count is read back out of it, so there is nothing to drift.
|
|
|
|
## Addons
|
|
|
|
Each addon is its own idempotent script, and `ADDONS` names the ones to install,
|
|
in order. Adding one is adding a file — there is no dispatcher to edit. An
|
|
overlay's `addons/<name>.sh` is found before rig's own.
|
|
|
|
**There is no ingress controller, deliberately.** They pin a narrow window of
|
|
Kubernetes versions, so depending on one would constrain which k8s a rig can be
|
|
built with — and running a trailing-edge control plane is often the point.
|
|
Services are reached through MetalLB and `type: LoadBalancer`, which carries no
|
|
such constraint and is also what a real cluster does.
|
|
|
|
rig's own addons make the cluster work:
|
|
|
|
| Addon | Does |
|
|
| --- | --- |
|
|
| `metallb` | gives `type: LoadBalancer` an address it can actually reach |
|
|
| `cert-manager` | a local CA, so TLS works offline |
|
|
| `metrics-server` | makes `kubectl top` work on kind |
|
|
|
|
What a workload needs — a database, a cache, a scheduler — belongs to its
|
|
overlay. [`examples/data`](examples/data/) carries postgres, redis and airflow as
|
|
plain manifests (no helm: a chart repo is a network dependency), with passwords
|
|
generated on first install and kept across re-runs:
|
|
|
|
```bash
|
|
OVERLAY=examples/data make cluster up
|
|
```
|