185 lines
7.4 KiB
Markdown
185 lines
7.4 KiB
Markdown
# rig
|
|
|
|
A runnable local model of a large, regulated estate — legacy and new side by
|
|
side. Its job is onboarding and exploration, not a production replica: most
|
|
services are deliberately mocked, because what has to be faithful is the
|
|
topology, not the workloads.
|
|
|
|
## Prerequisite
|
|
|
|
**Docker.** Nothing else — no curl, no jq, no python, no apt repositories.
|
|
|
|
### Starting from plain Windows
|
|
|
|
Everything here is bash and runs *inside* a Linux shell, so on a Windows machine
|
|
that means WSL. Nothing in rig installs WSL, and nothing will: `wsl --install`
|
|
enables Windows features and requires a reboot, which is not something a script
|
|
should do to a machine on your behalf — and there is no tested undo for it.
|
|
|
|
From an elevated PowerShell or Command Prompt, once:
|
|
|
|
```powershell
|
|
wsl --install
|
|
```
|
|
|
|
Then reboot and open the Linux shell it installed.
|
|
|
|
**If you cloned this on the Windows side, copy it into WSL before carrying on.**
|
|
WSL can reach the Windows drives at `/mnt/c`, and working from there mostly
|
|
functions — slowly — but file watching does not: that filesystem raises no
|
|
inotify events, so anything watching for edits silently stops seeing them.
|
|
|
|
```bash
|
|
cp -r /mnt/c/Users/<you>/rig ~/rig
|
|
cd ~/rig
|
|
```
|
|
|
|
`make deps` reports it if you are running from `/mnt/...`. Then carry on below.
|
|
|
|
If it fails, the usual causes give unhelpful messages:
|
|
|
|
| symptom | cause |
|
|
| --- | --- |
|
|
| "the virtual machine could not be started" | virtualization disabled in BIOS/UEFI |
|
|
| the command is not recognised | Windows build too old — needs 2004 or later |
|
|
| the install starts, then nothing works | a reboot is still pending |
|
|
|
|
Running the scripts from **Git Bash, MSYS or Cygwin does not work** — those look
|
|
close enough to a Linux shell to get started and then fail without `/proc` or a
|
|
docker socket. `ctrl/deps.sh` detects that and says so rather than letting you
|
|
find out the slow way.
|
|
|
|
## Read the docs first
|
|
|
|
```bash
|
|
make docs # serves on localhost, prints the URL
|
|
```
|
|
|
|
They run before anything is installed, which matters because they are the
|
|
instructions for everything else. No cluster and no toolchain required.
|
|
|
|
## Then
|
|
|
|
```bash
|
|
make check # report host and config problems; changes nothing
|
|
make deps # install the toolchain (add `core` on a managed machine)
|
|
make cluster up # build the cluster for the active profile
|
|
```
|
|
|
|
`make cluster up` also starts this environment's local registry and wires it
|
|
into the node, so an image built locally is pullable by the cluster without
|
|
going near docker.io:
|
|
|
|
```bash
|
|
make registry status # prints: endpoint localhost:<port>
|
|
docker build -t localhost:<port>/app:1 .
|
|
docker push localhost:<port>/app:1
|
|
kubectl --context kind-$(basename $PWD) run app --image=localhost:<port>/app:1
|
|
```
|
|
|
|
The port block is derived from the directory name, so two copies of rig never
|
|
collide:
|
|
|
|
```bash
|
|
make ports show # HTTP / HTTPS / TILT / REGISTRY
|
|
make cluster list # every cluster on this machine, with memory
|
|
make cluster free # stop the others if memory is tight
|
|
make cluster down # remove this cluster and its registry
|
|
```
|
|
|
|
**`make tilt` has nothing to run yet.** The target and its `tilt-up` / `tilt-down`
|
|
aliases exist so rig answers to the same spelling as every other project here,
|
|
but rig ships no `Tiltfile` — it builds the estate, it is not itself a service
|
|
with a dev loop. Add a `ctrl/Tiltfile` and the target works; until then it fails
|
|
on the missing file, not on anything rig did.
|
|
|
|
`make help` lists every target.
|
|
|
|
On a machine where Docker really is the only thing installed, `make deps` has
|
|
nothing to download with — see [BOOTSTRAP.md](BOOTSTRAP.md), which runs the
|
|
toolchain through the installer container and carries on to scaffolding and running
|
|
a new project.
|
|
|
|
## One directory is one environment
|
|
|
|
Copy this directory, rename it, run it. Cluster name, kubectl context, image
|
|
tags and the host port block all derive from the directory name, so copies never
|
|
collide and neither one's teardown can touch the other.
|
|
|
|
A copy of this directory is a **sibling** of it, named after the environment it
|
|
models (`acme-rig`). That is why the ignore rules for copies sit in the *parent*
|
|
repo's `.gitignore` rather than here: a rule in this directory cannot see a
|
|
directory beside it.
|
|
|
|
## Profiles
|
|
|
|
A profile is the shape of the cluster: how many nodes, which addons, whether the
|
|
apiserver audits. They live in `ctrl/env.d/`, and the active one is `PROFILE`.
|
|
|
|
| Profile | For |
|
|
| --- | --- |
|
|
| `minimal` | the default. One node, no addons, boots fast. |
|
|
| `client` | the regulated-estate shape — multi-node, audit on, registry mirror. |
|
|
| `offline` | air-gapped: everything from a preloaded local registry. |
|
|
| `data` | the cabinets an environment asks for. |
|
|
|
|
```bash
|
|
PROFILE=data make cluster up
|
|
PROFILE=data make addons install
|
|
make addons # what the active profile wants, and what exists
|
|
```
|
|
|
|
A profile names a **cluster shape** — a file in `ctrl/k8s/` — rather than
|
|
restating node count and audit as variables:
|
|
|
|
| shape | nodes | audit | used by |
|
|
| --- | --- | --- | --- |
|
|
| `kind-config.yaml.tpl` | 1 | off | `minimal`, `data` |
|
|
| `kind-config.audit.yaml.tpl` | 1 | on | `offline` |
|
|
| `kind-config.client.yaml.tpl` | 3 | on | `client` |
|
|
|
|
Both numbers are read back out of the chosen file, so the YAML is the only place
|
|
that decides and there is nothing to drift. The layout under `ctrl/k8s/` is the
|
|
same as every other project here — a kind config, a kustomize `base/`, an
|
|
`overlays/dev/` — see [`ctrl/k8s/README.md`](ctrl/k8s/README.md).
|
|
|
|
## Addons
|
|
|
|
Each addon is its own idempotent script in `ctrl/addons/`, and a profile names
|
|
the ones it wants in `ADDONS`. Adding one is adding a file — there is no
|
|
dispatcher to edit.
|
|
|
|
**There is no ingress controller, deliberately.** They pin a narrow window of
|
|
Kubernetes versions, so depending on one would constrain which k8s a rig can be
|
|
built with — and running a trailing-edge control plane to model a legacy estate
|
|
is the whole point. Services are reached through MetalLB and
|
|
`type: LoadBalancer`, which carries no such constraint and is also what a real
|
|
cluster does.
|
|
|
|
| Addon | Does |
|
|
| --- | --- |
|
|
| `metallb` | gives `type: LoadBalancer` an address it can actually reach |
|
|
| `cert-manager` | a local CA, so TLS works offline |
|
|
| `metrics-server` | makes `kubectl top` work on kind |
|
|
| `postgres` | database, in the `data` namespace |
|
|
| `redis` | cache and broker |
|
|
| `airflow` | scheduled pipelines; needs postgres and redis |
|
|
|
|
The last three are **cabinets**: a public service dropped in as-is, the upstream
|
|
image unmodified, reachable at a known address. A cabinet is declared once and
|
|
installs on either target — a `service.yml` composes it for a laptop, and these
|
|
install the same one here. The names match on purpose: each cabinet carries a
|
|
`rig_addon` field pointing at `ctrl/addons/<name>.sh`.
|
|
|
|
Plain manifests rather than helm charts, like every other addon: a chart repo is
|
|
a network dependency, and the `offline` profile exists precisely so there is a
|
|
path with none. Images are pinned in `ctrl/versions.env` and can be preloaded.
|
|
|
|
Passwords are generated on first install and kept across re-runs, so re-running
|
|
an addon never rotates a credential out from under something already connected:
|
|
|
|
```bash
|
|
kubectl -n data get secret postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d
|
|
kubectl -n data port-forward svc/airflow 8080:8080
|
|
```
|