Start here
rig builds a disposable Kubernetes environment on your machine, so you can see how a system fits together without access to any of it. It is for onboarding and exploration, not a production replica.
Most services in it are not real. What has to be faithful is the topology: names, ports, dependency order, who can reach whom, and how it fails. That is what makes a whole estate fit on a laptop. A real 20-service platform does not fit once in 14 GB; stubs of about 30 MB each fit three times.
The only prerequisite
Docker. No curl, no jq, no python, no apt repositories to configure.
# in the environment directory:
make check # is this machine ready? reports, never changes anything
make deps # install the pinned toolchain; safe to re-run
make cluster up # cluster + registry + addons; ports derive by themselves
Read make check before make deps: it lists what is
missing, including the steps only you can do, such as anything needing
sudo or a Windows-side restart.
make cluster up
It prints what the kind config locks in before spending the time, because
that config is fixed when the cluster is created. Re-running is safe and
continues where an interrupted run stopped, rather than reporting that the
cluster already exists and leaving every node NotReady.
make cluster up # built-in defaults — no profile needed
make cluster up PROFILE=mirror # images through an internal registry
make cluster up OVERLAY=examples/data # an overlay: what runs, kept outside rig
make cluster reset # destroy and rebuild: how an edited kind config takes effect
Checking on things
- make cluster list
- Every cluster on the machine, its memory cost and its port block. A new cluster usually fails to start because of an old one;
make cluster freestops them without deleting. - make check
- Host, toolchain, and whether this cluster fits, its ports, registry and addons. Only problems print;
make check allprints everything. - make check mem
- Memory in depth: what caps it, how far it climbs, and on WSL the
.wslconfigbackup and restore. - make docs
- Serves these pages from a throwaway container. It needs no cluster and no toolchain, because these are the instructions for building both.
Installation
The installer is a container, not a shell script, because a stock slim Debian has no curl, wget, jq, python3 or CA bundle: a shell installer could not make a verified HTTPS request, let alone check one. The container carries its own toolchain, so the host needs only Docker. Nothing runs inside it afterwards — kind, kubectl, tilt and jq run natively.
Every tool is a single binary at a pinned version, checked against a published SHA256. Node images are pinned by digest, so upgrading kind cannot move your Kubernetes version.
Two tiers
A managed or corporate-issued machine is not somewhere to install development tools by default, so installing cluster tooling is a decision rather than a side effect:
| tier | installs | for |
|---|---|---|
core | kubectl, jq | talk to a cluster someone else runs |
dev | + kind, tilt | build clusters and hot-reload into them |
make deps core # kubectl and jq only — nothing that creates a cluster
make deps # dev, the default
Air-gapped
make deps image full # bakes every binary into the image
docker save …-deps:full | gzip > rig.tgz
# carry that one file in, then:
docker load < rig.tgz && make cluster up PROFILE=offline # from env.d/offline.env.example
Environments
One folder is one environment. Running several versions of a system at once means several clusters on one machine, and everything that could collide is derived from the folder name (the overlay's, when one is named):
- cluster + context
platform-v2/buildsplatform-v2onkind-platform-v2.- port block
- Ten ports from a hash of the name, above 20000 — clear of 80, 443, 3000, 5432, 8000 and 8080.
- registry + images
- Named after the environment, so two copies never share one.
So two copies never collide, and neither one's make cluster down
can touch the other. make check shows the block;
bash ctrl/ports.sh persist freezes it into ctrl/.env.
Configuration layers
Weakest first, later wins: built-in defaults → pinned versions → a profile, if
you name one → the overlay's rig.env → ctrl/.env → the
caller's environment. So a value on the command line always beats every file.
Profiles
A profile says how this machine reaches the world. rig needs none.
| example | registry | for |
|---|---|---|
| none | local | the built-in defaults |
mirror.env.example | mirror | images through an internal registry |
offline.env.example | local | air-gapped |
Registry modes
| mode | what it does |
|---|---|
none | images are built straight into the node |
local | a registry container wired into the cluster |
mirror | that container as a pull-through cache of the corporate registry |
remote | no local container; pull direct with an imagePullSecret |
mirror is what a locked-down network looks like: images come from
the corporate registry, you do not hammer it, and you keep working when the
connection drops. It is also only reachable from where that registry is routable —
from a laptop, use local or offline.
Corporate CA trust is the usual failure. It has to reach three places:
the host Docker daemon, every node's containerd (nodes do not inherit host trust),
and any in-cluster client. Set REGISTRY_CA_FILE;
make check reports which is still missing. Without it the error is
x509: certificate signed by unknown authority.
LoadBalancer services
Real manifests use type: LoadBalancer. On a bare local cluster
those Services sit at EXTERNAL-IP <pending> forever, with no
error: the deployment looks healthy and is unreachable. The metallb
addon fixes that, so upstream manifests work unchanged. Its address pool is
derived from the cluster's Docker network at install time, because Docker picks
that subnet and it differs between machines.
Those addresses are reachable from the Linux side, including inside WSL. A browser on Windows has no route to them: use the environment's HTTP port instead, where an overlay's gateway listens (its kind config maps it to the host).
Networking and memory
The cluster uses kind's built-in networking, which does enforce standard NetworkPolicy — verified against a no-policy control. A pluggable CNI was tried and removed: it added policy features nothing here needs, in exchange for a slower boot and one more thing to get right at creation time.
Every cluster is a running container tree whether you use it or not.
make cluster list shows what each costs, and
make cluster free stops the others without deleting them.
Overlays
An overlay is one folder, kept outside rig's version control, holding a use case. rig reads it and never writes into it. Every piece is optional:
| in the overlay | what rig does with it |
|---|---|
rig.env | a config layer: addons, namespaces, images — anything a profile could set |
k8s/overlays/dev/ | the manifests the dev loop applies |
kind-config.yaml.tpl | the cluster's shape, when it needs its own (mounts, ports) |
addons/<name>.sh | addons, found before rig's own; each applies its own <name>.yaml |
Tiltfile | the workload's half of the dev loop, included by rig's |
cp -r examples/starter local/myenv # local/ is gitignored
OVERLAY=local/myenv make cluster up
With none named, rig runs its own examples/starter.
examples/data carries postgres, redis and airflow as an overlay's own
addons. A project can carry rig at ./rig, or keep it anywhere and name
it with RIG_DIR. .version names the release; the project
records the one it was last run against. See docs/notes/overlay.md.
What runs in it
Each component is one of three things, and switching between them is a one-line change:
- real
- Built from source and hot-reloaded. What you are working on — usually exactly one.
- mock
- A generic stub with canned responses. Everything you do not care about today.
- remote
- No pod at all: a Service of type
ExternalNamepointing at the real system. In-cluster DNS resolves identically, so callers never change.
Troubleshooting
Tilt stops noticing file changes
Almost always inotify limits, and it fails silently: nothing errors, changes
just stop being picked up. Defaults on WSL are far too low.
make check reports it and prints the fix.
Cluster creation dies halfway with a port error
Docker reports failed to bind host port … address already in use
partway through creating the cluster. make check tests every port in
this environment's block before anything is built.
Every node stays NotReady
Usually a cluster created with the default CNI disabled and the real CNI never
installed, after an interrupted first run. Run make cluster up again:
it finishes the missing steps instead of stopping because the cluster exists.
x509: certificate signed by unknown authority
Corporate CA trust has not reached one of the three places it needs to be. See Environments.
kubectl says the context does not exist
The cluster can exist while its context does not — a reset or a switched
KUBECONFIG loses it. make cluster up detects this and
re-exports the context.