Start here
A runnable local model of a large, regulated estate — legacy and new side by side.
rig builds a disposable Kubernetes environment on your machine so you can explore how a system fits together without needing access to any of it. Its job is onboarding and exploration, not a production replica.
Most services in it are deliberately not real. What has to be faithful is the topology — the names, the ports, the dependency order, who can reach whom, and how it fails. The workloads themselves are noise. This is what makes the whole estate fit on a laptop: a real 20-service platform will not fit even once on 14 GB, but mocks are about 30 MB each, so three faithful copies do.
The only prerequisite
Docker. No curl, no jq, no python, no apt repositories to configure.
# then, in the environment directory:
make check # is this machine ready? reports, never fixes
make deps # install the pinned toolchain
make cluster up # cluster + registry + addons; ports derive by themselves
Read make check before make deps. It never changes
anything — it prints what it found and, at the end, the steps it cannot perform
for you.
The steps
Start to finish, in order, with what each one actually does.
1 · make check
Asks whether this machine is ready. It changes nothing — it
reports what it found and, at the end, the things only a human can do
(anything needing sudo, or a Windows-side restart). Read it
before installing anything; it is faster than discovering the same problems
one failure at a time.
make check
2 · make deps
Installs the pinned toolchain — only what is missing — and tells you if its directory is not on PATH yet. Running it twice is safe.
make deps
3 · make cluster up
Builds the cluster, starts its registry and installs the profile's addons — there is nothing else to run first. It prints what the profile locks in before spending the time, because the kind config is fixed at creation and cannot be changed afterwards.
Re-running is safe and, more importantly, convergent: if a first attempt was interrupted before the CNI was installed, running it again finishes the job rather than reporting "already exists" and leaving every node permanently NotReady.
make cluster up # built-in defaults — no profile needed
make cluster up PROFILE=client # after copying env.d/client.env.example: cached registry
make cluster reset # destroy and rebuild — how an edited kind config takes effect
4 · make docs
Serves this page from a throwaway container. Works with no cluster and no toolchain, which is deliberate: these pages are the instructions for building everything else, so they cannot depend on it.
make docs
Checking on things
- make cluster list
- Every cluster on the machine, its memory cost and its port block. The usual reason a new one will not start is an old one you forgot about;
make cluster freefrees them without deleting. - make check
- Short: host, toolchain, and whether this cluster fits, its ports, registry and addons. Details only appear when something needs attention;
make check allprints every one. - make check mem
- Memory in depth: what caps it, how far it really climbs, and on WSL the
.wslconfigbackup and restore.
Running more than one
Copy the directory, rename it, and run make cluster up. Cluster name,
context, image tags and the port block all follow the directory name, so
the second environment collides with nothing and neither one's teardown can
reach the other.
cp -r rig ../platform-v2 && cd ../platform-v2
make cluster up
Installation
A container installs onto the host and then gets out of the way.
The installer is a container, not a shell script, for a specific reason: a
stock slim Debian has no curl, no wget, no
jq, no python3 and no CA bundle. A shell
installer could not make a verified HTTPS request, let alone check one. The
container carries its own toolchain, so the host needs nothing but Docker.
The cluster never runs inside that container. Everything it installs — kind, kubectl, tilt, jq — runs natively afterwards, so nothing pays a container tax during daily work.
Pinned and verified
Every tool is a single binary fetched at a pinned version and checked against a published SHA256. Node images are pinned by digest, so upgrading kind cannot silently move your Kubernetes version.
Not every machine should get cluster tooling
A managed or corporate-issued machine — the kind that holds the access you cannot get anywhere else — is not somewhere to install development tools by default. So the toolchain comes in two tiers:
| tier | installs | for |
|---|---|---|
core | kubectl, jq | talk to a cluster someone else runs |
dev | + kind, tilt | build clusters and hot-reload into them |
make deps core # kubectl and jq only — nothing that creates a cluster
make deps # dev, the default
Testing in situ on a managed machine is still possible — install
the dev tier deliberately when you need it. The point is that
it should be a decision rather than a side effect of installing.
The documentation itself needs neither tier: make docs
wants only Docker.
Air-gapped
make deps image full # bakes every binary into the image
docker save …-deps:full | gzip > rig.tgz
# carry that one file in, then:
docker load < rig.tgz && make cluster up PROFILE=offline # from env.d/offline.env.example
Environments
One directory is one environment. Copy it, rename it, run it.
Running several versions of a system at once means several clusters on one machine, not several machines. Everything that could collide is derived from the directory name:
- cluster + context
acmebank/buildsacmebankonkind-acmebank.- port block
- Ten ports from a hash of the name, in the 20000+ range — clear of 80, 443, 3000, 5432, 8000 and 8080.
- registry + images
- Named after the environment, so two copies never share one.
Two copies therefore never collide, and neither one's
make cluster down can touch the other. make check
shows the block; bash ctrl/ports.sh persist freezes it into
ctrl/.env if you want it fixed rather than derived.
Configuration layers
Weakest first, later wins: built-in defaults → pinned versions → a
profile, if you name one → ctrl/.env → the environment. So
make cluster up PROFILE=<name> always beats every file.
Profiles
Optional overlays — rig needs none.
| example | registry | for |
|---|---|---|
| none | local | the built-in defaults; no profile needed |
client.env.example | mirror | images through a corporate registry |
offline.env.example | local | air-gapped |
data.env.example | local | postgres, redis, airflow |
The kind config cannot be re-applied. Edit
ctrl/k8s/kind-config.yaml.tpl; it takes effect when the cluster is created. cluster up prints what it
locks in before spending the time, and make cluster reset is
the way out.
LoadBalancer services
Real manifests use type: LoadBalancer, because a real
cluster has one. On a bare local cluster those Services sit at
EXTERNAL-IP <pending> forever, with no error anywhere —
the deployment looks healthy and simply is not reachable.
The metallb addon fixes that, so the same manifests work
here as upstream and nothing has to be rewritten to NodePort. Its address
pool is derived from the cluster's Docker network at install time rather
than hardcoded, because Docker picks that subnet and it differs between
machines.
Where those addresses are reachable from. The pool lives on the Docker bridge, so LoadBalancer IPs work from the Linux side — including from inside WSL. A browser on Windows has no route to them. Use the ingress host ports for anything you need to open in a browser.
Networking
The cluster uses kind's built-in networking, which does enforce standard NetworkPolicy — verified against a no-policy control, not assumed. The widely repeated claim that it accepts policies and silently ignores them is out of date.
A pluggable CNI was tried and removed: it only added GlobalNetworkPolicy, policy tiers and egress-CIDR rules, none of which are needed yet, in exchange for a slower boot and one more thing that has to be right at creation time. Worth revisiting only when a policy the built-in cannot express actually comes up.
Memory
Every cluster is a running container tree whether you are using it or not.
make cluster list shows what exists and what it costs;
make cluster free stops the others without deleting them.
Registry
Local, cached, or straight to the corporate registry.
| mode | what it does |
|---|---|
none | images are built straight into the node |
local | a registry container wired into the cluster |
mirror | that container as a pull-through cache of the corporate registry |
remote | no local container; pull direct with an imagePullSecret |
mirror is what a locked-down network actually looks like:
images originate from the corporate registry, you do not hammer it, and you
keep working when the connection drops.
The corporate CA will bite you. A corporate
registry is usually behind an internal CA, and trust has to reach
three places: the host Docker daemon, every cluster node's containerd
(nodes do not inherit host trust), and any in-cluster client. Set
REGISTRY_CA_FILE and make check reports which is
still missing. The symptom otherwise is an opaque
x509: certificate signed by unknown authority.
Reachability also depends on where you are: if the registry is only
routable from a managed workspace, mirror and remote
will not resolve from a laptop at all. That is what local and
offline are for.
Architecture
The estate being modelled.
TODO — placeholder. The diagram below is
illustrative only: it shows how a mocked dependency, a service under active
work, and an unreachable remote system sit together. It is not the real
topology. Replace docs/graphs/03-architecture.dot with the
extracted platform diagrams, then run make docs graphs.
Each component is one of three things, and switching between them should be a one-line change rather than a rewrite:
- real
- Built from source and hot-reloaded. The thing you are actually working on — usually exactly one.
- mock
- A generic stub with canned responses. Everything you do not care about today.
- remote
- No pod at all: a Service of type ExternalName pointing at the real system. In-cluster DNS resolves identically, so callers never change.
The intended end state is that this diagram is generated from the running cluster rather than drawn by hand — so it becomes a report of what exists instead of a picture of what was once intended.
Troubleshooting
The failures that are hard to diagnose from their symptoms.
Tilt stops noticing file changes
Almost always inotify limits, and it fails silently —
nothing errors, changes just stop being picked up. Defaults on WSL are far too
low. make check reports it and prints the fix.
Cluster creation dies halfway with a port error
Docker reports failed to bind host port … address already in use
partway through creating the cluster. Run make check first — it
checks every port in this environment's block before anything is built.
Every node stays NotReady
Usually a cluster created with the default CNI disabled but the real CNI
never installed — typically an interrupted first run. Just run
make cluster up again: it converges rather than exiting early, and
will finish the missing steps.
x509: certificate signed by unknown authority
Corporate CA trust has not reached one of the three places it needs to be. See Registry.
kubectl says the context does not exist
The cluster can exist while its context does not — a reset or a switched
KUBECONFIG loses it. make cluster up detects this and
re-exports the context.