Merge branch 'rig-work'

This commit is contained in:
2026-09-22 06:47:55 -03:00
64 changed files with 1762 additions and 645 deletions

View File

@@ -14,7 +14,7 @@
# KIND_CONFIG spr's own kind config, which maps the rooms' gateway NodePorts
# REGISTRY_MODE=none rooms load images straight into the node
# PROFILE= ADDONS= set empty here, so rig's own ctrl/.env can never quietly
# pick a profile or add addons to spr's cluster
# OVERLAY= pick a profile, an overlay or addons for spr's cluster
#
# status stays here: it answers a question about rooms, not about the cluster.
set -e
@@ -27,6 +27,7 @@ rig() {
KIND_CONFIG="$SCRIPT_DIR/k8s/kind-config.yaml" \
REGISTRY_MODE=none \
PROFILE= \
OVERLAY= \
ADDONS= \
bash "$RIG_CTRL/cluster.sh" "$@"
}

14
rig/.gitignore vendored
View File

@@ -16,6 +16,14 @@ arch/*.dot
# binaries pulled by `make deps-bundle` for the air-gapped installer image
vendor
# Client rigs are NOT ignored here. A copy is a SIBLING of this directory
# (../acme-rig), so a rule in this file cannot see it — the rules live in the
# parent repo's .gitignore, anchored at its root, where `*-rig/` matches them.
# Overlays that live with no version control of their own, or clones of their own
# repos: rig reads them and never tracks them (docs/notes/overlay.md).
/local/
# A profile you activated (cp env.d/<name>.env.example env.d/<name>.env) is this
# machine's choice; only the examples are committed.
ctrl/env.d/*.env
# Only the default kit is committed; a kit for a local profile is this machine's.
standalone/*
!standalone/default/

View File

@@ -1,18 +1,9 @@
# From a machine with nothing on it to a project you can work in
# From a machine with nothing on it to an environment you can work in
The README says the prerequisite is Docker and nothing else. This is what that
actually looks like end to end: a bare Linux box, and a new project running under
actually looks like end to end: a bare Linux box, and an overlay running under
Tilt at the end of it.
A copy of this directory is a sibling of it, named after the environment it
models (`acme-rig`). Paths below are relative to the parent checkout.
It spans three repos because the work does. **rig** prepares the machine — the
pinned toolchain, the cluster, the port arithmetic. **all** owns the shape a
project takes, in `all/projects/templates/conventions.md` and the `broad`
scaffold beside it. **ppl** owns everything after local, and is where this
document stops.
Read it once before running anything. Three of the steps below need root and one
needs a logout, so knowing about them in advance is cheaper than meeting them
halfway through.
@@ -108,7 +99,7 @@ docker run --rm \
```
The image name follows the directory, like everything else here: in `rig`
it is `rig-deps`, in a copy called `acme-rig` it is `acme-rig-deps`. The
it is `rig-deps`, in a copy called `other-rig` it is `other-rig-deps`. The
tag is `deps` (or `full`, below), not `latest`.
None of the four arguments are guessable, so:
@@ -183,80 +174,30 @@ worth reading before rather than after. `make cluster free <names>` stops
clusters without deleting them; `docker start` brings them back untouched.
## Scaffold the project
## Start an overlay
The canonical layout is [`all/projects/templates/conventions.md`](../all/projects/templates/conventions.md).
Read it — it is short, opinionated, and exists precisely so nobody
reverse-engineers a layout from whichever repo they happened to open. What
follows is only the mechanical part.
What runs lives outside rig, in an overlay — see
[`docs/notes/overlay.md`](docs/notes/overlay.md). Start from rig's own:
```bash
SLUG=<slug> # short, lowercase, no separators
cp -r ~/wdir/semester/all/projects/templates/broad ~/wdir/semester/"$SLUG"
cd ~/wdir/semester/"$SLUG"
grep -rl '<slug>' ctrl | xargs sed -i "s/<slug>/$SLUG/g"
cp ctrl/k8s/.env.example ctrl/k8s/.env
git init && git add -A && git commit -m "scaffold $SLUG from broad"
cp -r examples/starter local/myenv # local/ is gitignored by rig
echo 'OVERLAY=local/myenv' >> ctrl/.env
make check # shows the overlay, its cluster and ports
```
`<slug>` is the only placeholder and it lives only under `ctrl/` — cluster name,
namespace, ConfigMap name, and the `NAME=` in `kind-up.sh` / `kind-down.sh`. One
sed does all of it.
The slug is the folder name, lowercase and short — `mpr`, `unt`, `nvi`. The
cluster takes that name and the context becomes `kind-<slug>`, derived by the
scaffold's Makefile from the directory, so there is nothing to edit for either.
**Pick the Tilt port deliberately.** `ctrl/k8s/.env.example` ships a value that
is already in use, so copying it unchanged puts two projects on one port:
The cluster takes the overlay's folder name and the context becomes
`kind-<name>`, so there is nothing to edit for either. Replace the two example
components under `k8s/base/`, and add your images and resources to the
overlay's `Tiltfile`. Check the manifests before `kind` spends minutes on anything
— this renders the whole tree without a cluster and catches a broken patch
immediately:
```bash
grep -h '^TILT_PORT=' ~/wdir/semester/*/ctrl/k8s/.env 2>/dev/null | sort
kubectl kustomize local/myenv/k8s/overlays/dev
```
Choose a free one in `1030010399` — the range ALL reserves in
`projects/index.json` under `policy` — avoiding `10350`, which is Tilt's own
default. Currently taken: `nvi` 10330, `unt` 10340, `mpr` 10360, `mlv` 10370,
`eth` 10380, `lng` 10390. This is the Tilt *web UI* port, not a service port;
each project owns its own service ports separately. The scaffold ships it blank
on purpose, so there is nothing to collide with until you choose.
The scaffold's `ctrl/k8s/` is the same shape as every other project here, and it
builds as shipped:
```
kind-config.yaml one node; gateway NodePort 30080 -> hostPort 8080
base/ namespace, configmap, app (Deployment + Service)
overlays/dev/ promotes the app Service to NodePort 30080
```
Check it before `kind` spends minutes on anything — this renders the whole tree
without a cluster and catches a broken patch immediately:
```bash
kubectl kustomize ctrl/k8s/overlays/dev
```
The workload is an nginx placeholder so a fresh copy reaches something that
answers; replace it. Keep `30080` in step between the overlay patch and
`kind-config.yaml`'s `containerPort` — the hostPort is this project's to pick.
Reachability is a plain kind port mapping: no ingress controller and no MetalLB.
Caddy maps `<slug>.local.ar` onto the host port (`~/wdir/semester/ppl/local/Caddyfile`),
with `*.local.ar` resolving to 127.0.0.1 through dnsmasq. That is the whole chain.
**For `ctrl/Tiltfile`, copy rig's** rather than a live project's. rig ships one
that derives its cluster, context, ports and manifest directory from
`ctrl/ports.sh active` instead of hardcoding a slug, and carries a catalogue of
the blocks every project here ends up needing. Copying from `unt` or `nvi` is
what the estate did until now, and it is why the same Tiltfile preamble exists
in six places with the slug typed in by hand five times each.
> **Two things in this document disagree with rig and are not settled.** It
> mandates Tilt ports in `1030010399`, while rig derives a block from the
> directory name at `20000+` so copies cannot collide — a rig-managed project
> takes rig's. And it names `ctrl/k8s/.env.example`, which is the `broad`
> scaffold's layout; rig's is `ctrl/.env.example`. Both are this document
> describing the house scaffold from inside rig's tree.
If the overlay is to be versioned, make `local/myenv` a repository of its own (rig
never tracks it), or keep it anywhere else and name it by path.
## Run it
@@ -266,20 +207,9 @@ make kind-up # idempotent create, then selects the context
make tilt-up # context + your assigned port
```
`tilt-up` passes `--context kind-<slug>` every time, which is the point of going
`tilt-up` passes `--context kind-<name>` every time, which is the point of going
through `make` at all: tilt cannot deploy into whichever cluster you last looked
at.
`make tilt-down` and `make kind-down` close the loop, and `make kind-reset` is
delete-and-recreate for when a cluster wedges.
## Register it
The project exists; now it is findable. Add an entry to
`~/wdir/semester/all/projects/index.json` and write its `projects/<slug>.md` beside the
others. Structured fields in the index, prose in the markdown.
Putting it on the CI server and deploying it is `ppl`'s half, and it starts at
`~/wdir/semester/ppl/ctrl/init-repo.sh` — gitea remote, then Woodpecker. That is a
different document.

View File

@@ -4,8 +4,10 @@
# Notes: docs/notes/Makefile.md
# Identity and ports, asked once of ctrl/ports.sh, read positionally (selftest pins the order):
# CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR
FACTS := $(shell bash ctrl/ports.sh active 2>/dev/null)
# CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR OVERLAY_DIR
# OVERLAY/CLUSTER given as make arguments are handed over explicitly: before make 4.4,
# $(shell) does not see them, and the context would follow the wrong environment.
FACTS := $(shell $(if $(OVERLAY),OVERLAY='$(OVERLAY)') $(if $(CLUSTER),CLUSTER='$(CLUSTER)') bash ctrl/ports.sh active 2>/dev/null)
SLUG := $(shell echo '$(notdir $(CURDIR))' | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9-' '-' | sed 's/^-*//; s/-*$$//')
# Fall back to the folder name, not empty, if ports.sh fails on a broken config.
CLUSTER := $(or $(word 1,$(FACTS)),$(SLUG))

View File

@@ -100,24 +100,15 @@ ones you never type, or add whatever your own projects already say: each alias
is two lines at the bottom of the file, calling the same script the canonical
target does.
**`make tilt` works on a fresh copy, unedited.** rig ships `ctrl/Tiltfile`, and
`k8s/base` already boots, so the dev loop comes up with the two examples running
and nothing to configure first.
**`make tilt` works on a fresh copy, unedited.** rig's `ctrl/Tiltfile` does
rig's part — identity, context guard, registry, the manifests — and then includes
the overlay's own Tiltfile. With no overlay named that is `examples/starter`, so
the dev loop comes up with its two examples running and nothing to configure.
It hardcodes nothing. It asks `ctrl/ports.sh active` for this environment's
cluster name, kube context, ports and manifest directory — the same values every
other rig script resolves through `ctrl/lib/config.sh` — so a copied and renamed
rig deploys into its own cluster with no edits. Every other project here writes
its slug into the Tiltfile five or six times by hand, which is exactly the
collision `kind-config.yaml.tpl` exists to avoid.
What it deploys is `MANIFESTS_DIR`, defaulting to rig's own `ctrl/k8s/overlays/dev`.
Point that at an overlay versioned elsewhere and rig stops owning the manifests.
Replace the examples, then add your images and resources in the two marked
sections. The catalogue below them holds the blocks that recur across every
project here — `docker_build`, resource ordering, gateway reload, port-forwards —
with the parts that are easy to get wrong already commented.
cluster name, kube context, ports and paths — the same values every other rig
script resolves through `ctrl/lib/config.sh` — so a copied and renamed rig, or a
moved overlay, deploys into its own cluster with no edits.
`make help` lists every target.
@@ -126,81 +117,82 @@ nothing to download with — see [BOOTSTRAP.md](BOOTSTRAP.md), which runs the
toolchain through the installer container and carries on to scaffolding and running
a new project.
## One directory is one environment
## What runs is an overlay
rig is the machine: toolchain, cluster, registry, port block, the dev loop's
plumbing. What runs on it is an **overlay** — one folder, outside rig's version
control, holding a use case: its settings (`rig.env`), its manifests
(`k8s/overlays/dev`), its images and Tiltfile, its addons, its kind config if it
needs its own. rig reads it and never writes into it. See
[`docs/notes/overlay.md`](docs/notes/overlay.md).
```bash
cp -r examples/starter local/myenv # local/ is gitignored
OVERLAY=local/myenv make cluster up # or OVERLAY=local/myenv in ctrl/.env
OVERLAY=local/myenv make tilt
```
An overlay can also be a repo of its own, anywhere, or a project folder that
carries rig at `<project>/rig/` with a three-line forwarding Makefile.
Copy this directory, rename it, run it. Cluster name, kubectl context, image
tags and the host port block all derive from the directory name, so copies never
collide and neither one's teardown can touch the other.
## One environment per folder
A copy of this directory is a **sibling** of it, named after the environment it
models (`acme-rig`). That is why the ignore rules for copies sit in the *parent*
repo's `.gitignore` rather than here: a rule in this directory cannot see a
directory beside it.
Cluster name, kubectl context, image tags and the host port block all derive
from a folder name — the overlay's when one is named, else rig's own — so copies
never collide and neither one's teardown can touch the other. Two overlays run
side by side from one rig; two copies of rig do too.
## Profiles
**rig needs no profile.** With none named it runs on built-in defaults: one node,
no addons, a local registry, the newest Kubernetes version it pins. A profile is
an optional overlay — a file in `ctrl/env.d/`, named by `PROFILE` — for when you
want different addons or registry.
rig ships **examples**, not active profiles, because each one is a use case rather
than something every rig needs. Copy one to use it:
an optional file in `ctrl/env.d/`, named by `PROFILE`, that says how this machine
reaches the world. rig ships two as **examples**; copy one to use it (the copy is
gitignored):
| example | what it changes |
| --- | --- |
| `client.env.example` | images through a mirror of a corporate registry |
| `mirror.env.example` | images through a pull-through cache of an internal registry |
| `offline.env.example` | air-gapped: everything from a preloaded local registry |
| `data.env.example` | databases and a scheduler: postgres, redis, airflow |
```bash
cp ctrl/env.d/data.env.example ctrl/env.d/data.env
PROFILE=data make cluster up # installs the profile's addons too
PROFILE=data make check # what the profile wants, and what exists
cp ctrl/env.d/mirror.env.example ctrl/env.d/mirror.env
PROFILE=mirror make cluster up
```
The **cluster itself** is one file, `ctrl/k8s/kind-config.yaml.tpl` (one node).
To change it — more nodes, other port mappings — edit it and `make cluster
reset`. The node count is read back out of it, so there is nothing to drift. The layout under `ctrl/k8s/` is the
same as every other project here — a kind config, a kustomize `base/`, an
`overlays/dev/` — see [`ctrl/k8s/README.md`](ctrl/k8s/README.md).
The layers, weakest first: built-in defaults < `ctrl/versions.env` < the profile
< the overlay's `rig.env` < `ctrl/.env` < your command line.
The **cluster itself** is one file: the overlay's `kind-config.yaml.tpl` if it has
one, else rig's `ctrl/k8s/kind-config.yaml.tpl` (one node). To change it — more
nodes, other port mappings, mounts — edit it and `make cluster reset`. The node
count is read back out of it, so there is nothing to drift.
## Addons
Each addon is its own idempotent script in `ctrl/addons/`, and a profile names
the ones it wants in `ADDONS`. Adding one is adding a file — there is no
dispatcher to edit.
Each addon is its own idempotent script, and `ADDONS` names the ones to install,
in order. Adding one is adding a file — there is no dispatcher to edit. An
overlay's `addons/<name>.sh` is found before rig's own.
**There is no ingress controller, deliberately.** They pin a narrow window of
Kubernetes versions, so depending on one would constrain which k8s a rig can be
built with — and running a trailing-edge control plane to model a legacy estate
is the whole point. Services are reached through MetalLB and
`type: LoadBalancer`, which carries no such constraint and is also what a real
cluster does.
built with — and running a trailing-edge control plane is often the point.
Services are reached through MetalLB and `type: LoadBalancer`, which carries no
such constraint and is also what a real cluster does.
rig's own addons make the cluster work:
| Addon | Does |
| --- | --- |
| `metallb` | gives `type: LoadBalancer` an address it can actually reach |
| `cert-manager` | a local CA, so TLS works offline |
| `metrics-server` | makes `kubectl top` work on kind |
| `postgres` | database, in the `data` namespace |
| `redis` | cache and broker |
| `airflow` | scheduled pipelines; needs postgres and redis |
The last three are **cabinets**: a public service dropped in as-is, the upstream
image unmodified, reachable at a known address. A cabinet is declared once and
installs on either target — a `service.yml` composes it for a laptop, and these
install the same one here. The names match on purpose: each cabinet carries a
`rig_addon` field pointing at `ctrl/addons/<name>.sh`.
Plain manifests rather than helm charts, like every other addon: a chart repo is
a network dependency, and the `offline` profile exists precisely so there is a
path with none. Images are pinned in `ctrl/versions.env` and can be preloaded.
Passwords are generated on first install and kept across re-runs, so re-running
an addon never rotates a credential out from under something already connected:
What a workload needs — a database, a cache, a scheduler — belongs to its
overlay. [`examples/data`](examples/data/) carries postgres, redis and airflow as
plain manifests (no helm: a chart repo is a network dependency), with passwords
generated on first install and kept across re-runs:
```bash
kubectl -n data get secret postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d
kubectl -n data port-forward svc/airflow 8080:8080
OVERLAY=examples/data make cluster up
```

68
rig/STALE.md Normal file
View File

@@ -0,0 +1,68 @@
# rig — withdrawn assumptions
**Everything in this file is no longer true.**
It exists so the live docs stay short and a withdrawn assumption cannot quietly
return: each entry carries a **check**, and `ctrl/selftest.sh` runs every one of
them under "withdrawn stays withdrawn". A retraction that is only prose is one
nobody re-reads.
- **Do not restate these** in a plan, a README or a comment; point here (`✖ S2`).
- **Ids are stable.**
- **Only withdrawn things belong here.** A warning that is still actionable is a
live rule and stays where it is.
---
**✖ S1 — "rig supplies `ctrl/Tiltfile` but does not own it: replace the examples and
add your images in its marked sections."** *(ctrl/Tiltfile, README, 2026-09-13)*
Withdrawn 2026-09-22. Every project that used rig edited rig's own file, so no
update to rig could land without merging those edits by hand. rig's `ctrl/Tiltfile`
is now rig's: identity, context guard, registry, manifests, namespaces. The
workload's half is the overlay's own `Tiltfile`, which rig's includes.
**Check:** `ctrl/Tiltfile` includes the overlay's Tiltfile and has no "Images" section
of its own.
**✖ S2 — "A second environment is a copy of rig renamed after it (`../<name>-rig`), and
its use case is written into the copy."** *(README, BOOTSTRAP, .gitignore, 2026-08)*
Withdrawn 2026-09-22. A use case is an overlay, kept outside rig's version control
(`docs/notes/overlay.md`); rig itself is replaced as a whole. Copying rig still gives
a separate environment, but carries no use case of its own.
**Check:** rig's `.gitignore` ignores `/local/`; no `acme` example name remains in rig.
**✖ S3 — "rig's addons include the services a workload needs (a database, a cache, a
scheduler), described in the host project's vocabulary."** *(ctrl/addons/, README,
versions.env, 2026-08)* Withdrawn 2026-09-22. Which services a workload needs is
not rig's business. `ctrl/addons/` holds what makes a cluster work; workload addons
live with overlays, and `examples/data` carries them as an example.
**Check:** `ctrl/addons/` holds exactly cert-manager, metallb and metrics-server;
`versions.env` pins no workload image.
**✖ S4 — "The dev loop's namespace is named after the cluster."** *(ctrl/Tiltfile,
2026-09-13)* Withdrawn 2026-09-22. The Tiltfile created and grouped
`<CLUSTER>:namespace` while the example manifests declared `rig`, so every copy not
named `rig` stopped at load: `No object identified by the fragment
"acme-rig:namespace"`. The first real use worked around it with a Namespace named
`to_be_replaced` that its overlay renamed. The Tiltfile now creates the namespaces
the manifests use and groups the ones they declare.
**Check:** `ctrl/Tiltfile` does not build a namespace name from `CLUSTER`.
**✖ S5 — "`MANIFESTS_DIR` defaults to `ctrl/k8s/overlays/dev`, rig's own examples."**
*(lib/config.sh, .env.example, 2026-09-13)* Withdrawn 2026-09-22. The default is the
overlay's `k8s/overlays/dev`; rig's examples are `examples/starter`. The old value,
still pinned by older `.env` files, is ignored while that folder does not exist, and
`make check` says to delete it.
**Check:** `ctrl/k8s/overlays` does not exist; `.env.example` does not set
`MANIFESTS_DIR`.
**✖ S6 — "The example profiles are `client`, `data` and `offline`."** *(ctrl/env.d/,
2026-09-17)* Withdrawn 2026-09-22. "client" names the customer, not a registry
mode: the example is `mirror`. `data` was a workload, not a way of reaching the
world: it is the `examples/data` overlay.
**Check:** `ctrl/env.d/` holds no `client` or `data` example.
**✖ S7 — "BOOTSTRAP spans three repos: rig prepares the machine, the house repos own the
project's shape and everything after local."** *(BOOTSTRAP.md, 2026-08)* Withdrawn
2026-09-22. rig's docs describe rig. The house scaffold and registration sections
moved out of rig; BOOTSTRAP now ends with starting an overlay.
**Check:** no house path or host name (`semester`, `local.ar`) in rig.

View File

@@ -6,6 +6,11 @@
# need no profile at all. Copy an env.d/*.env.example to <name>.env to add one.
PROFILE=
# The overlay: one folder, outside rig's version control, holding what runs —
# its settings (rig.env), manifests, addons, Tiltfile. Relative to rig's folder,
# or absolute. Unset: rig's own examples/starter. See docs/notes/overlay.md.
# OVERLAY=local/<name>
# Cluster name; the kubectl context becomes kind-<CLUSTER>.
# LEAVE UNSET: it defaults to this folder's name, which keeps the folder copyable.
# CLUSTER=
@@ -17,9 +22,8 @@ PROFILE=
# TILT_PORT=
# REGISTRY_PORT=
# Where the application manifests live; repoint at their own repo, e.g.
# MANIFESTS_DIR=../platform-manifests/overlays/dev
MANIFESTS_DIR=ctrl/k8s/overlays/dev
# Where the manifests live. Leave unset: the overlay's k8s/overlays/dev.
# MANIFESTS_DIR=../platform-manifests/overlays/dev
# Where the installer fetches the pinned binaries from:
# upstream (needs internet) | artifactory (generic repo) | baked (in the image)

View File

@@ -27,8 +27,10 @@ CMD ["install"]
# ---------------------------------------------------------------------------
# deps-full — same image, binaries baked in, works with no network at all.
# deps-full — same image, binaries and the addons' manifests baked in, works with no network.
FROM deps AS deps-full
RUN /work/rigdeps.sh fetch --to /opt/rig/bin
RUN /work/rigdeps.sh fetch --to /opt/rig/bin \
&& /work/rigdeps.sh manifests --to /opt/rig/manifests
ENV DEPS_SOURCE=baked \
BAKED_BIN=/opt/rig/bin
BAKED_BIN=/opt/rig/bin \
BAKED_MANIFESTS=/opt/rig/manifests

View File

@@ -1,6 +1,6 @@
# The dev loop. `make tilt` from the project root, or `cd ctrl && tilt up`.
# Works unedited; replace the two EXAMPLES and add yours in the marked sections.
# rig supplies this file but does not own it, and nothing here names this directory.
# The dev loop, rig's half. `make tilt` from rig's folder, or from an overlay's forwarder.
# rig owns this file: who we are, the context guard, the registry, the overlay's manifests.
# The workload's half is the overlay's own Tiltfile, included at the end; edit that one.
# Notes: docs/notes/Tiltfile.md
# ── who we are, and on which ports ─────────────────────────────────────────
@@ -12,10 +12,9 @@ HTTP = _facts[2]
HTTPS = _facts[3]
TILT = _facts[4]
REGISTRY = _facts[5]
# Where the manifests live (MANIFESTS_DIR, see k8s/README.md).
# The value is REPO-ROOT relative and this file runs in ctrl/, so prefix '../'.
MANIFESTS = '../' + _facts[6]
# Absolute paths, or '-' when there is none.
MANIFESTS = '' if _facts[6] == '-' else _facts[6]
OVERLAY = '' if _facts[7] == '-' else _facts[7]
# ── refuse to deploy into the wrong cluster ────────────────────────────────
# Tilt fixes the context before parsing this file, so it can only be refused here.
@@ -25,73 +24,44 @@ if k8s_context() != CTX:
fail("Wrong kubectl context: '%s'. This is %s — run: make tilt, or tilt up --context %s"
% (k8s_context(), CLUSTER, CTX))
# The namespace has to exist before anything lands in it, and kustomize does not
# guarantee ordering across resources. Creating it here is idempotent.
local('kubectl --context %s create namespace %s --dry-run=client -o yaml | kubectl --context %s apply -f -'
% (CTX, CLUSTER, CTX), quiet=True)
# ── images go to this environment's own registry ───────────────────────────
# Fail closed: name the registry rather than let Tilt infer it, or a miss pushes
# an unqualified image to docker.io.
default_registry('localhost:' + REGISTRY)
k8s_yaml(kustomize(MANIFESTS))
# ── Images ─────────────────────────────────────────────────────────────────
# (nothing yet — rig's examples run upstream images. Add docker_build calls here.)
# ── Resources ──────────────────────────────────────────────────────────────
# (nothing yet — add k8s_resource calls here to name and order what you deploy.)
# Everything with no dev loop of its own, gathered so it does not clutter the UI.
k8s_resource(
objects=[CLUSTER + ':namespace'],
new_name='infra',
)
# ── the overlay's manifests ────────────────────────────────────────────────
# Every namespace they use must exist before anything lands in it, and kustomize
# does not order resources, so create them here (idempotent). The Namespaces they
# declare are grouped as 'infra', whatever they are named.
if MANIFESTS:
_yaml = kustomize(MANIFESTS)
k8s_yaml(_yaml)
_declared = []
_namespaces = {}
for _o in decode_yaml_stream(_yaml):
if not _o:
continue
_md = _o.get('metadata') or {}
if _o.get('kind') == 'Namespace':
_declared.append(_md.get('name'))
_namespaces[_md.get('name')] = True
elif _md.get('namespace'):
_namespaces[_md.get('namespace')] = True
for _ns in sorted(_namespaces.keys()):
local('kubectl --context %s create namespace %s --dry-run=client -o yaml | kubectl --context %s apply -f -'
% (CTX, _ns, CTX), quiet=True)
if _declared:
k8s_resource(objects=[_n + ':namespace' for _n in _declared], new_name='infra')
# ═══════════════════════════════════════════════════════════════════════════
# Catalogue — paste what you need, delete the rest.
# Commented out so this file runs as-is.
# ═══════════════════════════════════════════════════════════════════════════
#
# ── build an image ─────────────────────────────────────────────────────────
# context= is the REPO ROOT ('..'); dockerfile= is relative to THIS file (ctrl/).
# So every COPY is repo-root relative, even for files beside the Dockerfile.
#
# docker_build(
# CLUSTER + '-api', # must match `image:` in the manifest —
# context='..', # that string is the only thing
# dockerfile='Dockerfile.api', # connecting the two
# ignore=['.git', 'def', '.venv', 'node_modules', '__pycache__'],
# live_update=[sync('../api', '/app/api')],
# )
#
# ── name and order a resource ──────────────────────────────────────────────
# k8s_resource('api', resource_deps=['postgres'], labels=['app'])
# k8s_resource('gateway', resource_deps=['api', 'ui'], labels=['app'])
#
# ── reload the gateway when its config changes ─────────────────────────────
# A hash-less configMapGenerator ConfigMap never changes name, so edits do NOT
# roll the pod on their own.
#
# local_resource(
# 'gateway-reload',
# cmd='kubectl --context %s -n %s rollout restart deployment/gateway' % (CTX, CLUSTER),
# deps=['k8s/base/Caddyfile'],
# resource_deps=['gateway'],
# auto_init=False,
# )
#
# ── an overlay whose secretGenerator reads outside its own directory ───────
# kustomize refuses to read above the kustomization root unless told to. Only
# add this if you actually have such a generator; it loosens a safety check.
#
# k8s_yaml(kustomize(MANIFESTS, flags=['--load-restrictor=LoadRestrictionsNone']))
#
# ── reach a service directly, bypassing the gateway ────────────────────────
# Prefer the gateway; host ports are shared machine-wide. If you need one, take it
# from this environment's own port block.
#
# k8s_resource('postgres', port_forwards=[str(int(HTTP) + 5) + ':5432'])
# ── the workload's half: the overlay's Tiltfile ────────────────────────────
# Included, so its relative paths resolve from the overlay's own folder. It reads
# these facts with os.getenv and never needs a path back into rig.
os.putenv('RIG_CLUSTER', CLUSTER)
os.putenv('RIG_CONTEXT', CTX)
os.putenv('RIG_HTTP_PORT', HTTP)
os.putenv('RIG_HTTPS_PORT', HTTPS)
os.putenv('RIG_TILT_PORT', TILT)
os.putenv('RIG_REGISTRY', 'localhost:' + REGISTRY)
os.putenv('RIG_OVERLAY_DIR', OVERLAY)
if OVERLAY and os.path.exists(OVERLAY + '/Tiltfile'):
include(OVERLAY + '/Tiltfile')

View File

@@ -1,6 +1,6 @@
#!/usr/bin/env bash
# Install the addons the active profile asked for, in the order listed.
# One idempotent script per addon in ctrl/addons/.
# Install the addons the configuration asks for (ADDONS), in the order listed.
# One idempotent script per addon: the overlay's addons/<name>.sh first, then rig's ctrl/addons/.
# Usage: addons.sh install | list
# Notes: docs/notes/addons.md
set -euo pipefail
@@ -9,26 +9,51 @@ cd "$(dirname "$0")"
source ./lib/config.sh
load_config
# Every addon runs from here, wherever its file lives, so it can source ./lib/config.sh.
export RIG_CTRL="$PWD"
# The script for one addon name: the overlay's, else rig's; empty if neither.
addon_path() {
local ov=""
if [ -n "$OVERLAY_DIR" ]; then ov="$(_from_ctrl "$OVERLAY_DIR")/addons/$1.sh"; fi
if [ -n "$ov" ] && [ -f "$ov" ]; then
echo "$ov"
elif [ -f "addons/$1.sh" ]; then
echo "addons/$1.sh"
fi
}
install() {
if [ -z "${ADDONS// /}" ]; then
echo "no addons in profile '$PROFILE_NAME'"
echo "no addons asked for (ADDONS is empty)"
return
fi
local a
local a p
for a in $ADDONS; do
if [ ! -f "addons/${a}.sh" ]; then
echo "no such addon: addons/${a}.sh" >&2
p=$(addon_path "$a")
if [ -z "$p" ]; then
echo "no such addon: $a (looked in the overlay's addons/ and ctrl/addons/)" >&2
exit 1
fi
echo "addon: $a"
bash "addons/${a}.sh"
bash "$p"
done
}
list() {
echo "profile '$PROFILE_NAME' wants: ${ADDONS:-none}"
echo "wanted: ${ADDONS:-none}"
echo "available:"
ls addons/*.sh 2>/dev/null | xargs -n1 basename | sed 's/\.sh$//' | sed 's/^/ /'
local f
if [ -n "$OVERLAY_DIR" ]; then
for f in "$(_from_ctrl "$OVERLAY_DIR")"/addons/*.sh; do
[ -e "$f" ] || continue
printf ' %-16s overlay\n' "$(basename "$f" .sh)"
done
fi
for f in addons/*.sh; do
[ -e "$f" ] || continue
printf ' %-16s rig\n' "$(basename "$f" .sh)"
done
}
case "${1:-list}" in

View File

@@ -2,7 +2,7 @@
# cert-manager plus a self-signed cluster issuer (offline local CA).
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."
cd "${RIG_CTRL:-$(dirname "$0")/..}"
source ./lib/config.sh
load_config
@@ -12,7 +12,9 @@ K="kubectl --context ${KUBECONTEXT}"
if $K get deployment -n cert-manager cert-manager >/dev/null 2>&1; then
echo " already installed"
else
$K apply -f "https://github.com/cert-manager/cert-manager/releases/download/${CERT_MANAGER_VERSION}/cert-manager.yaml"
# The pinned manifest, verified on disk — never a URL applied directly.
manifest=$(bash ./deps.sh manifest CERT_MANAGER)
$K apply -f "$manifest"
fi
echo " waiting for cert-manager..."

View File

@@ -3,7 +3,7 @@
# The pool is derived from the kind Docker network at install time.
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."
cd "${RIG_CTRL:-$(dirname "$0")/..}"
source ./lib/config.sh
load_config
@@ -46,7 +46,9 @@ echo " kind network $subnet → pool ${pool_start}-${pool_end}"
if $K get deployment -n metallb-system controller >/dev/null 2>&1; then
echo " already installed"
else
$K apply -f "https://raw.githubusercontent.com/metallb/metallb/${METALLB_VERSION}/config/manifests/metallb-native.yaml"
# The pinned manifest, verified on disk — never a URL applied directly.
manifest=$(bash ./deps.sh manifest METALLB)
$K apply -f "$manifest"
fi
# `rollout status`, not `kubectl wait`: wait errors out while the pod doesn't exist yet.

View File

@@ -2,7 +2,7 @@
# metrics-server — makes `kubectl top` work (patched with --kubelet-insecure-tls for kind).
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."
cd "${RIG_CTRL:-$(dirname "$0")/..}"
source ./lib/config.sh
load_config
@@ -10,7 +10,9 @@ load_config
K="kubectl --context ${KUBECONTEXT}"
if ! $K get deployment -n kube-system metrics-server >/dev/null 2>&1; then
$K apply -f "https://github.com/kubernetes-sigs/metrics-server/releases/download/${METRICS_SERVER_VERSION}/components.yaml"
# The pinned manifest, verified on disk — never a URL applied directly.
manifest=$(bash ./deps.sh manifest METRICS_SERVER)
$K apply -f "$manifest"
fi
$K patch deployment metrics-server -n kube-system --type=json \

View File

@@ -69,10 +69,34 @@ port_busy() {
echo
echo "rig"
echo " cluster ${CLUSTER} (${KUBECONTEXT}) profile ${PROFILE_NAME}, ${NODES} node(s), registry ${REGISTRY_MODE}"
if [ -n "${OVERLAY:-}" ]; then
echo " overlay $(basename "$(_abs_from_ctrl "$OVERLAY_DIR")") ($(_abs_from_ctrl "$OVERLAY_DIR"))"
elif [ -n "$OVERLAY_DIR" ]; then
fact " overlay none named — rig's own ${OVERLAY_DIR}"
fi
if [ -n "$VERBOSE" ] && [ -n "$OVERLAY_DIR" ]; then
ov=$(_from_ctrl "$OVERLAY_DIR") pieces=""
for piece in rig.env k8s/overlays/dev kind-config.yaml.tpl addons Tiltfile; do
[ -e "$ov/$piece" ] && pieces+="$piece "
done
echo " provides: ${pieces:-nothing rig reads}"
fi
fact " manifests ${MANIFESTS_DIR:-none}"
fact " kind config ${KIND_CONFIG}"
fact " ingress ${INGRESS_MODE}"
if [ ! -f ./.env ]; then
fact " .env none — built-in defaults (cp ctrl/.env.example ctrl/.env to set values)"
fi
if [ -n "${STALE_MANIFESTS_DIR:-}" ]; then
echo " ! .env MANIFESTS_DIR=${STALE_MANIFESTS_DIR} is the old default; rig's examples moved"
echo " to examples/ — delete that line from ctrl/.env (ignored until then)"
fi
# registry.sh points containerd at certs.d, which only works if the kind config says so,
# and a kind config is fixed at creation: a project's own file that drops it fails silently.
if [ "$REGISTRY_MODE" != none ] && ! grep -q 'config_path *= *"/etc/containerd/certs.d"' "$KIND_CONFIG"; then
echo " ! kind ${KIND_CONFIG} lacks the containerd config_path patch that registry mode"
echo " '${REGISTRY_MODE}' needs — copy it from ctrl/k8s/kind-config.yaml.tpl"
fi
# ── memory: does this cluster fit right now? Warns; never blocks. ──────────
total_mb=$(mb_of MemTotal)
@@ -157,7 +181,11 @@ elif [ "$mine" -eq 1 ]; then
else
echo " ports ${list% } free"
fi
fact " derived from the directory name; pin them: bash ctrl/ports.sh persist"
if [ -n "${OVERLAY:-}" ]; then
fact " derived from the overlay's folder name"
else
fact " derived from the directory name; pin them: bash ctrl/ports.sh persist"
fi
# ── what `make cluster up` wires in beside the cluster ─────────────────────
REG_NAME="${CLUSTER}-registry"
@@ -167,7 +195,9 @@ else
fact " registry no container yet — 'make cluster up' starts it"
fi
echo " addons ${ADDONS:-none}"
fact " available: $(ls addons/*.sh 2>/dev/null | xargs -n1 basename | sed 's/\.sh$//' | tr '\n' ' ')"
if [ -n "$VERBOSE" ]; then
bash ./addons.sh list | sed -n '3,$p' | sed 's/^/ /'
fi
# The CA reaches three places and only one of them is ours. Report the other two.
if [ -n "${REGISTRY_CA_FILE:-}" ]; then

View File

@@ -15,6 +15,7 @@ up() {
# Say what this profile locks in BEFORE spending minutes building it:
# the kind config is fixed at creation and cannot be changed later.
echo "creating cluster '$CLUSTER' from profile '$PROFILE_NAME'"
echo " overlay ${OVERLAY_DIR:-none}"
echo " kind config ${KIND_CONFIG}"
echo " nodes $NODES"
echo " image $NODE_IMAGE"

View File

@@ -2,7 +2,8 @@
# rig:standalone rigdeps detect
# Toolchain installer: detect the host, install pinned tools into $OUT_BIN, report
# host actions it will not perform (no sudo, no apt). Usually via `make deps`.
# Usage: deps.sh [detect [all] | list | verify [core|dev] | fetch [core|dev] [--to DIR] | install [core|dev]]
# Usage: deps.sh [detect [all] | list | verify [core|dev] | fetch [core|dev] [--to DIR] | install [core|dev]
# | manifest NAME | manifests [--to DIR]]
# Notes: docs/notes/deps.md
set -euo pipefail
@@ -719,6 +720,51 @@ require_linux
cmd="${1:-install}"
[ $# -gt 0 ] && shift
# ── manifests rig's own addons apply ───────────────────────────────────────
# Pinned (URL + SHA256), fetched through the same DEPS_SOURCE resolver as the
# binaries and verified, then applied from disk: an offline machine needs no
# network for them. Default home: vendor/manifests/ in rig's folder (gitignored).
MANIFESTS_HOME="${MANIFESTS_HOME:-$(cd .. && pwd)/vendor/manifests}"
BAKED_MANIFESTS="${BAKED_MANIFESTS:-/opt/rig/manifests}"
MANIFEST_NAMES="METALLB CERT_MANAGER METRICS_SERVER"
manifest_path() { # NAME dir
local v="${1}_VERSION"
echo "$2/$(echo "$1" | tr 'A-Z_' 'a-z-')-${!v}.yaml"
}
# Make one pinned manifest present and verified in dir; print only its path.
fetch_manifest() { # NAME dir
local name="$1" dir="$2" url_var="${1}_MANIFEST_URL" sha_var="${1}_MANIFEST_SHA256" file
if [ -z "${!url_var:-}" ] || [ -z "${!sha_var:-}" ]; then
echo "no pinned manifest for $name (${url_var} / ${sha_var} unset)" >&2
exit 1
fi
file=$(manifest_path "$name" "$dir")
if [ -f "$file" ] && [ "$($SHA "$file" | awk '{print $1}')" = "${!sha_var}" ]; then
echo "$file"
return
fi
mkdir -p "$dir"
if [ "$DEPS_SOURCE" = baked ]; then
cp "$(manifest_path "$name" "$BAKED_MANIFESTS")" "$file.tmp"
else
download "$(resolve_url "${!url_var}")" "$file.tmp"
fi
verify "$file.tmp" "${!sha_var}" "$name manifest"
mv "$file.tmp" "$file"
echo "$file"
}
fetch_manifests() { # [--to DIR]
local dest="$MANIFESTS_HOME" n
if [ "${1:-}" = --to ]; then dest="$(abspath "${2:?--to needs a directory}")"; fi
echo "fetching the addons' manifests into $dest (source: $DEPS_SOURCE)"
for n in $MANIFEST_NAMES; do
echo " $n $(fetch_manifest "$n" "$dest")"
done
}
# Baked mode copies binaries already in the image, so it needs no downloader.
need_downloads() {
require_amd64
@@ -732,9 +778,13 @@ case "$cmd" in
verify) verify_tools "${1:-dev}" ;;
fetch) need_downloads; fetch "$@" ;;
install) need_downloads; install "${1:-dev}" ;;
*) echo "usage: $0 [detect [all]|list|verify|fetch|install]" >&2
manifest) need_downloads
fetch_manifest "${1:?usage: $0 manifest <METALLB|CERT_MANAGER|METRICS_SERVER>}" "$MANIFESTS_HOME" ;;
manifests) need_downloads; fetch_manifests "$@" ;;
*) echo "usage: $0 [detect [all]|list|verify|fetch|install|manifest NAME|manifests]" >&2
echo " install [core|dev] (default dev)" >&2
echo " fetch [core|dev] [--to DIR]" >&2
echo " manifests [--to DIR] the addons' pinned manifests, verified" >&2
echo " OUT_BIN=<dir> overrides the install directory" >&2
exit 1 ;;
esac

View File

@@ -1,26 +0,0 @@
# EXAMPLE PROFILE (optional): copy to data.env, then PROFILE=data; overlays the defaults.
# data — postgres, redis and airflow (upstream images) in the `data` namespace.
# Notes: docs/notes/env.md
PROFILE_NAME=data
K8S_VERSION=v1_36
# Order matters: addons.sh installs in the order listed, and airflow refuses to
# start without a metadata database, so postgres comes first.
ADDONS="metallb postgres redis airflow"
# local, not none — see the defaults in lib/config.sh: `none` has no outward-push guard.
REGISTRY_MODE=local
INGRESS_MODE=hostport
DNS_MODE=hosts
# Namespace for the dependency containers.
DATA_NAMESPACE=data
# Postgres identity. The password is generated once by postgres.sh and kept.
POSTGRES_DB=app
POSTGRES_USER=app
POSTGRES_STORAGE=2Gi
AIRFLOW_ADMIN_USER=admin
# Reach the databases with port-forward, e.g.
# kubectl -n data port-forward svc/postgres 5432:5432

View File

@@ -1,8 +1,9 @@
# EXAMPLE PROFILE (optional): copy to client.env, then PROFILE=client; overlays the defaults.
# client — images via a pull-through cache of the corporate registry, TLS and metrics addons.
# EXAMPLE PROFILE (optional): copy to mirror.env, then PROFILE=mirror; overlays the defaults.
# mirror — images via a pull-through cache of an internal registry, TLS and metrics addons.
# A profile says how this machine reaches the world; what runs is an overlay's business.
# Notes: docs/notes/env.md
PROFILE_NAME=client
PROFILE_NAME=mirror
K8S_VERSION=v1_36
ADDONS="metallb cert-manager metrics-server"
REGISTRY_MODE=mirror
@@ -15,6 +16,6 @@ DNS_MODE=hosts
# HTTPS_PORT=443
# Set these in ctrl/.env (gitignored), not here:
# REGISTRY_REMOTE_URL=https://artifactory.corp.example/artifactory/api/docker/docker-virtual
# REGISTRY_REMOTE_URL=https://registry.internal.example/api/docker/docker-virtual
# REGISTRY_USER / REGISTRY_PASSWORD
# REGISTRY_CA_FILE=/path/to/corp-root-ca.crt
# REGISTRY_CA_FILE=/path/to/internal-root-ca.crt

View File

@@ -1,60 +0,0 @@
# `ctrl/k8s` — the cluster, and what runs on it
Same layout as every other project here: a kind config, a kustomize `base/`,
and an `overlays/dev/` that patches it.
```
kind-config.yaml.tpl the cluster itself — nodes, ports
base/ the components, as plain manifests
overlays/dev/ how this rig differs from the base
```
## Why the cluster config is a template
Every other project checks in a literal `kind-config.yaml`, because there is
exactly one `unt` and one `nvi`. A rig is copied and renamed to make a second
environment, and both the cluster name and the host port block follow the
directory name — so a literal would make every copy collide on both.
`ctrl/cluster.sh` renders it with `sed`, substituting `${CLUSTER}`,
`${NODE_IMAGE}`, `${HTTP_PORT}` and `${HOST_WORKDIR}`. Not `envsubst`: that is
`gettext-base`, which a minimal Debian does not have, and Docker being the only
prerequisite is the one promise rig makes.
**To change the cluster, edit this file** — more nodes, other port mappings —
then `make cluster reset`: a kind config is fixed at creation, not re-applied.
`lib/config.sh` reads the node count back out of it, so nothing restates it.
A project that builds its own cluster through rig passes its own file as
`KIND_CONFIG=<path>`; it is rendered the same way.
## `base/` — replace these
**The two components in `base/` are examples, not the system.** They exist so
the real manifests have a shape to be written against.
The real ones are expected to be versioned **separately from the installer**
they change on a different cadence, by different people, under different review.
Point `MANIFESTS_DIR` in `ctrl/.env` at their overlay and rig stops owning them:
```
MANIFESTS_DIR=../platform-manifests/overlays/dev
```
Until then it defaults to `ctrl/k8s/overlays/dev`.
### The three states a component can be in
Switching between them should be a one-line change, never a rewrite. The DNS
name stays the same in every case, so callers never know the difference:
| state | what exists | when |
| --- | --- | --- |
| **real** | an image built from source, hot-reloaded | the one thing you are working on |
| **mock** | a stub returning canned responses (`example-mock.yaml`) | everything else — most of the estate |
| **remote** | no pod at all, just a Service (`example-remote.yaml`) | when the real system is reachable and you want it |
Most components should be **mock**. What has to be faithful is the topology —
names, ports, dependency order, who can reach whom, how it fails. The workloads
are noise, and mocking them is what makes several copies of a large estate fit
on one laptop.

View File

@@ -1,5 +1,5 @@
# The cluster. Add nodes or port mappings here, then `make cluster reset`.
# ctrl/cluster.sh substitutes (sed): CLUSTER, NODE_IMAGE, HTTP_PORT, HOST_WORKDIR
# ctrl/cluster.sh substitutes (sed): CLUSTER, NODE_IMAGE, HTTP_PORT, HOST_WORKDIR, OVERLAY_DIR
# lib/config.sh reads the node count back from this file.
# Notes: docs/notes/kind-config.md
kind: Cluster

View File

@@ -1,19 +1,34 @@
# Shared config loading: how the config layers compose. Sourced, never executed.
# Precedence, weakest first: defaults < versions.env < env.d/<profile> < .env < caller's env.
# Precedence, weakest first:
# defaults < versions.env < env.d/<profile> < <overlay>/rig.env < .env < caller's env.
# Run from ctrl/.
# Notes: docs/notes/config.md
# Per-invocation overrides: restored after the files are read, so the caller wins.
# NODES is deliberately not here (read from the kind config).
CONFIG_OVERRIDABLE="PROFILE CLUSTER K8S_VERSION KIND_CONFIG ADDONS
CONFIG_OVERRIDABLE="PROFILE OVERLAY CLUSTER K8S_VERSION KIND_CONFIG ADDONS
REGISTRY_MODE INGRESS_MODE DNS_MODE TILT_PORT
SOURCE ARCH DEPS_SOURCE HTTP_PORT HTTPS_PORT
REGISTRY_PORT MANIFESTS_DIR"
# The repo folder's name, reduced to a DNS label kind accepts as a cluster name.
# rig's own example, used when no overlay is named (relative to rig's root).
DEFAULT_OVERLAY=examples/starter
# A rig-root-relative path as seen from ctrl/; absolute paths pass through.
_from_ctrl() { case "$1" in /*) echo "$1" ;; *) echo "../$1" ;; esac; }
# The same, absolute. Empty if it does not exist.
_abs_from_ctrl() { (cd "$(_from_ctrl "$1")" 2>/dev/null && pwd); }
# The environment's folder — the overlay's when one is named, else rig's —
# reduced to a DNS label kind accepts as a cluster name.
default_cluster_name() {
local n
n=$(basename "$(cd .. && pwd)")
if [ -n "${OVERLAY:-}" ]; then
n=$(basename "$(_abs_from_ctrl "$OVERLAY_DIR")")
else
n=$(basename "$(cd .. && pwd)")
fi
n=$(echo "$n" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9-' '-')
n=$(echo "$n" | sed 's/^-*//; s/-*$//')
echo "${n:-rig}"
@@ -45,15 +60,59 @@ load_config() {
_config_restore "$saved"
# A profile is optional; naming one that does not exist is an error.
local profile="${PROFILE:-}"
local profile="${PROFILE:-}" layered=""
if [ -n "$profile" ] && [ "$profile" != default ]; then
if [ ! -f "./env.d/${profile}.env" ]; then
echo "no such profile: env.d/${profile}.env" >&2
if [ -d "../examples/${profile}" ]; then
echo " it is an example overlay now: OVERLAY=examples/${profile}" >&2
fi
echo "available: $(config_profiles | tr '\n' ' ')" >&2
exit 1
fi
set -a
source "./env.d/${profile}.env"
set +a
layered=1
fi
# The overlay: one folder, outside rig, holding a use case (docs/notes/overlay.md).
# Named ones must exist; with none named, rig's own example is used if present.
OVERLAY_DIR=""
if [ -n "${OVERLAY:-}" ]; then
OVERLAY_DIR="${OVERLAY%/}"
if [ ! -d "$(_from_ctrl "$OVERLAY_DIR")" ]; then
echo "no overlay at OVERLAY=${OVERLAY} (relative to rig's folder, or absolute)" >&2
exit 1
fi
elif [ -d "../${DEFAULT_OVERLAY}" ]; then
OVERLAY_DIR="$DEFAULT_OVERLAY"
fi
# Its rig.env may not choose the profile or the overlay (both are chosen before
# it loads), and the paths it sets are relative to the overlay.
local ov_env="" m_before k_before
if [ -n "$OVERLAY_DIR" ]; then ov_env="$(_from_ctrl "$OVERLAY_DIR")/rig.env"; fi
if [ -n "$ov_env" ] && [ -f "$ov_env" ]; then
if grep -qE '^[[:space:]]*(export[[:space:]]+)?(PROFILE|OVERLAY)=' "$ov_env"; then
echo "$ov_env: an overlay's rig.env cannot set PROFILE or OVERLAY (they choose it)" >&2
exit 1
fi
m_before="${MANIFESTS_DIR-}" k_before="${KIND_CONFIG-}"
set -a
source "$ov_env"
set +a
if [ "${MANIFESTS_DIR-}" != "$m_before" ]; then
case "$MANIFESTS_DIR" in /*|none|"") ;; *) MANIFESTS_DIR="${OVERLAY_DIR}/${MANIFESTS_DIR}" ;; esac
fi
if [ "${KIND_CONFIG-}" != "$k_before" ]; then
case "$KIND_CONFIG" in /*|"") ;; *) KIND_CONFIG="$(dirname "$ov_env")/${KIND_CONFIG}" ;; esac
fi
layered=1
fi
# The machine and the caller still win over both.
if [ -n "$layered" ]; then
set -a
if [ -z "${RIG_PORTABLE:-}" ] && [ -f ./.env ]; then source ./.env; fi
set +a
_config_restore "$saved"
@@ -86,9 +145,25 @@ load_config() {
TILT_PORT="${TILT_PORT:-$((base + 2))}"
REGISTRY_PORT="${REGISTRY_PORT:-$((base + 3))}"
# Where the workload's manifests live, repo-root relative; always resolved.
# See k8s/README.md.
MANIFESTS_DIR="${MANIFESTS_DIR:-ctrl/k8s/overlays/dev}"
# Where the workload's manifests live, relative to rig's folder (or absolute):
# the overlay's k8s/overlays/dev unless something names another. `none`: rig
# applies none (the overlay's Tiltfile does). A named folder must exist.
if [ "${MANIFESTS_DIR:-}" = ctrl/k8s/overlays/dev ] && [ ! -d ../ctrl/k8s/overlays/dev ]; then
# The old default, pinned by an older .env.example; rig's examples moved.
STALE_MANIFESTS_DIR="$MANIFESTS_DIR"
MANIFESTS_DIR=""
fi
if [ -z "${MANIFESTS_DIR:-}" ] && [ -n "$OVERLAY_DIR" ] \
&& [ -d "$(_from_ctrl "$OVERLAY_DIR")/k8s/overlays/dev" ]; then
MANIFESTS_DIR="$OVERLAY_DIR/k8s/overlays/dev"
fi
MANIFESTS_DIR="${MANIFESTS_DIR:-}"
if [ "$MANIFESTS_DIR" = none ]; then
MANIFESTS_DIR=""
elif [ -n "$MANIFESTS_DIR" ] && [ ! -d "$(_from_ctrl "$MANIFESTS_DIR")" ]; then
echo "no manifests at MANIFESTS_DIR=${MANIFESTS_DIR} (relative to rig's folder, or absolute)" >&2
exit 1
fi
# Profiles name a k8s minor (v1_36); versions.env holds the pinned digest.
local var="NODE_IMAGE_${K8S_VERSION}"
@@ -98,9 +173,13 @@ load_config() {
exit 1
fi
# The cluster is one file: k8s/kind-config.yaml.tpl. To change it, edit it.
# KIND_CONFIG is only "use this file instead", for a project that builds its
# own cluster through rig (a path relative to ctrl/, or absolute).
# The cluster is one file: the overlay's kind-config.yaml.tpl if it has one,
# else rig's k8s/kind-config.yaml.tpl. KIND_CONFIG is "use this file instead",
# for a project that builds its own cluster through rig (relative to ctrl/, or absolute).
if [ -z "${KIND_CONFIG:-}" ] && [ -n "$OVERLAY_DIR" ] \
&& [ -f "$(_from_ctrl "$OVERLAY_DIR")/kind-config.yaml.tpl" ]; then
KIND_CONFIG="$(_from_ctrl "$OVERLAY_DIR")/kind-config.yaml.tpl"
fi
KIND_CONFIG="${KIND_CONFIG:-./k8s/kind-config.yaml.tpl}"
if [ ! -f "$KIND_CONFIG" ]; then
echo "no kind config at KIND_CONFIG=${KIND_CONFIG}" >&2
@@ -117,13 +196,15 @@ load_config() {
}
# Render the kind config to stdout with sed (not envsubst) over an explicit variable list.
# HOST_WORKDIR must be a host path: the host dockerd resolves hostPath entries.
# HOST_WORKDIR and OVERLAY_DIR must be host paths: the host dockerd resolves hostPath entries.
render_kind_config() {
local host_workdir="${HOST_WORKDIR:-$(cd .. && pwd)}"
local host_workdir="${HOST_WORKDIR:-$(cd .. && pwd)}" overlay_dir=""
if [ -n "$OVERLAY_DIR" ]; then overlay_dir=$(_abs_from_ctrl "$OVERLAY_DIR"); fi
sed -e "s|\${CLUSTER}|${CLUSTER}|g" \
-e "s|\${NODE_IMAGE}|${NODE_IMAGE}|g" \
-e "s|\${HTTP_PORT}|${HTTP_PORT}|g" \
-e "s|\${HOST_WORKDIR}|${host_workdir}|g" \
-e "s|\${OVERLAY_DIR}|${overlay_dir}|g" \
"$KIND_CONFIG"
}

View File

@@ -22,11 +22,19 @@ derive() {
}
# Resolved facts for consumers outside bash, space-separated, positional:
# CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR
# Read this, not `derive` (which ignores ctrl/.env).
# CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR OVERLAY_DIR
# The two paths are absolute, or - when there is none. Read this, not `derive`.
active() {
load_config
echo "$CLUSTER $KUBECONTEXT $HTTP_PORT $HTTPS_PORT $TILT_PORT $REGISTRY_PORT $MANIFESTS_DIR"
local m="-" o="-"
if [ -n "$MANIFESTS_DIR" ]; then m=$(_abs_from_ctrl "$MANIFESTS_DIR"); fi
if [ -n "$OVERLAY_DIR" ]; then o=$(_abs_from_ctrl "$OVERLAY_DIR"); fi
case "$m$o" in
*[[:space:]]*)
echo "a path here holds whitespace, and these facts are split on spaces: $m $o" >&2
exit 1 ;;
esac
echo "$CLUSTER $KUBECONTEXT $HTTP_PORT $HTTPS_PORT $TILT_PORT $REGISTRY_PORT $m $o"
}
show() {
@@ -55,6 +63,13 @@ _row() {
# rewritten — an override stays an override.
persist() {
derive
# ctrl/.env belongs to this rig, not to an overlay: a pin written now would
# follow every overlay this rig later runs, and two of them would collide.
if [ -n "${OVERLAY:-}" ]; then
echo "not pinning: OVERLAY is set, and ctrl/.env would carry this block to every overlay" >&2
echo " its ports stay derived from its folder name ($CLUSTER): $DERIVED_HTTP-$DERIVED_REGISTRY" >&2
exit 1
fi
[ -f ./.env ] || cp ./.env.example ./.env
local wrote=0 key val

View File

@@ -23,6 +23,17 @@ check() { # name, expected, actual
note() { printf '\n%s\n' "$1"; }
# A scratch copy of rig for a check to change freely. local/ (overlays, possibly
# someone else's) and def/ (scratch) never ride along, and neither do this
# machine's PROFILE/OVERLAY/CLUSTER choices: a check sets what it tests.
copy_rig() { # dest-dir
mkdir -p "$1"
tar -C .. --exclude=./local --exclude=./def -cf - . | tar -C "$1" -xf -
if [ -f "$1/ctrl/.env" ]; then
sed -i '/^PROFILE=/d; /^OVERLAY=/d; /^CLUSTER=/d; /^MANIFESTS_DIR=/d' "$1/ctrl/.env"
fi
}
# Resolve one key the way every rig script does, in a clean shell so the
# caller's exported value is the only thing in play.
resolved() {
@@ -33,12 +44,12 @@ resolved() {
note "rig needs no profile"
# No env.d/ must still resolve and generate a kit; an unknown profile stays an error.
NP="$(mktemp -d)"
cp -r .. "$NP/rig"; rm -rf "$NP/rig/ctrl/env.d"; sed -i '/^PROFILE=/d' "$NP/rig/ctrl/.env" 2>/dev/null
copy_rig "$NP/rig"; rm -rf "$NP/rig/ctrl/env.d"
check "no env.d: config resolves" "default" \
"$(cd "$NP/rig/ctrl" && bash -c 'source ./lib/config.sh; load_config >/dev/null && echo "$PROFILE_NAME"' 2>&1)"
check "no env.d: the k8s version comes from the pins" "yes" \
"$(cd "$NP/rig/ctrl" && bash -c 'source ./lib/config.sh; load_config >/dev/null && [ -n "$NODE_IMAGE" ] && echo yes' 2>&1)"
check "no env.d: ports.sh active works" "7" \
check "no env.d: ports.sh active works" "8" \
"$(cd "$NP/rig/ctrl" && bash ports.sh active 2>/dev/null | wc -w)"
check "no env.d: a kit is generated for the defaults" "yes" \
"$( (cd "$NP/rig/ctrl" && rm -rf ../standalone/*/ && bash standalone.sh write >/dev/null 2>&1) && [ -f "$NP/rig/standalone/default/rigdeps.sh" ] && echo yes || echo no)"
@@ -50,13 +61,14 @@ rm -rf "$NP"
note "the ports.sh active contract"
# ports.sh active is read positionally by the Makefile and Tiltfile: pin field count and order.
FACTS="$(bash ports.sh active)"
check "active: exactly 7 fields" "7" "$(printf '%s' "$FACTS" | wc -w)"
read -r F_CLUSTER F_CTX F_HTTP F_HTTPS F_TILT F_REG F_MANIFESTS <<< "$FACTS"
check "active: exactly 8 fields" "8" "$(printf '%s' "$FACTS" | wc -w)"
read -r F_CLUSTER F_CTX F_HTTP F_HTTPS F_TILT F_REG F_MANIFESTS F_OVERLAY <<< "$FACTS"
check "active: field 2 is kind-<cluster>" "kind-$F_CLUSTER" "$F_CTX"
check "active: fields 3-6 are numeric" "yes" \
"$([[ "$F_HTTP$F_HTTPS$F_TILT$F_REG" =~ ^[0-9]+$ ]] && echo yes || echo no)"
check "active: field 7 is a path" "yes" \
"$([ -n "$F_MANIFESTS" ] && [ "${F_MANIFESTS#-}" = "$F_MANIFESTS" ] && echo yes || echo no)"
# Absolute, or - when there is none: an empty field would shift every later one.
check "active: fields 7-8 are absolute paths or -" "yes" \
"$(for f in "$F_MANIFESTS" "$F_OVERLAY"; do case "$f" in -|/*) ;; *) echo no; exit; esac; done; echo yes)"
# derive answers a different question and must keep its own shape: it reports
# what the directory name implies, ignoring ctrl/.env, so nothing should
# configure itself from it.
@@ -76,7 +88,9 @@ test_value() {
KIND_CONFIG) echo "$PWD/k8s/kind-config.yaml.tpl" ;;
*_PORT) echo "19999" ;;
CLUSTER) echo "selftest-name" ;;
MANIFESTS_DIR) echo "../elsewhere/overlays/dev" ;;
# Named folders must exist, and must not be the default.
MANIFESTS_DIR) echo "examples/starter/k8s/base" ;;
OVERLAY) echo "examples/data" ;;
ADDONS) echo "metallb" ;;
*) echo "selftest-sentinel" ;;
esac
@@ -110,8 +124,8 @@ trap 'rm -rf "$TMP"' EXIT
mkdir -p "$TMP/My_Proj"
cp -r . "$TMP/My_Proj/ctrl"
# A pinned CLUSTER in .env would be an override, not a derivation, and this
# check is about the derivation.
sed -i '/^CLUSTER=/d' "$TMP/My_Proj/ctrl/.env" 2>/dev/null
# check is about the derivation. (An OVERLAY would be another derivation.)
sed -i '/^CLUSTER=/d; /^OVERLAY=/d' "$TMP/My_Proj/ctrl/.env" 2>/dev/null
COPY="$(cd "$TMP/My_Proj/ctrl" && bash ports.sh active)"
check "a dir named My_Proj derives a DNS label" "my-proj" "$(awk '{print $1}' <<< "$COPY")"
check "and a context to match" "kind-my-proj" "$(awk '{print $2}' <<< "$COPY")"
@@ -129,9 +143,88 @@ check "derive_port_base my-proj" "21030" "$(derive_port_base my-proj)"
note "rig stays standalone"
# rig must be copyable out of its host project: no references to the host.
# The pattern is assembled from fragments so this file does not match itself.
HOST_PAT="$(printf '%s' 'sole' 'print' '|\b' 'sp' 'r\b')"
# The host project's word for a backing service counts too: rig described its
# workload addons with it until they left. local/ holds overlays, which may say anything.
HOST_PAT="$(printf '%s' 'sole' 'print' '|\b' 'sp' 'r\b' '|' 'cab' 'inet')"
check "no host-project references" "0" \
"$(cd .. && grep -rIl -iE "$HOST_PAT" . --exclude-dir=def 2>/dev/null | wc -l)"
"$(cd .. && grep -rIl -iE "$HOST_PAT" . --exclude-dir=def --exclude-dir=local 2>/dev/null | wc -l)"
note "what runs is an overlay; rig only reads it"
# docs/notes/overlay.md. Every check runs in a scratch copy with its own overlay.
OV="$TMP/overlay-proof"; copy_rig "$OV/rig"
OVR="$OV/rig"
mkdir -p "$OVR/local/My_Env/addons" "$OVR/local/My_Env/k8s/prod" "$OVR/ctrl/env.d"
printf 'ADDONS="from-profile"\nDATA_NAMESPACE=from-profile\n' > "$OVR/ctrl/env.d/selftest.env"
cat > "$OVR/local/My_Env/rig.env" <<'EOF'
ADDONS="metallb"
DATA_NAMESPACE=from-overlay
MANIFESTS_DIR=k8s/prod
SELFTEST_SENTINEL=selftest-overlay-sentinel
EOF
printf 'resources: []\n' > "$OVR/local/My_Env/k8s/prod/kustomization.yaml"
printf '#!/usr/bin/env bash\necho "overlay-metallb from $PWD with ${RIG_CTRL:-no RIG_CTRL}"\n' \
> "$OVR/local/My_Env/addons/metallb.sh"
in_ov() { (cd "$OVR/ctrl" && "$@"); }
ov_key() { # key [env assignments...]
local k="$1"; shift
in_ov env "$@" bash -c 'source ./lib/config.sh; load_config >/dev/null 2>&1; printf "%s" "${!1}"' _ "$k"
}
# With nothing named, rig behaves as it did before overlays: same name, ports, addons, nodes.
check "no overlay: the cluster, ports and addons of before" "rig kind-rig 20310 20311 20312 20313" \
"$(in_ov bash ports.sh active | awk '{print $1, $2, $3, $4, $5, $6}')"
check "no overlay: no addons, one node, rig's own kind config" "|1|./k8s/kind-config.yaml.tpl" \
"$(ov_key ADDONS)|$(ov_key NODES)|$(ov_key KIND_CONFIG)"
# The overlay's rig.env sits between the profile and ctrl/.env; the caller beats all.
check "rig.env beats the profile" "from-overlay" \
"$(ov_key DATA_NAMESPACE PROFILE=selftest OVERLAY=local/My_Env)"
echo 'DATA_NAMESPACE=from-dotenv' >> "$OVR/ctrl/.env"
check "ctrl/.env beats rig.env" "from-dotenv" \
"$(ov_key DATA_NAMESPACE PROFILE=selftest OVERLAY=local/My_Env)"
sed -i '/^DATA_NAMESPACE=from-dotenv$/d' "$OVR/ctrl/.env"
check "the caller beats rig.env" "from-caller" \
"$(ov_key ADDONS OVERLAY=local/My_Env ADDONS=from-caller)"
# Identity follows the overlay's folder, so one rig serves several without collisions.
check "identity follows the overlay's folder" "my-env kind-my-env" \
"$(in_ov env OVERLAY=local/My_Env bash ports.sh active | awk '{print $1, $2}')"
check "paths in rig.env are relative to the overlay" "$OVR/local/My_Env/k8s/prod" \
"$(in_ov env OVERLAY=local/My_Env bash ports.sh active | awk '{print $7}')"
check "a named overlay that does not exist is an error" "yes" \
"$(in_ov env OVERLAY=local/nope bash ports.sh active >/dev/null 2>&1 && echo no || echo yes)"
printf 'PROFILE=x\n' > "$OV/bad-rig.env"; mkdir -p "$OVR/local/bad"; cp "$OV/bad-rig.env" "$OVR/local/bad/rig.env"
check "rig.env may not choose the profile or the overlay" "yes" \
"$(in_ov env OVERLAY=local/bad bash ports.sh active >/dev/null 2>&1 && echo no || echo yes)"
# Addons: the overlay's is found before rig's own, and runs from rig's ctrl/.
check "an overlay's addon comes before rig's of the same name" \
"overlay-metallb from $OVR/ctrl with $OVR/ctrl" \
"$(in_ov env OVERLAY=local/My_Env ADDONS=metallb bash addons.sh install 2>&1 | grep '^overlay-metallb')"
# ctrl/.env is this rig's: a pinned block would follow every overlay.
check "ports.sh persist refuses while an overlay is set" "yes" \
"$(in_ov env OVERLAY=local/My_Env bash ports.sh persist >/dev/null 2>&1 && echo no || echo yes)"
# make's $(shell) must see an OVERLAY given as a make argument (make < 4.4 does not pass it).
check "make -n tilt OVERLAY=... asks for the overlay's context" "kind-data" \
"$(cd .. && make --no-print-directory -n tilt OVERLAY=examples/data 2>/dev/null | grep -m1 'tilt ' | sed -n 's/.*--context \([^ ]*\).*/\1/p')"
# rig reads an overlay and never writes into it; its values never reach a committed kit.
sum_ov() { (cd "$OVR/local/My_Env" && find . -type f | sort | xargs sha256sum | sha256sum); }
before=$(sum_ov)
echo 'OVERLAY=local/My_Env' >> "$OVR/ctrl/.env"
in_ov bash ports.sh active >/dev/null 2>&1
in_ov bash addons.sh list >/dev/null 2>&1
in_ov bash -c 'source ./lib/config.sh; load_config >/dev/null; render_kind_config >/dev/null' 2>/dev/null
in_ov bash standalone.sh write >/dev/null 2>&1
in_ov bash standalone.sh export "$OV/export" >/dev/null 2>&1
check "rig writes nothing into an overlay" "$before" "$(sum_ov)"
check "an overlay's values never reach a committed kit" "0" \
"$(grep -rlE 'selftest-overlay-sentinel|local/My_Env' "$OVR/standalone" 2>/dev/null | wc -l)"
check "a committed kit holds no path of this machine" "0" \
"$(grep -rlF "$OVR" "$OVR/standalone" 2>/dev/null | wc -l)"
note "the Tiltfile hardcodes nothing"
@@ -139,6 +232,7 @@ note "the Tiltfile hardcodes nothing"
check "no literal kind-<name>" "0" "$(grep -cE "['\"]kind-[a-z0-9]" Tiltfile)"
check "guards on the variable" "1" "$(grep -c 'allow_k8s_contexts(CTX)' Tiltfile)"
check "asks ports.sh for facts" "1" "$(grep -c "local('bash ports.sh active'" Tiltfile)"
check "hands over to the overlay's Tiltfile" "1" "$(grep -c "include(OVERLAY + '/Tiltfile')" Tiltfile)"
note "standalone kits are generated, current, and call only real verbs"
@@ -168,7 +262,8 @@ check "there is a kit for every profile" "$(config_profiles | wc -l)" "$kits"
# An export carries this machine's choices but never its credentials; committed kits carry neither.
# Proven with sentinel values in a scratch copy, since the real ctrl/.env may leave them empty.
SX="$TMP/export-proof"; mkdir -p "$SX"; cp -r .. "$SX/rig"
SX="$TMP/export-proof"; copy_rig "$SX/rig"
mkdir -p "$SX/selftest-sentinel-choice/overlays/dev" # a named MANIFESTS_DIR must exist
cat >> "$SX/rig/ctrl/.env" <<'EOF'
REGISTRY_USER=selftest-sentinel-user
REGISTRY_PASSWORD=selftest-sentinel-password
@@ -187,16 +282,87 @@ check "export: refuses to write inside the repository" "yes" \
"$( (bash standalone.sh export ../standalone/selftest-mine >/dev/null 2>&1) && echo no || echo yes)"
note "optional — needs tilt and this rig's cluster"
# Tilt needs a cluster context to parse the Tiltfile, so this is skipped without one.
if ! command -v tilt >/dev/null; then
printf ' skip tilt is not installed\n'
elif ! kubectl config get-contexts -o name 2>/dev/null | grep -qx "$F_CTX"; then
printf " skip no %s context — run 'make cluster up' to include this\n" "$F_CTX"
note "rig's addons apply verified files, never URLs"
# The offline profile must need no network for manifests: each addon asks deps.sh for a
# pinned manifest, verified on disk (versions.md). Their images still need preloading.
check "no rig addon applies a URL" "0" \
"$(cat addons/*.sh | grep -cE 'apply -f "?https?://')"
check "every manifest an addon asks for is pinned with a sum" "" \
"$(for n in $(grep -ohE 'deps\.sh manifest [A-Z_]+' addons/*.sh | awk '{print $3}' | sort -u); do
grep -q "^${n}_MANIFEST_URL=" versions.env && grep -q "^${n}_MANIFEST_SHA256=[0-9a-f]\{64\}$" versions.env \
|| printf '%s ' "$n"; done)"
note "withdrawn stays withdrawn (STALE.md)"
# One check per entry; the reasoning is in STALE.md, not here.
check "✖ S1 rig's Tiltfile has no Images section of its own" "0" "$(grep -c '^# ── Images' Tiltfile)"
check "✖ S2 local/ is where overlays live, and ignored" "yes" \
"$(grep -qx '/local/' ../.gitignore && echo yes || echo no)"
# Patterns assembled from fragments so this file does not match itself.
COPIES_PAT="$(printf '%s' 'ac' 'me-rig|ac' 'mebank')"
HOUSE_PAT="$(printf '%s' 'semes' 'ter|local' '\.ar\b')"
check "✖ S2 no example environment name from the copies era" "0" \
"$(cd .. && grep -rIlE "$COPIES_PAT" . --exclude-dir=def --exclude-dir=local --exclude=STALE.md 2>/dev/null | wc -l)"
check "✖ S3 ctrl/addons makes the cluster work, nothing more" "cert-manager metallb metrics-server" \
"$(ls addons/ | sed 's/\.sh$//' | sort | xargs)"
check "✖ S3 versions.env pins no workload image" "0" \
"$(grep -cE '^(POSTGRES|REDIS|AIRFLOW)_IMAGE=' versions.env)"
check "✖ S4 no namespace named after the cluster" "0" "$(grep -c "CLUSTER + ':namespace'" Tiltfile)"
check "✖ S5 rig's examples left ctrl/k8s" "no" "$([ -d k8s/overlays ] && echo yes || echo no)"
check "✖ S5 .env.example does not pin MANIFESTS_DIR" "0" "$(grep -c '^MANIFESTS_DIR=' .env.example)"
check "✖ S6 no client or data example profile" "0" \
"$(ls env.d/ | grep -cE '^(client|data)\.')"
check "✖ S7 no house path or host name in rig" "0" \
"$(cd .. && grep -rIlE "$HOUSE_PAT" . --exclude-dir=def --exclude-dir=local --exclude=STALE.md 2>/dev/null | wc -l)"
note "the dev loop parses — needs tilt and kubectl, not a cluster"
# A throwaway kubeconfig with kind-named entries (Tilt trusts kind contexts) and a
# kubectl that swallows `apply`: the Tiltfile evaluates for real, nothing is contacted.
# The second run is a copy under another name, the case that once failed at load.
if ! command -v tilt >/dev/null || ! command -v kubectl >/dev/null; then
printf ' skip tilt or kubectl is not installed\n'
else
FK="$TMP/fake-kube"; mkdir -p "$FK"
real_kubectl=$(command -v kubectl)
printf '#!/usr/bin/env bash\nfor a in "$@"; do [ "$a" = apply ] && { cat >/dev/null; exit 0; }; done\nexec %q "$@"\n' \
"$real_kubectl" > "$FK/kubectl"
chmod +x "$FK/kubectl"
parses() { # cluster-name [env...] -> the manifests Tilt would deploy, or the error
local name="$1"; shift
cat > "$FK/kubeconfig" <<EOF
apiVersion: v1
kind: Config
clusters: [{name: kind-$name, cluster: {server: "https://127.0.0.1:9"}}]
contexts: [{name: kind-$name, context: {cluster: kind-$name, user: kind-$name}}]
users: [{name: kind-$name, user: {token: selftest}}]
current-context: kind-$name
EOF
env "$@" KUBECONFIG="$FK/kubeconfig" PATH="$FK:$PATH" \
timeout 120 tilt alpha tiltfile-result --context "kind-$name" > "$FK/out.json" 2> "$FK/err" \
&& grep -o '"Name": *"[^"]*"' "$FK/out.json" | sed 's/.*"\([^"]*\)"$/\1/' | sort -u | xargs \
|| grep -m1 -iE 'error|no object' "$FK/err"
}
check "the starter overlay parses" "example-service infra uncategorized" "$(parses rig)"
check "and under another name" "example-service infra uncategorized" \
"$(parses selftest-copy CLUSTER=selftest-copy)"
check "the data overlay parses" "items-api uncategorized" "$(parses data OVERLAY=examples/data)"
fi
note "the examples are overlays that work as shipped"
# They are what a real overlay is copied from, so they must at least parse.
bad=""
for f in ../examples/*/addons/*.sh; do [ -e "$f" ] && { bash -n "$f" 2>/dev/null || bad+="$f "; }; done
check "every example addon parses" "" "$bad"
if command -v python3 >/dev/null; then
bad=""
for f in ../examples/*/dags/*.py; do
[ -e "$f" ] && { python3 -c 'import ast, sys; ast.parse(open(sys.argv[1]).read())' "$f" 2>/dev/null || bad+="$f "; }
done
check "every example DAG parses" "" "$bad"
else
out="$(tilt alpha tiltfile-result --context "$F_CTX" 2>&1)"
check "Tiltfile evaluates" "yes" \
"$(printf '%s' "$out" | grep -q '"Manifests"' && echo yes || echo "no: $(printf '%s' "$out" | tail -1)")"
printf ' skip python3 is not installed\n'
fi

View File

@@ -30,25 +30,28 @@ COMPOSE_VERSION=5.5.1
COMPOSE_SHA256=db1889184726840f75c4f9c001048430d4f25b3be3cb084d3ddd762bc0aed576
COMPOSE_URL=https://github.com/docker/compose/releases/download/v${COMPOSE_VERSION}/docker-compose-linux-x86_64
# Node images for KIND_VERSION, pinned by digest; profiles pick one via K8S_VERSION.
# Older entries are kept deliberately (legacy-estate simulation).
# Node images for KIND_VERSION, pinned by digest; K8S_VERSION picks one (default: the newest).
# Older entries are kept deliberately, for targets that run an older Kubernetes.
NODE_IMAGE_v1_36=kindest/node:v1.36.1@sha256:3489c7674813ba5d8b1a9977baea8a6e553784dab7b84759d1014dbd78f7ebd5
NODE_IMAGE_v1_35=kindest/node:v1.35.5@sha256:ce977ae6d65918d0b58a5f8b5e940429c2ce42fa3a5619ec2bbc60b949c0ac95
NODE_IMAGE_v1_34=kindest/node:v1.34.8@sha256:02722c2dedddcfc00febf5d27fbeb9b7b2c14294c82109ff4a85d89ac9ba3256
NODE_IMAGE_v1_33=kindest/node:v1.33.12@sha256:3f5c8443c620245e4d355cfe09e96a91ead32ceaa569d3f1ca9edf0cb2fe2ff4
# Images pulled at runtime (registry, mocks). Pinned by tag; the registry mode
# decides where they are pulled FROM.
# Images pulled at runtime. Pinned by tag; the registry mode decides where they are pulled FROM.
REGISTRY_IMAGE=registry:2
STUB_IMAGE=python:3.12-slim
# Addons, installed by ctrl/addons/<name>.sh when listed in a profile's ADDONS.
# rig's own addons (ctrl/addons/<name>.sh), installed when ADDONS names them.
CERT_MANAGER_VERSION=v1.21.1
METRICS_SERVER_VERSION=v0.9.0
METALLB_VERSION=v0.16.0
# Cabinets — unmodified upstream images, usable on compose or in the cluster.
# Pinned by tag; bump freely, and preload them for the offline profile.
POSTGRES_IMAGE=postgres:16-alpine
REDIS_IMAGE=redis:7-alpine
AIRFLOW_IMAGE=apache/airflow:2.10.4
# The manifests those addons apply, fetched and verified like the binaries
# (`deps.sh manifest <NAME>`), so an offline machine needs no network for them.
# Sums from the release's own asset digest; metallb publishes none, see versions.md.
CERT_MANAGER_MANIFEST_URL=https://github.com/cert-manager/cert-manager/releases/download/${CERT_MANAGER_VERSION}/cert-manager.yaml
CERT_MANAGER_MANIFEST_SHA256=5f6a499b8c1857d57f560f536e0dcc830914b45c420899fe7ad0692c8624e408
METRICS_SERVER_MANIFEST_URL=https://github.com/kubernetes-sigs/metrics-server/releases/download/${METRICS_SERVER_VERSION}/components.yaml
METRICS_SERVER_MANIFEST_SHA256=1cec29a5267809306a2c6ec74a3e449abbb705b4a8beed0c8a1963910f72c79b
METALLB_MANIFEST_URL=https://raw.githubusercontent.com/metallb/metallb/${METALLB_VERSION}/config/manifests/metallb-native.yaml
METALLB_MANIFEST_SHA256=b0b9be2802f10aa32d45308b4457d06cde0c70544712c8d0cf5511657ffd2b69
METALLB_MANIFEST_GIT_BLOB=7fbda334cc3ac0aaabdcb081af4f543feb3c2f9f

View File

@@ -5,12 +5,12 @@ digraph rig_environment {
node [fontname="Helvetica" fontsize=11 style=filled color="#1e2a4a" fontcolor="#e8eaf0" shape=box]
edge [fontname="Helvetica" fontsize=9 fontcolor="#8892a8" color="#4a5568"]
label="One environment per directory — copies never collide"
label="One environment per folder — copies never collide"
labelloc=t
fontsize=16
fontcolor="#0066ff"
dirname [label="directory name\ne.g. acmebank/" fillcolor="#1f6feb" fontcolor="#ffffff" shape=octagon]
dirname [label="folder name\nthe overlay's, else rig's\ne.g. platform-v2/" fillcolor="#1f6feb" fontcolor="#ffffff" shape=octagon]
subgraph cluster_derived {
label="Everything below is derived from it"
@@ -18,11 +18,11 @@ digraph rig_environment {
color="#1e2a4a"
fontcolor="#8892a8"
cname [label="cluster name\nacmebank" fillcolor="#121829"]
ctx [label="kubectl context\nkind-acmebank" fillcolor="#121829"]
img [label="image tag\nacmebank-deps" fillcolor="#121829"]
ports [label="port block\n2130021309" fillcolor="#121829"]
reg [label="registry container\nacmebank-registry" fillcolor="#121829"]
cname [label="cluster name\nplatform-v2" fillcolor="#121829"]
ctx [label="kubectl context\nkind-platform-v2" fillcolor="#121829"]
img [label="image tag\nplatform-v2-deps" fillcolor="#121829"]
ports [label="port block\n2043020439" fillcolor="#121829"]
reg [label="registry container\nplatform-v2-registry" fillcolor="#121829"]
}
subgraph cluster_config {
@@ -32,9 +32,10 @@ digraph rig_environment {
fontcolor="#8892a8"
versions [label="versions.env\npinned toolchain" fillcolor="#121829"]
profile [label="env.d/<profile>.env\noptional: addons · registry" fillcolor="#121829"]
profile [label="env.d/<profile>.env\noptional: registry · mirror" fillcolor="#121829"]
overlay [label="<overlay>/rig.env\noptional: what runs" fillcolor="#121829"]
localenv [label="ctrl/.env\nsecrets, overrides" fillcolor="#121829"]
shell [label="the environment\nPROFILE=client make …" fillcolor="#1a3a1a" fontcolor="#00c853"]
shell [label="the environment\nOVERLAY=local/x make …" fillcolor="#1a3a1a" fontcolor="#00c853"]
}
dirname -> cname
@@ -44,7 +45,8 @@ digraph rig_environment {
dirname -> reg
versions -> profile [label="overridden by"]
profile -> localenv [label="overridden by"]
profile -> overlay [label="overridden by"]
overlay -> localenv [label="overridden by"]
localenv -> shell [label="overridden by" color="#00c853"]
cluster [label="kind cluster" fillcolor="#1a1a3a" fontcolor="#0066ff" shape=octagon]

View File

@@ -4,166 +4,181 @@
<!-- Generated by graphviz version 14.1.2 (0)
-->
<!-- Title: rig_environment Pages: 1 -->
<svg width="962pt" height="481pt"
viewBox="0.00 0.00 962.00 481.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 476.83)">
<svg width="981pt" height="585pt"
viewBox="0.00 0.00 981.00 585.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 580.74)">
<title>rig_environment</title>
<polygon fill="#0a0e17" stroke="none" points="-4,4 -4,-476.83 958,-476.83 958,4 -4,4"/>
<text xml:space="preserve" text-anchor="middle" x="477" y="-453.63" font-family="Helvetica,sans-Serif" font-size="16.00" fill="#0066ff">One environment per directory — copies never collide</text>
<polygon fill="#0a0e17" stroke="none" points="-4,4 -4,-580.74 977,-580.74 977,4 -4,4"/>
<text xml:space="preserve" text-anchor="middle" x="486.5" y="-557.54" font-family="Helvetica,sans-Serif" font-size="16.00" fill="#0066ff">One environment per folder — copies never collide</text>
<g id="clust1" class="cluster">
<title>cluster_derived</title>
<polygon fill="#0a0e17" stroke="#1e2a4a" stroke-dasharray="5,2" points="8,-65 8,-144.5 596,-144.5 596,-65 8,-65"/>
<text xml:space="preserve" text-anchor="middle" x="302" y="-125.3" font-family="Helvetica,sans-Serif" font-size="16.00" fill="#8892a8">Everything below is derived from it</text>
<polygon fill="#0a0e17" stroke="#1e2a4a" stroke-dasharray="5,2" points="8,-65 8,-144.5 615,-144.5 615,-65 8,-65"/>
<text xml:space="preserve" text-anchor="middle" x="311.5" y="-125.3" font-family="Helvetica,sans-Serif" font-size="16.00" fill="#8892a8">Everything below is derived from it</text>
</g>
<g id="clust2" class="cluster">
<title>cluster_config</title>
<polygon fill="#0a0e17" stroke="#1e2a4a" stroke-dasharray="5,2" points="604,-65 604,-437.33 946,-437.33 946,-65 604,-65"/>
<text xml:space="preserve" text-anchor="middle" x="775" y="-418.13" font-family="Helvetica,sans-Serif" font-size="16.00" fill="#8892a8">Configuration — weakest first, later wins</text>
<polygon fill="#0a0e17" stroke="#1e2a4a" stroke-dasharray="5,2" points="623,-65 623,-541.24 965,-541.24 965,-65 623,-65"/>
<text xml:space="preserve" text-anchor="middle" x="794" y="-522.04" font-family="Helvetica,sans-Serif" font-size="16.00" fill="#8892a8">Configuration — weakest first, later wins</text>
</g>
<!-- dirname -->
<g id="node1" class="node">
<title>dirname</title>
<polygon fill="#1f6feb" stroke="#1e2a4a" points="370.11,-197.44 370.11,-219.63 324.94,-235.33 261.06,-235.33 215.89,-219.63 215.89,-197.44 261.06,-181.75 324.94,-181.75 370.11,-197.44"/>
<text xml:space="preserve" text-anchor="middle" x="293" y="-211.59" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#ffffff">directory name</text>
<text xml:space="preserve" text-anchor="middle" x="293" y="-198.09" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#ffffff">e.g. acmebank/</text>
<polygon fill="#1f6feb" stroke="#1e2a4a" points="411.98,-203.49 411.98,-234.25 346.97,-255.99 255.03,-255.99 190.02,-234.25 190.02,-203.49 255.03,-181.75 346.97,-181.75 411.98,-203.49"/>
<text xml:space="preserve" text-anchor="middle" x="301" y="-228.67" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#ffffff">folder name</text>
<text xml:space="preserve" text-anchor="middle" x="301" y="-215.17" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#ffffff">the overlay&#39;s, else rig&#39;s</text>
<text xml:space="preserve" text-anchor="middle" x="301" y="-201.67" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#ffffff">e.g. platform&#45;v2/</text>
</g>
<!-- cname -->
<g id="node2" class="node">
<title>cname</title>
<polygon fill="#121829" stroke="#1e2a4a" points="104,-109 16,-109 16,-73 104,-73 104,-109"/>
<text xml:space="preserve" text-anchor="middle" x="60" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">cluster name</text>
<text xml:space="preserve" text-anchor="middle" x="60" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">acmebank</text>
<text xml:space="preserve" text-anchor="middle" x="60" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">platform&#45;v2</text>
</g>
<!-- dirname&#45;&gt;cname -->
<g id="edge1" class="edge">
<title>dirname&#45;&gt;cname</title>
<path fill="none" stroke="#4a5568" d="M228.68,-192.51C192.88,-182.36 148.52,-166.7 113,-144.5 101.4,-137.25 90.34,-127.04 81.36,-117.55"/>
<polygon fill="#4a5568" stroke="#4a5568" points="84.07,-115.32 74.76,-110.26 78.88,-120.02 84.07,-115.32"/>
<path fill="none" stroke="#4a5568" d="M217.9,-193.66C183.86,-181.7 145.02,-165.31 113,-144.5 101.69,-137.15 90.82,-127.09 81.89,-117.75"/>
<polygon fill="#4a5568" stroke="#4a5568" points="84.65,-115.58 75.31,-110.58 79.49,-120.31 84.65,-115.58"/>
</g>
<!-- ctx -->
<g id="node3" class="node">
<title>ctx</title>
<polygon fill="#121829" stroke="#1e2a4a" points="223.75,-109 122.25,-109 122.25,-73 223.75,-73 223.75,-109"/>
<text xml:space="preserve" text-anchor="middle" x="173" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">kubectl context</text>
<text xml:space="preserve" text-anchor="middle" x="173" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">kind&#45;acmebank</text>
<polygon fill="#121829" stroke="#1e2a4a" points="228,-109 122,-109 122,-73 228,-73 228,-109"/>
<text xml:space="preserve" text-anchor="middle" x="175" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">kubectl context</text>
<text xml:space="preserve" text-anchor="middle" x="175" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">kind&#45;platform&#45;v2</text>
</g>
<!-- dirname&#45;&gt;ctx -->
<g id="edge2" class="edge">
<title>dirname&#45;&gt;ctx</title>
<path fill="none" stroke="#4a5568" d="M265.77,-181.32C245.77,-162.07 218.77,-136.06 199.05,-117.08"/>
<polygon fill="#4a5568" stroke="#4a5568" points="201.55,-114.63 191.91,-110.21 196.69,-119.67 201.55,-114.63"/>
<path fill="none" stroke="#4a5568" d="M264.56,-181.46C244.01,-160.94 218.85,-135.8 200.43,-117.41"/>
<polygon fill="#4a5568" stroke="#4a5568" points="203.11,-115.13 193.56,-110.54 198.16,-120.08 203.11,-115.13"/>
</g>
<!-- img -->
<g id="node4" class="node">
<title>img</title>
<polygon fill="#121829" stroke="#1e2a4a" points="344.12,-109 241.88,-109 241.88,-73 344.12,-73 344.12,-109"/>
<text xml:space="preserve" text-anchor="middle" x="293" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">image tag</text>
<text xml:space="preserve" text-anchor="middle" x="293" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">acmebank&#45;deps</text>
<polygon fill="#121829" stroke="#1e2a4a" points="355.88,-109 246.12,-109 246.12,-73 355.88,-73 355.88,-109"/>
<text xml:space="preserve" text-anchor="middle" x="301" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">image tag</text>
<text xml:space="preserve" text-anchor="middle" x="301" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">platform&#45;v2&#45;deps</text>
</g>
<!-- dirname&#45;&gt;img -->
<g id="edge3" class="edge">
<title>dirname&#45;&gt;img</title>
<path fill="none" stroke="#4a5568" d="M293,-181.32C293,-163.19 293,-139.07 293,-120.47"/>
<polygon fill="#4a5568" stroke="#4a5568" points="296.5,-120.67 293,-110.67 289.5,-120.67 296.5,-120.67"/>
<path fill="none" stroke="#4a5568" d="M301,-181.46C301,-162.23 301,-138.96 301,-120.97"/>
<polygon fill="#4a5568" stroke="#4a5568" points="304.5,-120.98 301,-110.98 297.5,-120.98 304.5,-120.98"/>
</g>
<!-- ports -->
<g id="node5" class="node">
<title>ports</title>
<polygon fill="#121829" stroke="#1e2a4a" points="451.38,-109 362.62,-109 362.62,-73 451.38,-73 451.38,-109"/>
<text xml:space="preserve" text-anchor="middle" x="407" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">port block</text>
<text xml:space="preserve" text-anchor="middle" x="407" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">2130021309</text>
<polygon fill="#121829" stroke="#1e2a4a" points="462.38,-109 373.62,-109 373.62,-73 462.38,-73 462.38,-109"/>
<text xml:space="preserve" text-anchor="middle" x="418" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">port block</text>
<text xml:space="preserve" text-anchor="middle" x="418" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">2043020439</text>
</g>
<!-- dirname&#45;&gt;ports -->
<g id="edge4" class="edge">
<title>dirname&#45;&gt;ports</title>
<path fill="none" stroke="#4a5568" d="M318.87,-181.32C337.78,-162.15 363.29,-136.3 382,-117.34"/>
<polygon fill="#4a5568" stroke="#4a5568" points="384.47,-119.81 389,-110.24 379.49,-114.9 384.47,-119.81"/>
<path fill="none" stroke="#4a5568" d="M334.84,-181.46C353.92,-160.94 377.28,-135.8 394.38,-117.41"/>
<polygon fill="#4a5568" stroke="#4a5568" points="396.49,-120.29 400.73,-110.58 391.36,-115.52 396.49,-120.29"/>
</g>
<!-- reg -->
<g id="node6" class="node">
<title>reg</title>
<polygon fill="#121829" stroke="#1e2a4a" points="588.38,-109 469.62,-109 469.62,-73 588.38,-73 588.38,-109"/>
<text xml:space="preserve" text-anchor="middle" x="529" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">registry container</text>
<text xml:space="preserve" text-anchor="middle" x="529" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">acmebank&#45;registry</text>
<polygon fill="#121829" stroke="#1e2a4a" points="607.12,-109 480.88,-109 480.88,-73 607.12,-73 607.12,-109"/>
<text xml:space="preserve" text-anchor="middle" x="544" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">registry container</text>
<text xml:space="preserve" text-anchor="middle" x="544" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">platform&#45;v2&#45;registry</text>
</g>
<!-- dirname&#45;&gt;reg -->
<g id="edge5" class="edge">
<title>dirname&#45;&gt;reg</title>
<path fill="none" stroke="#4a5568" d="M350.86,-190.3C383.87,-179.35 425.43,-163.64 460,-144.5 474.27,-136.6 488.83,-125.98 500.87,-116.36"/>
<polygon fill="#4a5568" stroke="#4a5568" points="502.85,-119.26 508.36,-110.21 498.41,-113.84 502.85,-119.26"/>
<path fill="none" stroke="#4a5568" d="M373.79,-190.51C404.52,-177.93 440.23,-161.93 471,-144.5 485.65,-136.2 500.9,-125.58 513.66,-116.06"/>
<polygon fill="#4a5568" stroke="#4a5568" points="515.41,-119.13 521.26,-110.3 511.18,-113.56 515.41,-119.13"/>
</g>
<!-- cluster -->
<g id="node11" class="node">
<g id="node12" class="node">
<title>cluster</title>
<polygon fill="#1a1a3a" stroke="#1e2a4a" points="460.81,-10.54 460.81,-25.46 429.29,-36 384.71,-36 353.19,-25.46 353.19,-10.54 384.71,0 429.29,0 460.81,-10.54"/>
<text xml:space="preserve" text-anchor="middle" x="407" y="-14.3" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#0066ff">kind cluster</text>
<polygon fill="#1a1a3a" stroke="#1e2a4a" points="471.81,-10.54 471.81,-25.46 440.29,-36 395.71,-36 364.19,-25.46 364.19,-10.54 395.71,0 440.29,0 471.81,-10.54"/>
<text xml:space="preserve" text-anchor="middle" x="418" y="-14.3" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#0066ff">kind cluster</text>
</g>
<!-- cname&#45;&gt;cluster -->
<g id="edge9" class="edge">
<g id="edge10" class="edge">
<title>cname&#45;&gt;cluster</title>
<path fill="none" stroke="#4a5568" d="M93.35,-72.51C99.75,-69.67 106.48,-67 113,-65 189.55,-41.49 281.19,-29.58 341.58,-23.85"/>
<polygon fill="#4a5568" stroke="#4a5568" points="341.71,-27.35 351.35,-22.96 341.07,-20.38 341.71,-27.35"/>
<path fill="none" stroke="#4a5568" d="M93.06,-72.61C99.54,-69.72 106.38,-67.01 113,-65 193.43,-40.54 289.9,-28.78 352.5,-23.34"/>
<polygon fill="#4a5568" stroke="#4a5568" points="352.6,-26.85 362.28,-22.53 352.02,-19.87 352.6,-26.85"/>
</g>
<!-- ports&#45;&gt;cluster -->
<g id="edge10" class="edge">
<g id="edge11" class="edge">
<title>ports&#45;&gt;cluster</title>
<path fill="none" stroke="#4a5568" d="M407,-72.81C407,-65.23 407,-56.1 407,-47.54"/>
<polygon fill="#4a5568" stroke="#4a5568" points="410.5,-47.54 407,-37.54 403.5,-47.54 410.5,-47.54"/>
<path fill="none" stroke="#4a5568" d="M418,-72.81C418,-65.23 418,-56.1 418,-47.54"/>
<polygon fill="#4a5568" stroke="#4a5568" points="421.5,-47.54 418,-37.54 414.5,-47.54 421.5,-47.54"/>
</g>
<!-- versions -->
<g id="node7" class="node">
<title>versions</title>
<polygon fill="#121829" stroke="#1e2a4a" points="750.38,-401.83 643.62,-401.83 643.62,-365.83 750.38,-365.83 750.38,-401.83"/>
<text xml:space="preserve" text-anchor="middle" x="697" y="-386.88" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">versions.env</text>
<text xml:space="preserve" text-anchor="middle" x="697" y="-373.38" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">pinned toolchain</text>
<polygon fill="#121829" stroke="#1e2a4a" points="764.38,-505.74 657.62,-505.74 657.62,-469.74 764.38,-469.74 764.38,-505.74"/>
<text xml:space="preserve" text-anchor="middle" x="711" y="-490.79" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">versions.env</text>
<text xml:space="preserve" text-anchor="middle" x="711" y="-477.29" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">pinned toolchain</text>
</g>
<!-- profile -->
<g id="node8" class="node">
<title>profile</title>
<polygon fill="#121829" stroke="#1e2a4a" points="781.5,-318.58 612.5,-318.58 612.5,-282.58 781.5,-282.58 781.5,-318.58"/>
<text xml:space="preserve" text-anchor="middle" x="697" y="-303.63" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">env.d/&lt;profile&gt;.env</text>
<text xml:space="preserve" text-anchor="middle" x="697" y="-290.13" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">optional: addons · registry</text>
<polygon fill="#121829" stroke="#1e2a4a" points="788.75,-422.49 633.25,-422.49 633.25,-386.49 788.75,-386.49 788.75,-422.49"/>
<text xml:space="preserve" text-anchor="middle" x="711" y="-407.54" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">env.d/&lt;profile&gt;.env</text>
<text xml:space="preserve" text-anchor="middle" x="711" y="-394.04" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">optional: registry · mirror</text>
</g>
<!-- versions&#45;&gt;profile -->
<g id="edge6" class="edge">
<title>versions&#45;&gt;profile</title>
<path fill="none" stroke="#4a5568" d="M697,-365.59C697,-355.32 697,-342.03 697,-330.21"/>
<polygon fill="#4a5568" stroke="#4a5568" points="700.5,-330.58 697,-320.58 693.5,-330.58 700.5,-330.58"/>
<text xml:space="preserve" text-anchor="middle" x="728.5" y="-339.28" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">overridden by</text>
<path fill="none" stroke="#4a5568" d="M711,-469.51C711,-459.24 711,-445.94 711,-434.13"/>
<polygon fill="#4a5568" stroke="#4a5568" points="714.5,-434.49 711,-424.49 707.5,-434.49 714.5,-434.49"/>
<text xml:space="preserve" text-anchor="middle" x="742.5" y="-443.19" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">overridden by</text>
</g>
<!-- localenv -->
<!-- overlay -->
<g id="node9" class="node">
<title>localenv</title>
<polygon fill="#121829" stroke="#1e2a4a" points="752.88,-226.54 637.12,-226.54 637.12,-190.54 752.88,-190.54 752.88,-226.54"/>
<text xml:space="preserve" text-anchor="middle" x="695" y="-211.59" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">ctrl/.env</text>
<text xml:space="preserve" text-anchor="middle" x="695" y="-198.09" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">secrets, overrides</text>
<title>overlay</title>
<polygon fill="#121829" stroke="#1e2a4a" points="772.25,-339.24 649.75,-339.24 649.75,-303.24 772.25,-303.24 772.25,-339.24"/>
<text xml:space="preserve" text-anchor="middle" x="711" y="-324.29" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">&lt;overlay&gt;/rig.env</text>
<text xml:space="preserve" text-anchor="middle" x="711" y="-310.79" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">optional: what runs</text>
</g>
<!-- profile&#45;&gt;localenv -->
<!-- profile&#45;&gt;overlay -->
<g id="edge7" class="edge">
<title>profile&#45;&gt;localenv</title>
<path fill="none" stroke="#4a5568" d="M696.61,-282.22C696.34,-269.76 695.96,-252.69 695.64,-238.23"/>
<polygon fill="#4a5568" stroke="#4a5568" points="699.14,-238.28 695.42,-228.36 692.14,-238.43 699.14,-238.28"/>
<text xml:space="preserve" text-anchor="middle" x="727.68" y="-256.03" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">overridden by</text>
<title>profile&#45;&gt;overlay</title>
<path fill="none" stroke="#4a5568" d="M711,-386.26C711,-375.99 711,-362.69 711,-350.88"/>
<polygon fill="#4a5568" stroke="#4a5568" points="714.5,-351.24 711,-341.24 707.5,-351.24 714.5,-351.24"/>
<text xml:space="preserve" text-anchor="middle" x="742.5" y="-359.94" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">overridden by</text>
</g>
<!-- shell -->
<!-- localenv -->
<g id="node10" class="node">
<title>localenv</title>
<polygon fill="#121829" stroke="#1e2a4a" points="768.88,-236.87 653.12,-236.87 653.12,-200.87 768.88,-200.87 768.88,-236.87"/>
<text xml:space="preserve" text-anchor="middle" x="711" y="-221.92" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">ctrl/.env</text>
<text xml:space="preserve" text-anchor="middle" x="711" y="-208.42" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#e8eaf0">secrets, overrides</text>
</g>
<!-- overlay&#45;&gt;localenv -->
<g id="edge8" class="edge">
<title>overlay&#45;&gt;localenv</title>
<path fill="none" stroke="#4a5568" d="M711,-302.76C711,-287.79 711,-265.95 711,-248.44"/>
<polygon fill="#4a5568" stroke="#4a5568" points="714.5,-248.67 711,-238.67 707.5,-248.67 714.5,-248.67"/>
<text xml:space="preserve" text-anchor="middle" x="742.5" y="-276.69" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">overridden by</text>
</g>
<!-- shell -->
<g id="node11" class="node">
<title>shell</title>
<polygon fill="#1a3a1a" stroke="#1e2a4a" points="763.38,-109 614.62,-109 614.62,-73 763.38,-73 763.38,-109"/>
<text xml:space="preserve" text-anchor="middle" x="689" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#00c853">the environment</text>
<text xml:space="preserve" text-anchor="middle" x="689" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#00c853">PROFILE=client make …</text>
<polygon fill="#1a3a1a" stroke="#1e2a4a" points="791,-109 631,-109 631,-73 791,-73 791,-109"/>
<text xml:space="preserve" text-anchor="middle" x="711" y="-94.05" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#00c853">the environment</text>
<text xml:space="preserve" text-anchor="middle" x="711" y="-80.55" font-family="Helvetica,sans-Serif" font-size="11.00" fill="#00c853">OVERLAY=local/x make …</text>
</g>
<!-- localenv&#45;&gt;shell -->
<g id="edge8" class="edge">
<g id="edge9" class="edge">
<title>localenv&#45;&gt;shell</title>
<path fill="none" stroke="#00c853" d="M694.11,-190.49C693.16,-172.16 691.63,-142.72 690.49,-120.79"/>
<polygon fill="#00c853" stroke="#00c853" points="694,-120.81 689.99,-111.01 687.01,-121.18 694,-120.81"/>
<text xml:space="preserve" text-anchor="middle" x="724.21" y="-155.2" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">overridden by</text>
<path fill="none" stroke="#00c853" d="M711,-200.63C711,-180.03 711,-145.27 711,-120.62"/>
<polygon fill="#00c853" stroke="#00c853" points="714.5,-120.95 711,-110.95 707.5,-120.95 714.5,-120.95"/>
<text xml:space="preserve" text-anchor="middle" x="742.5" y="-155.2" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">overridden by</text>
</g>
<!-- shell&#45;&gt;cluster -->
<g id="edge11" class="edge">
<g id="edge12" class="edge">
<title>shell&#45;&gt;cluster</title>
<path fill="none" stroke="#4a5568" stroke-dasharray="5,2" d="M628.29,-72.6C618.83,-69.99 609.17,-67.38 600,-65 553.72,-52.99 500.89,-40.48 462.22,-31.54"/>
<polygon fill="#4a5568" stroke="#4a5568" points="463.2,-28.18 452.67,-29.35 461.63,-35 463.2,-28.18"/>
<path fill="none" stroke="#4a5568" stroke-dasharray="5,2" d="M648.24,-72.58C638.47,-69.98 628.47,-67.37 619,-65 570.38,-52.84 514.82,-40.22 474.45,-31.28"/>
<polygon fill="#4a5568" stroke="#4a5568" points="475.23,-27.87 464.71,-29.13 473.72,-34.7 475.23,-27.87"/>
</g>
</g>
</svg>

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 12 KiB

View File

@@ -240,6 +240,7 @@
<a href="#steps">The steps</a>
<a href="#install">Installation</a>
<a href="#environments">Environments</a>
<a href="#overlays">Overlays</a>
<a href="#profiles">Profiles</a>
<a href="#registry">Registry</a>
<a href="#architecture">Architecture</a>
@@ -303,7 +304,8 @@ make cluster up <span class="c"># cluster + registry + addons; ports derive by
finishes the job rather than reporting "already exists" and leaving every
node permanently NotReady.</p>
<pre><code>make cluster up <span class="c"># built-in defaults — no profile needed</span>
make cluster up PROFILE=client <span class="c"># after copying env.d/client.env.example: cached registry</span>
make cluster up PROFILE=mirror <span class="c"># after copying env.d/mirror.env.example: cached registry</span>
make cluster up OVERLAY=examples/data <span class="c"># an overlay: what runs, kept outside rig</span>
make cluster reset <span class="c"># destroy and rebuild — how an edited kind config takes effect</span>
</code></pre>
@@ -321,12 +323,12 @@ make cluster reset <span class="c"># destroy and rebuild — how an
</dl>
<h3>Running more than one</h3>
<p>Copy the directory, rename it, and run <code>make cluster up</code>. Cluster name,
context, image tags and the port block all follow the directory name, so
the second environment collides with nothing and neither one's teardown can
reach the other.</p>
<pre><code>cp -r rig ../platform-v2 &amp;&amp; cd ../platform-v2
make cluster up
<p>Name another overlay, or copy the directory and rename it. Cluster name,
context, image tags and the port block all follow the folder name — the
overlay's, or rig's — so the second environment collides with nothing and
neither one's teardown can reach the other.</p>
<pre><code>OVERLAY=local/platform-v2 make cluster up
cp -r rig ../platform-v3 &amp;&amp; cd ../platform-v3 &amp;&amp; make cluster up
</code></pre>
</div>
</section>
@@ -382,16 +384,16 @@ docker load &lt; rig.tgz &amp;&amp; make cluster up PROFILE=offline <span clas
<section class="section" id="environments">
<h2>Environments</h2>
<p class="lede">One directory is one environment. Copy it, rename it, run it.</p>
<p class="lede">One folder is one environment — an overlay's, or rig's own. Copy it, rename it, run it.</p>
<div class="graph-container">
<a href="viewer.html?src=graphs/02-environment.svg"><img src="graphs/02-environment.svg" alt="Environment derivation"></a>
</div>
<div class="prose">
<p>Running several versions of a system at once means several clusters on one
machine, not several machines. Everything that could collide is derived from
the directory name:</p>
the folder name (the overlay's when one is named):</p>
<dl>
<dt>cluster + context</dt><dd><code>acmebank/</code> builds <code>acmebank</code> on <code>kind-acmebank</code>.</dd>
<dt>cluster + context</dt><dd><code>platform-v2/</code> builds <code>platform-v2</code> on <code>kind-platform-v2</code>.</dd>
<dt>port block</dt><dd>Ten ports from a hash of the name, in the 20000+ range — clear of 80, 443, 3000, 5432, 8000 and 8080.</dd>
<dt>registry + images</dt><dd>Named after the environment, so two copies never share one.</dd>
</dl>
@@ -402,25 +404,50 @@ docker load &lt; rig.tgz &amp;&amp; make cluster up PROFILE=offline <span clas
<h3>Configuration layers</h3>
<p>Weakest first, later wins: built-in defaults → pinned versions → a
profile, if you name one → <code>ctrl/.env</code> the environment. So
profile, if you name one → the overlay's <code>rig.env</code>
<code>ctrl/.env</code> → the environment. So
<code>make cluster up PROFILE=&lt;name&gt;</code> always beats every file.</p>
</div>
</section>
<section class="section" id="overlays">
<h2>Overlays</h2>
<p class="lede">What runs lives outside rig — rig reads it and never writes into it.</p>
<div class="prose">
<p>An overlay is one folder, kept outside rig's version control, holding a
use case. Every piece is optional:</p>
<table>
<tr><th>in the overlay</th><th>what rig does with it</th></tr>
<tr><td><code>rig.env</code></td><td>a config layer: addons, namespaces, images — anything a profile could set</td></tr>
<tr><td><code>k8s/overlays/dev/</code></td><td>the manifests the dev loop applies</td></tr>
<tr><td><code>kind-config.yaml.tpl</code></td><td>the cluster's shape, when it needs its own (mounts, ports)</td></tr>
<tr><td><code>addons/&lt;name&gt;.sh</code></td><td>addons, found before rig's own</td></tr>
<tr><td><code>Tiltfile</code></td><td>the workload's half of the dev loop, included by rig's</td></tr>
</table>
<pre><code>cp -r examples/starter local/myenv <span class="c"># local/ is gitignored</span>
OVERLAY=local/myenv make cluster up
</code></pre>
<p>With none named, rig runs its own <code>examples/starter</code>.
<code>examples/data</code> carries postgres, redis and airflow as an
overlay's own addons. A project can also carry rig at
<code>&lt;project&gt;/rig/</code> and be the overlay itself, with a
three-line forwarding Makefile — see <code>docs/notes/overlay.md</code>.</p>
</div>
</section>
<section class="section" id="profiles">
<h2>Profiles</h2>
<p class="lede">Optional overlays — rig needs none.</p>
<p class="lede">How this machine reaches the world — optional; rig needs none.</p>
<div class="prose">
<table>
<tr><th>example</th><th>registry</th><th>for</th></tr>
<tr><td><i>none</i></td><td>local</td><td>the built-in defaults; no profile needed</td></tr>
<tr><td><code>client.env.example</code></td><td>mirror</td><td>images through a corporate registry</td></tr>
<tr><td><code>mirror.env.example</code></td><td>mirror</td><td>images through an internal registry</td></tr>
<tr><td><code>offline.env.example</code></td><td>local</td><td>air-gapped</td></tr>
<tr><td><code>data.env.example</code></td><td>local</td><td>postgres, redis, airflow</td></tr>
</table>
<div class="note"><p><b>The kind config cannot be re-applied.</b> Edit
<code>ctrl/k8s/kind-config.yaml.tpl</code>; it takes effect when the cluster is created. <code>cluster up</code> prints what it
<code>ctrl/k8s/kind-config.yaml.tpl</code> (or the overlay's own); it takes effect when the cluster is created. <code>cluster up</code> prints what it
locks in before spending the time, and <code>make cluster reset</code> is
the way out.</p></div>

View File

@@ -15,7 +15,13 @@ docker build -f ctrl/Dockerfile.deps --target deps -t <slug>-deps .
docker build -f ctrl/Dockerfile.deps --target deps-full -t <slug>-deps:full .
```
`deps-full` bakes every pinned binary in at build time. `docker save` it and you have the whole installer as one file to carry into an air-gapped network.
`deps-full` bakes every pinned binary in at build time, and the manifests rig's own addons apply (metallb, cert-manager, metrics-server). `docker save` it and you have the whole installer as one file to carry into an air-gapped network. There, put the manifests where the addons look for them:
```
docker run --rm -v "$PWD/vendor:/out/vendor" rig-deps:full manifests --to /out/vendor/manifests
```
Each is verified against its pin on the way out, and again when an addon uses it. The addons' container images still have to be preloaded into the local registry: the manifests reference quay.io and registry.k8s.io, and registry mirroring covers docker.io only.
## Packages

View File

@@ -1,33 +1,35 @@
# ctrl/Dockerfile.example
# examples/starter/Dockerfile.example
## Naming
EXAMPLE — a component image. Copy, rename, replace. Named like the manifest it feeds and the resource it becomes:
```
ctrl/Dockerfile.api -> image <cluster>-api -> image: in k8s/base/api.yaml
Dockerfile.api -> image <cluster>-api -> image: in k8s/base/api.yaml
```
That image string is the ONLY thing connecting the three. Nothing checks it; a typo shows up as a pod stuck in ImagePullBackOff pulling from the public index, which reads like a network problem and is not one.
## COPY paths are repo-root relative (the one that catches everyone)
## COPY paths are relative to the build context
The Tiltfile passes two paths with DIFFERENT bases, in adjacent arguments:
The overlay's Tiltfile runs from the overlay's own folder (rig's ctrl/Tiltfile includes it), so both paths in `docker_build` are relative to the overlay:
```
context='..' the REPO ROOT (the Tiltfile is in ctrl/)
dockerfile='Dockerfile.api' relative to the TILTFILE, so ctrl/Dockerfile.api
context='.' the overlay folder (or a subfolder, e.g. 'repodir/api')
dockerfile='Dockerfile.api' relative to the overlay's Tiltfile too
```
So every COPY is resolved against the repo root, NOT against the Dockerfile's directory. A file sitting right beside it is still reached as `ctrl/`:
Every COPY is resolved against the context, NOT against the Dockerfile's directory. With `context='.'` and the Dockerfile in a subfolder, a file sitting right beside it is still reached through that subfolder:
```
COPY ctrl/nginx.conf /etc/nginx/conf.d/default.conf # correct
COPY nginx.conf /etc/nginx/conf.d/default.conf # fails — no such file
COPY docker/nginx.conf /etc/nginx/conf.d/default.conf # correct, Dockerfile in docker/
COPY nginx.conf /etc/nginx/conf.d/default.conf # fails — no such file in the context
```
Nothing warns you. The build just cannot find a file that is visibly there.
Before overlays, rig's own ctrl/Tiltfile built with `context='..'` (the repository root) and the Dockerfile in ctrl/, which is the same trap one level up.
## Dependency layer
Dependencies first, in their own layer: they change far less often than the code, so a source edit does not reinstall them on every rebuild.
@@ -37,7 +39,7 @@ Dependencies first, in their own layer: they change far less often than the code
The sync in the Tiltfile's `docker_build` must land where this image expects it:
```
live_update=[sync('../api', '/app/api')]
live_update=[sync('api', '/app/api')]
```
matches `COPY api/ ./api/` with `WORKDIR /app`. If the two disagree, Tilt syncs into a path nothing reads and the container keeps serving the built copy — edits appear to do nothing, with no error anywhere.

View File

@@ -13,21 +13,23 @@ make tilt / docs work on it, read about it
The logic lives in the scripts, never here: `make cluster up` -> ctrl/cluster.sh up.
Config layers, weakest first: built-in defaults < ctrl/versions.env (pinned toolchain) < ctrl/env.d/<profile>.env (optional) < ctrl/.env (local, gitignored) < the environment. So `make cluster up PROFILE=<name>` beats them all. See [config.md](config.md).
Config layers, weakest first: built-in defaults < ctrl/versions.env (pinned toolchain) < ctrl/env.d/<profile>.env (optional) < <overlay>/rig.env (optional) < ctrl/.env (local, gitignored) < the environment. So `make cluster up PROFILE=<name>` beats them all. See [config.md](config.md) and [overlay.md](overlay.md).
Start with: `make check && make deps && make cluster up`
## FACTS
Identity follows the FOLDER NAME, so this directory can be copied elsewhere, renamed, and run as a separate environment with no edits. ctrl/.env overrides it when you want a name that differs from the directory.
Identity follows the FOLDER NAME the overlay's when one is named, else this directory's so either can be copied elsewhere, renamed, and run as a separate environment with no edits. ctrl/.env overrides it when you want a name that differs from the folder.
Asked once, of ctrl/ports.sh, which resolves it through lib/config.sh:
```
CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR
CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR OVERLAY_DIR
```
Read positionally, so the order is a contract; ctrl/selftest.sh pins it.
Read positionally, so the order is a contract; ctrl/selftest.sh pins it. The two paths are absolute, or `-` when there is none, so the count never shifts.
`OVERLAY` and `CLUSTER` given as make arguments (`make tilt OVERLAY=local/x`) are handed to that `$(shell ...)` explicitly. Before make 4.4, `$(shell)` runs with make's own environment and does not see command-line variables, while the recipes do: tilt would then be told one context and the Tiltfile would guard on another. An overlay's forwarder avoids the question by putting `OVERLAY` in the environment.
This used to be sed over ctrl/.env plus a slug computed in the Makefile, which is a SECOND derivation of values lib/config.sh already owns, and the two could disagree about the port after `ports.sh persist`, or about the name for any directory whose sanitised form differs from its raw one. One source now; the Tiltfile reads the same line.

View File

@@ -2,23 +2,19 @@
## Purpose and ownership
This file ships with rig and works unedited: rig's own k8s/base already boots, so `make tilt` comes up with a running cluster and no editing at all. What it deploys is two EXAMPLES — replace them, and add your own images and resources in the two marked sections near the bottom. The catalogue after them has the blocks to paste, with the parts that are easy to get wrong already commented.
rig's half of the dev loop, and rig's file: who we are, the context guard, the registry, the overlay's manifests and the namespaces they use. The workload's half — images, resource names and order, port-forwards — is the overlay's own `Tiltfile`, which this one includes at the end (see [overlay.md](overlay.md)). With no overlay named that is `examples/starter/Tiltfile`, so `make tilt` on a fresh clone comes up with the two examples running and nothing to edit.
rig supplies this file; it does not own it. Nothing in rig reads it back, and nothing here is regenerated — edit it freely, the way you would edit k8s/base/example-mock.yaml. rig owns the machine, you own the workload.
Splitting it is what lets rig be replaced as a whole (✖ S1 in STALE.md).
## Nothing hardcoded to this directory
Nothing in the Tiltfile is hardcoded to this directory, deliberately. Every other project here writes its slug into the Tiltfile five or six times by hand, so a copy of the project deploys into the original's cluster until someone remembers to edit all of them. A rig is meant to be copied and renamed, so it asks instead.
Nothing in the Tiltfile is hardcoded to this directory, deliberately. Every other project here writes its slug into the Tiltfile five or six times by hand, so a copy of the project deploys into the original's cluster until someone remembers to edit all of them. A rig is meant to be copied and renamed, and an overlay moved, so it asks instead.
## Who we are, and on which ports
One question to rig, answered by ctrl/ports.sh, which resolves it through lib/config.sh — the same path every other rig script takes. That is the point: the cluster name is NOT the bare directory name (it is lowercased and reduced to a DNS label), and the ports honour anything pinned in ctrl/.env. Recomputing either of those here in Starlark is how two copies end up disagreeing about which cluster they are talking to.
## Where the manifests live
rig's own manifests are the default; point MANIFESTS_DIR in ctrl/.env at an overlay versioned somewhere else and rig stops owning them — see k8s/README.md. Real manifests usually change on a different cadence, by different people, under different review.
The value is REPO-ROOT relative, because that is the root everything else in rig is expressed against. The Tiltfile runs in ctrl/, so prefix rather than assume: `'../' + 'ctrl/k8s/overlays/dev'` and `'../' + '../platform/overlays/dev'` are both right, where stripping a leading `ctrl/` would only fix the first.
The manifests and the overlay arrive as absolute paths, or `-` when there is none.
## Refuse to deploy into the wrong cluster
@@ -28,23 +24,20 @@ Tilt snapshots the kubectl context at startup, BEFORE parsing this file, so it c
Fail closed. Tilt can usually infer the kind registry on its own, but "usually" is an inference, and when it misses, an unqualified name like `app` quietly means docker.io/library/app — a push to the public index instead of the registry two lines away. rig runs that registry; name it.
## Catalogue
The catalogue holds the shapes that recur across every project here, with the reasoning kept next to them. They are comments so the file runs as-is.
## Catalogue: build an image
## Namespaces
The one genuinely non-obvious thing in the whole corpus: `context` and `dockerfile` are relative to DIFFERENT directories, in adjacent arguments, and nothing warns you.
Every namespace the manifests use has to exist before anything lands in it, and kustomize does not guarantee ordering across resources, so the Tiltfile creates them first (idempotent). The Namespaces the manifests declare are grouped as the `infra` resource, whatever they are called.
- `context=` the REPO ROOT — the Tiltfile is in ctrl/, so `'..'`
- `dockerfile=` relative to THIS file — so `'Dockerfile.api'` is ctrl/Dockerfile.api
Nothing here assumes a namespace is named after the cluster (✖ S4).
Every COPY inside those Dockerfiles is therefore repo-root relative: a file sitting BESIDE the Dockerfile is still reached as `COPY ctrl/nginx.conf`.
## Handing over to the overlay
## Catalogue: reload the gateway when its config changes
The facts are published as environment variables (`os.putenv`) and the overlay's Tiltfile is `include()`d. An included Tiltfile runs from its own folder: `os.getcwd()`, `local()` and every relative path in it resolve from the overlay, so it needs no path back into rig and reads the facts with `os.getenv`:
A Caddyfile arriving via configMapGenerator with disableNameSuffixHash does NOT roll the pod — the ConfigMap name never changes, so nothing tells the Deployment anything happened. Without the `gateway-reload` local_resource you edit the routes and watch nothing take effect.
```
RIG_CLUSTER RIG_CONTEXT RIG_HTTP_PORT RIG_HTTPS_PORT RIG_TILT_PORT RIG_REGISTRY RIG_OVERLAY_DIR
```
## Catalogue: reach a service directly, bypassing the gateway
## Catalogue
For a DB client or an admin UI. Prefer routing through the gateway: host ports are a single shared namespace across every project on this machine, which is why rig derives a block per environment in the first place. If you do need one, take it from this environment's own block rather than picking a number.
The blocks that recur across projects moved with the workload's half: `examples/starter/Tiltfile` carries them, commented, with the parts that are easy to get wrong explained next to them — building an image, a shared base built once, naming and ordering resources, reloading a gateway on a config change, kustomize flags, and reaching a service directly.

View File

@@ -2,18 +2,12 @@
## addons.sh
Each addon is its own idempotent script in `ctrl/addons/` — adding one is adding
a file, not editing a dispatcher.
Each addon is its own idempotent script — adding one is adding a file, not editing a dispatcher. `ADDONS` names them, in install order; the overlay's `addons/<name>.sh` is found before rig's `ctrl/addons/<name>.sh`, and every one runs from rig's `ctrl/` with `RIG_CTRL` exported, wherever its file lives (see [overlay.md](overlay.md)).
## airflow.sh
## What rig ships, and what it does not
Airflow needs a metadata database before it will start at all, so the script
refuses rather than rolls a pod that will CrashLoopBackOff while the real problem
(postgres missing from `ADDONS`) stays invisible in the logs.
rig's own addons make the *cluster* work, and are useless outside one: metallb, cert-manager, metrics-server. Things a workload happens to need — a database, a cache, a scheduler — are the workload's, and which workload needs which is not rig's business, so they live with the overlay. `examples/data/addons/` has postgres, redis and airflow as a worked example; an overlay that wants them copies them in.
One pod on `standalone`, matching the compose cabinet: migration, admin user,
scheduler and webserver in a single container. The official chart's five
deployments model an installation; switching this on means wanting pipelines.
## cert-manager.sh
@@ -47,27 +41,3 @@ kind nodes serve kubelet metrics over a self-signed cert, so the standard
manifest never becomes ready without `--kubelet-insecure-tls`. That is fine here
(it is a local cluster) and is the single most common reason metrics-server sits
at 0/1 on kind.
## postgres.sh — cabinets
A cabinet is a public service dropped into the environment as-is — the upstream
image, unmodified, reachable at a known address. `postgres.sh` is the cluster
half of it; the compose half is a `service.yml` beside a `cabinet.json`. The
declaration is made once and both paths read it, so nothing is remembered twice.
## postgres.sh — plain manifests, one replica
Plain manifests rather than a helm chart, matching the other addons: a chart repo
is a network dependency, and the offline example profile exists precisely so
there is a path with none. The image is pinned in `ctrl/versions.env` and can be
preloaded into a local registry like every other image here.
One replica on a PVC. This models a dependency for local work, not a
highly-available database, and pretending otherwise on a kind node would be a
more elaborate lie rather than a more useful one.
## redis.sh
Cache, and the broker anything queue-shaped runs on. No persistence: a broker
that loses its queue on restart is the honest local model, and a PVC here buys
nothing but a volume to clean up.

View File

@@ -42,3 +42,9 @@ ports, registry, addons — and adds detail only where something needs attention
as the full report did before 2026-09-17. `deps.sh detect all` is the same switch for the host part,
so the standalone `rigdeps.sh detect` is short too. Changed because the long report buried the few
lines that mattered.
## overlay and kind config
The rig block names the overlay when one is set (with `all`: what it provides — rig.env, manifests, kind config, addons, Tiltfile — and where the manifests and kind config resolved to).
Two `!` lines belong to overlays. An older ctrl/.env that still pins `MANIFESTS_DIR=ctrl/k8s/overlays/dev` — rig's examples, before they moved — is reported; load_config ignores it until then. And a kind config without the containerd `config_path` patch is reported whenever a registry mode needs it: registry.sh writes per-host config into certs.d, containerd only reads it if the cluster was created with that patch, and an overlay's own kind file replaces rig's whole template, so dropping it is easy and fails silently.

View File

@@ -7,7 +7,8 @@ The ecosystem convention is that scripts are standalone with no shared log libra
```
built-in defaults in load_config; fill only what nothing else set
ctrl/versions.env pinned toolchain + image digests (committed)
ctrl/env.d/<profile> addons, registry (OPTIONAL, examples ship as *.env.example)
ctrl/env.d/<profile> how this machine reaches the world (OPTIONAL, examples ship as *.env.example)
<overlay>/rig.env what runs: addons, namespaces, images (OPTIONAL, lives with the overlay)
ctrl/.env machine-local values and secrets (gitignored)
the caller's env `make cluster up PROFILE=<name>` (always wins)
```
@@ -20,11 +21,15 @@ Run from ctrl/.
Values a user can reasonably override per-invocation. Anything set in the environment when load_config runs is restored after the files are read. NODES is deliberately NOT here: it is read back out of the kind config, so the file is the one place that decides it.
REGISTRY_PORT and MANIFESTS_DIR were missing here while ctrl/.env set them, so the caller's env silently LOST to the file for those two, breaking the one precedence rule the header states. Both are now listed; the other twelve are unchanged.
REGISTRY_PORT and MANIFESTS_DIR were missing here while ctrl/.env set them, so the caller's env silently LOST to the file for those two, breaking the one precedence rule the header states. Both are now listed. OVERLAY joined with overlays, for the same reason: it is chosen per machine or per call.
## default_cluster_name
The containing folder's name, reduced to something kind accepts as a cluster name (a DNS label: lowercase alphanumerics and dashes). Run from ctrl/, so the repo root is the parent.
The environment's folder name — the overlay's when one is named, else rig's own — reduced to something kind accepts as a cluster name (a DNS label: lowercase alphanumerics and dashes). Run from ctrl/, so rig's folder is the parent.
## _from_ctrl, _abs_from_ctrl
Paths in the config are relative to rig's folder (MANIFESTS_DIR, OVERLAY) or to ctrl/ (KIND_CONFIG), or absolute. Scripts run from ctrl/, so `_from_ctrl` turns a rig-relative path into one usable from there, and `_abs_from_ctrl` into an absolute one for consumers outside bash (ports.sh active, the kind config's hostPath entries).
## derive_port_base
@@ -38,9 +43,15 @@ RIG_PORTABLE skips the machine-local layer. config_snapshot sets it, so a genera
A profile is an optional overlay, never a prerequisite. rig assumes no configuration: with no profile named, or no env.d/ at all, it runs on the built-in defaults. What IS an error is naming a profile that does not exist, because a typo must not quietly fall back to something else.
## load_config: overlays
An overlay is one folder, outside rig's version control, that holds what runs ([overlay.md](overlay.md)). `OVERLAY` names it; a named overlay that does not exist is an error, like a named profile. With none named, rig's own `examples/starter` is used if it is present — it sets nothing, so a plain rig resolves as it did before overlays — and a rig copied without `examples/` still resolves, with no manifests.
Its `rig.env` is layered after the profile and before ctrl/.env. It may not set PROFILE or OVERLAY, which are chosen before it loads, and the paths it sets are relative to the overlay (load_config rewrites them as it loads the file), so an overlay can be moved without editing it.
## load_config: identity follows the folder
Identity follows the FOLDER, so copying this directory somewhere else and renaming it yields a distinct environment with no further edits. Without this, two copies would share one cluster and `make cluster down` in either would destroy the other's.
Identity follows the FOLDER — the overlay's when one is named, else rig's — so copying either somewhere else and renaming it yields a distinct environment with no further edits. Without this, two copies would share one cluster and `make cluster down` in either would destroy the other's. It is also what lets a project carry rig at `<project>/rig/` without every such project's cluster being called `rig`.
## load_config: host ports
@@ -48,7 +59,7 @@ Host ports are a single shared namespace, so unlike the cluster name they cannot
## load_config: MANIFESTS_DIR
Where the workload's manifests live, repo-root relative. Defaulted here so it is always resolved rather than sometimes-set: it is the seam that lets the real manifests be versioned away from the installer, and a consumer should not have to know whether anyone filled it in. See k8s/README.md.
Where the workload's manifests live, relative to rig's folder or absolute: the overlay's `k8s/overlays/dev` unless something names another. It is the seam that lets the real manifests be versioned away from the installer. `none` means rig applies none (the overlay's Tiltfile does). A named folder that does not exist is an error; the old default `ctrl/k8s/overlays/dev`, pinned by older .env files, is ignored while that folder does not exist and reported by `make check`.
## load_config: NODE_MB
@@ -60,7 +71,7 @@ It is set here rather than in check.sh because the memory tool and every standal
Renders the kind config to stdout. sed rather than envsubst: envsubst is gettext-base, absent from a minimal Debian, and Docker is meant to be the only prerequisite. The variable list is explicit so a template cannot quietly start depending on something the caller does not set.
hostPath entries are resolved by the HOST dockerd, so HOST_WORKDIR must stay a host path even when this runs inside the installer container.
hostPath entries are resolved by the HOST dockerd, so HOST_WORKDIR and OVERLAY_DIR must stay host paths even when this runs inside the installer container. `${OVERLAY_DIR}` renders to the overlay's absolute path, for mounting its folders into the nodes.
## What a standalone kit needs to know

View File

@@ -156,3 +156,7 @@ The "put OUT_BIN on PATH" advice is only worth giving when something actually la
## main: argument shift
Read the command, THEN shift — and shift only if there is something there. A bare `shift` with no positional parameters returns 1, and under `set -e` that ended the script before a single line was printed: running this with no arguments at all, the documented default, did nothing and said nothing.
## manifest / manifests
The manifests rig's own addons apply are pinned in versions.env like the binaries, and fetched by the same code: `resolve_url` for the source (upstream, artifactory, baked), `verify` for the sum. `manifest <NAME>` makes one present in `vendor/manifests/` (rig's folder, gitignored) and prints only its path, so an addon can apply it; a cached copy whose sum still matches is reused, one that does not is fetched again. `manifests [--to DIR]` fetches all three, which is how the deps-full image bakes them and how an offline machine is given them. Why the addons stopped applying URLs is in versions.md.

View File

@@ -12,12 +12,18 @@ The kubectl context becomes kind-<CLUSTER>. LEAVE THIS UNSET unless you need a n
LEAVE UNSET — they derive from the directory name so several environments coexist without negotiating (see ctrl/ports.sh). `make check` shows this environment's block; `bash ctrl/ports.sh persist` writes it into ctrl/.env so it stops being derived and becomes fixed. Set a value only to override.
## ctrl/.env.example: OVERLAY
The folder that holds what runs — its settings (`rig.env`), manifests, addons, Tiltfile — kept outside rig's version control: `local/<name>` (gitignored), or a repo of its own anywhere. Relative to rig's folder, or absolute. Unset, rig runs its own `examples/starter`. The cluster, context and port block follow the overlay's folder name. See [overlay.md](overlay.md).
## ctrl/.env.example: MANIFESTS_DIR
Where the application manifests live. The real ones are expected to be versioned separately from this installer — they change on a different cadence, by different people. Repoint this at their repo and rig stops owning them:
Where the manifests live. Leave it unset: the overlay's `k8s/overlays/dev` is the default. Set it only to point somewhere else, relative to rig's folder or absolute:
MANIFESTS_DIR=../platform-manifests/overlays/dev
Older copies of this file set `MANIFESTS_DIR=ctrl/k8s/overlays/dev`, rig's examples before they moved to `examples/`. That value is ignored while the folder does not exist, and `make check` says to delete the line.
## ctrl/.env.example: DEPS_SOURCE
Where the installer fetches the pinned binaries from.
@@ -36,11 +42,13 @@ Corporate root CA, if Artifactory is fronted by an internal CA (it usually is).
## env.d/*.env.example: profiles in general
EXAMPLE PROFILES. rig needs none of these: with no profile it runs on its built-in defaults (lib/config.sh). To use one, copy it to <name>.env in ctrl/env.d/ and name it — PROFILE=<name> in ctrl/.env, or on the command line. It then overlays the defaults; anything it does not set, they still supply.
EXAMPLE PROFILES. rig needs none of these: with no profile it runs on its built-in defaults (lib/config.sh). To use one, copy it to <name>.env in ctrl/env.d/ and name it — PROFILE=<name> in ctrl/.env, or on the command line. It then overlays the defaults; anything it does not set, they still supply. An activated `<name>.env` is gitignored: it is this machine's choice.
## env.d/client.env.example
A profile says how this machine reaches the world — a registry mirror, an air-gapped install. What runs is an overlay's business ([overlay.md](overlay.md)); its `rig.env` layers above the profile.
client — images through a pull-through cache of the corporate registry, with TLS and metrics addons. More nodes or port mappings: edit k8s/kind-config.yaml.tpl.
## env.d/mirror.env.example
mirror — images through a pull-through cache of an internal registry, with TLS and metrics addons. More nodes or port mappings: edit the kind config (rig's, or the overlay's).
### Real ports (80/443)
@@ -48,27 +56,9 @@ Ports derive from the directory name by default (see ctrl/ports.sh), so several
Opt in to the real ports only when this is the ONLY environment and nothing else owns :80. They fail to bind otherwise, and docker reports it as an opaque "failed to bind host port 0.0.0.0:80/tcp: address already in use" halfway through cluster creation. `make check` checks before you spend the time. Uncommenting also means only one environment can exist at a time.
## env.d/data.env.example
data — databases and a scheduler for an environment that needs them: postgres, redis and airflow, each an upstream image run unmodified.
Everything lands in the `data` namespace (DATA_NAMESPACE to move it), so `make cluster reset` on the app namespace leaves the databases alone.
Costs roughly 2-3 GB with airflow, under 1 without. Airflow's first boot runs the whole metadata migration, so expect a few minutes before it is ready.
### Postgres password
The password is not in the profile: postgres.sh generates one on first install and keeps it across re-runs, so re-running the addon never rotates the credential out from under whatever is already connected.
### Reaching the databases
Ports derive from the directory name by default — see ctrl/ports.sh. Reach the databases with port-forward rather than binding more host ports:
kubectl -n data port-forward svc/postgres 5432:5432
kubectl -n data port-forward svc/airflow 8080:8080
## env.d/offline.env.example
offline — air-gapped. Everything comes from a local registry that was loaded ahead of time; nothing reaches the internet. Pair with the deps-full image (DEPS_SOURCE=baked) so the toolchain install is offline too.
offline — air-gapped. Everything comes from a local registry that was loaded ahead of time; nothing reaches the internet. Pair with the deps-full image (DEPS_SOURCE=baked) so the toolchain install is offline too, and so the manifests metallb and the other addons apply come from the image, verified, rather than from GitHub (see Dockerfile.deps.md). Their container images still have to be preloaded.
The heavier addons are left out to keep first boot viable.

View File

@@ -4,11 +4,17 @@
The cluster: one node by default — add nodes or port mappings by editing the file, then `make cluster reset`.
It is a TEMPLATE rather than a plain kind-config.yaml because a rig is copied and renamed to make a second environment, and both the cluster name and the host port follow the directory. A checked-in literal would make every copy collide on both. ctrl/cluster.sh renders it with sed — not envsubst, which is gettext-base and absent from a minimal Debian, and rig's whole premise is that Docker is the only prerequisite.
It is a TEMPLATE rather than a plain kind-config.yaml because a rig (or an overlay) is copied and renamed to make a second environment, and both the cluster name and the host port follow the folder. A checked-in literal would make every copy collide on both — which is exactly why every other project here, with its literal kind-config.yaml, has only one of itself. ctrl/cluster.sh renders it with sed — not envsubst, which is gettext-base and absent from a minimal Debian, and rig's whole premise is that Docker is the only prerequisite.
A kind config is fixed at creation: to change the cluster, edit the file, then `make cluster reset`. lib/config.sh reads the node count back out of it, so nothing restates it.
## An overlay's own kind config
An overlay may carry its own `kind-config.yaml.tpl` (see [overlay.md](overlay.md)); it replaces this whole file, rendered the same way, so start from a copy of this one. Keep the containerd `config_path` patch: `make check` reports its absence whenever a registry mode needs it. A project that builds its own cluster through rig can also pass any file as `KIND_CONFIG=<path>`.
## Substituted variables
Substituted by ctrl/cluster.sh: CLUSTER, NODE_IMAGE, HTTP_PORT, HOST_WORKDIR. The header comment names them without the `${...}` braces so that line survives the substitution.
Substituted by ctrl/cluster.sh: CLUSTER, NODE_IMAGE, HTTP_PORT, HOST_WORKDIR (rig's folder), OVERLAY_DIR (the overlay's folder, for mounts). The header comment names them without the `${...}` braces so that line survives the substitution.
## Node count

171
rig/docs/notes/overlay.md Normal file
View File

@@ -0,0 +1,171 @@
# Overlays: what runs lives outside rig
## Why
rig is the machine: the toolchain, the cluster, the registry, the port block, the
dev loop's plumbing. What runs on it — the services, their manifests, their
images, their settings — belongs to whoever owns that work, changes at a
different rate, and often cannot be shared at all. Keeping both in one tree meant
editing rig's own files to use it, and then carrying those edits into every copy.
So the use case is one folder, the **overlay**, kept outside rig's version
control. rig reads it; rig never writes into it and never knows what is in it.
The dependency points one way: an overlay knows about rig, rig knows about
overlays in general and about none in particular.
## Where an overlay lives
```
rig/local/<name>/ gitignored by rig: an overlay with no version control of its own,
or a clone of its own repo
anywhere/<name>/ a repo of its own, named by path
<project>/ a project folder that carries rig at <project>/rig/ (the vendored
layout, below)
```
Name it with `OVERLAY` — in `ctrl/.env` for this machine, or per call:
```bash
OVERLAY=local/myenv make cluster up
```
Relative paths are relative to rig's folder. With none named, rig uses its own
`examples/starter`, which sets nothing, so a plain rig behaves as it always did.
An overlay that is named and missing is an error; it never falls back.
## What rig reads from it
Every piece is optional.
| in the overlay | what rig does with it |
| --- | --- |
| `rig.env` | a config layer (below). Any key a profile could set. |
| `k8s/overlays/dev/` | the default `MANIFESTS_DIR`: rig's Tiltfile applies it with kustomize |
| `kind-config.yaml.tpl` | the default `KIND_CONFIG`: the cluster's shape, rendered like rig's own |
| `addons/<name>.sh` | an addon, found before rig's `ctrl/addons/<name>.sh` of the same name |
| `Tiltfile` | the workload's half of the dev loop, included by rig's `ctrl/Tiltfile` |
Anything else in the folder is the overlay's own business: Dockerfiles, DAGs,
folders of repos or data it mounts, its `.gitignore`, the secrets its kustomize
generators read. rig does not look.
## Layers
```
built-in defaults < ctrl/versions.env < ctrl/env.d/<profile>.env < <overlay>/rig.env < ctrl/.env < the caller
```
A profile says how this machine reaches the world (a registry mirror, an
air-gapped install); an overlay says what runs. `ctrl/.env` is still this
machine's, and the caller still wins over everything.
`rig.env` may not set `PROFILE` or `OVERLAY`: both are chosen before it loads.
The paths it sets (`MANIFESTS_DIR`, `KIND_CONFIG`) are relative to the overlay.
`MANIFESTS_DIR=none` means rig applies no manifests and the overlay's Tiltfile
does, e.g. when kustomize needs flags.
## Identity
With an overlay named, the cluster, the kubectl context and the port block
follow the overlay folder's name, sanitised the same way a rig folder's name is.
One rig can therefore serve several overlays, each in its own cluster, and a
project that carries rig at `./rig` does not name every cluster `rig`.
`ports.sh persist` refuses while an overlay is set: it writes to rig's
`ctrl/.env`, and a pin there would follow every overlay.
## The Tiltfile handoff
rig's `ctrl/Tiltfile` does rig's part — the context guard, `default_registry`,
the manifests, the namespaces they use — then publishes the facts and includes
the overlay's `Tiltfile`:
```
RIG_CLUSTER RIG_CONTEXT RIG_HTTP_PORT RIG_HTTPS_PORT RIG_TILT_PORT RIG_REGISTRY RIG_OVERLAY_DIR
```
Read them with `os.getenv`. An included Tiltfile runs from its own folder, so
every relative path in it (`docker_build` contexts, `sync`, `deps`, `local`) is
relative to the overlay — it never needs a path back into rig.
## Addons
An addon is a bash script run by `ctrl/addons.sh` from rig's `ctrl/`, with
`RIG_CTRL` exported. It starts like this, sources the config and does its work:
```bash
cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
source ./lib/config.sh
load_config
```
`OVERLAY_DIR` is set, so an addon can find files beside it
(`$(_from_ctrl "$OVERLAY_DIR")/...`). rig's own `ctrl/addons/` holds only what
makes a cluster work (metallb, cert-manager, metrics-server);
`examples/data/addons/` shows workload ones.
## The kind config
An overlay's `kind-config.yaml.tpl` replaces rig's whole file, so start from a
copy of `ctrl/k8s/kind-config.yaml.tpl` and keep its containerd `config_path`
patch: `registry.sh` needs it, and `make check` says so when it is missing.
`${OVERLAY_DIR}` renders to the overlay's absolute path, for mounts:
```yaml
extraMounts:
- hostPath: ${OVERLAY_DIR}/datadir
containerPath: /rig/datadir
```
A kind config is fixed when the cluster is created: after changing it,
`make cluster reset`.
## The vendored layout
A project folder can carry rig inside it and be the overlay itself:
```
<project>/
Makefile the forwarder below
rig.env k8s/ Tiltfile kind-config.yaml.tpl addons/ ...
rig/ rig, placed as it is; tracked or ignored by the project, its call
```
The forwarder runs rig with `OVERLAY` set to this folder. It passes `OVERLAY` in
the environment, not as a make argument, so rig's own `$(shell ...)` sees it
under make 4.3 as well:
```make
HERE := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
ifeq ($(wildcard $(HERE)/rig/Makefile),)
$(error rig/ is missingthis folder is an overlay; put rig in ./rig)
endif
GOALS := $(or $(MAKECMDGOALS),help)
.PHONY: $(GOALS)
$(firstword $(GOALS)):
@OVERLAY='$(HERE)' $(MAKE) --no-print-directory -C '$(HERE)/rig' $(GOALS)
$(wordlist 2,$(words $(GOALS)),$(GOALS)):
@:
```
The cluster is then named after `<project>`, exactly as a copied rig named
`<project>` was, so moving a copied rig to this layout keeps its cluster and ports.
`rig.env` holds no secrets, by this contract. A repository whose `.gitignore` has a
broad `*.env` (a common secrets rule) would still hide it, so an overlay living in
such a repo re-includes it in its own `.gitignore`: `!rig.env`.
## Moving a copied rig to an overlay
A rig copied into a project and edited there splits cleanly:
| was, in the copy | goes to |
| --- | --- |
| `ctrl/k8s/base`, `ctrl/k8s/overlays` | `k8s/` |
| the workload parts of `ctrl/Tiltfile` | `Tiltfile` (paths now relative to the overlay) |
| `ctrl/env.d/<name>.env` | `rig.env` |
| edits to `ctrl/k8s/kind-config.yaml.tpl` | `kind-config.yaml.tpl` (`${HOST_WORKDIR}``${OVERLAY_DIR}`) |
| workload addons | `addons/` |
| Dockerfiles for the workload | beside the Tiltfile |
| `ctrl/.env` | `rig/ctrl/.env` (this machine's; drop a `MANIFESTS_DIR=ctrl/k8s/overlays/dev` line) |
| everything else of rig's | replaced by rig as it is |

View File

@@ -20,12 +20,18 @@ Derivation is a default, not a decision. On first use the resolved block is writ
The resolved facts a consumer outside bash needs, machine-readable:
```
CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR
CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR OVERLAY_DIR
```
Identity and ports together, because they are one fact set: both derive from the directory name so that copies never collide. A consumer needs all of them or none, and fetching them separately is how two end up disagreeing. MANIFESTS_DIR rides along because the one consumer that needs the addressing is the one that needs to know what to deploy.
Identity and ports together, because they are one fact set: both derive from a folder name (the overlay's when one is named) so that copies never collide. A consumer needs all of them or none, and fetching them separately is how two end up disagreeing. MANIFESTS_DIR and OVERLAY_DIR ride along because the one consumer that needs the addressing is the one that needs to know what to deploy and whose Tiltfile to include.
Space-separated, so MANIFESTS_DIR must not contain spaces. Everything else in rig already assumes that of paths; kind, docker and kubectl all do.
The two paths are absolute, or `-` when there is none: an empty field would shift every later one. OVERLAY_DIR was appended rather than inserted, so readers that take fields by position kept their indexes.
Space-separated, so the paths must not contain whitespace; `active` refuses rather than print a line that splits wrong. Everything else in rig already assumes that of paths; kind, docker and kubectl all do.
## persist, with an overlay
`persist` writes into ctrl/.env, which belongs to this rig, not to an overlay. With OVERLAY set, a block pinned there would follow every overlay this rig later runs, and two of them would then share ports — the collision the derivation exists to prevent. So it refuses and says so; an overlay's ports stay derived from its folder name.
`derive` answers a DIFFERENT question (what the directory name alone implies) and deliberately ignores ctrl/.env. Configuring anything from it would silently contradict the rule that "anything already in ctrl/.env wins". `active` is what anything downstream should read.

View File

@@ -18,7 +18,7 @@ rig assumes no configuration. A profile is an overlay on built-in defaults, so a
## the ports.sh active contract
`ports.sh active` is read POSITIONALLY by two other files: the Makefile takes `$(word 2)` and `$(word 5)`, the Tiltfile takes `_facts[0]..[6]`. Insert a field in the middle and nothing errors: Tilt simply guards on the wrong context or binds the wrong port. The field count and order are the contract, so they are pinned here rather than left to whoever edits `ports.sh` next.
`ports.sh active` is read POSITIONALLY by two other files: the Makefile takes `$(word 2)` and `$(word 5)`, the Tiltfile takes `_facts[0]..[7]`. Insert a field in the middle and nothing errors: Tilt simply guards on the wrong context or binds the wrong port. The field count and order are the contract, so they are pinned here rather than left to whoever edits `ports.sh` next. The two paths are absolute or `-`, never empty: an empty field would shift the ones after it just the same.
## the caller's env beats the files
@@ -44,7 +44,21 @@ Not a change-detector. The block is derived, never stored, so if the derivation
rig sits inside a host project's tree but must be copyable straight out of it: no imports, no paths, no assumption the host is there. This grep is the whole test of that claim, and until it was added it lived only in prose and in whoever remembered to run it.
The pattern is assembled from fragments so the file does not match ITSELF. Writing it literally would fail forever; excluding the file instead would put a blind spot in the one check that guards the boundary.
The pattern is assembled from fragments so the file does not match ITSELF. Writing it literally would fail forever; excluding the file instead would put a blind spot in the one check that guards the boundary. It includes the host project's word for a backing service, which rig's workload addons carried until they left, and skips `local/`, where overlays live and may say anything.
## scratch copies
Every check that changes something does it in a copy made by `copy_rig`: without `local/` (overlays, possibly someone else's, possibly large) and `def/`, and without this machine's `PROFILE`, `OVERLAY`, `CLUSTER` and `MANIFESTS_DIR` choices, so a check sets exactly what it tests.
## what runs is an overlay; rig only reads it
The overlay decisions (docs/notes/overlay.md), each against a throwaway overlay in a scratch copy: with nothing named, the same cluster, ports, addons, node count and kind config as before overlays existed; `rig.env` between the profile and `ctrl/.env`, the caller above all; identity from the overlay's folder; its paths relative to itself; a named overlay that does not exist, or a `rig.env` that tries to choose the profile or the overlay, is an error; an overlay's addon found before rig's own and run from rig's `ctrl/`; `persist` refusing; `make -n tilt OVERLAY=...` asking for the overlay's context (make before 4.4 would not pass it to `$(shell)`).
And the two that make an overlay safe to hold someone else's work: rig writes nothing into it (a checksum of the folder before and after `active`, `addons list`, a kind render, `standalone write` and `export`), and nothing from it — neither a value nor its path — reaches a committed kit.
## withdrawn stays withdrawn
One check per entry in STALE.md, each asserting that the withdrawn thing has not come back. The reasoning stays in STALE.md; the check is what makes it more than prose.
## the Tiltfile hardcodes nothing
@@ -62,6 +76,16 @@ Each kit's Makefile exists so nothing wrapping these scripts has to GUESS how to
An export is "take the setup I have here somewhere else", so it carries this machine's CHOICES (profile, ports, manifest dir) and never its credentials: `ctrl/.env` can hold registry and mirror logins next to those choices. The committed per-profile kits carry neither, since they must be the same on any machine. Proven with sentinel values in a scratch copy, because the real `ctrl/.env` may have those keys empty, and an empty value proves nothing.
## optional: Tiltfile evaluates
## the dev loop parses — tilt and kubectl, no cluster
Parsing the Tiltfile for real is the only way to know it still evaluates. Tilt snapshots a kubectl context first, but it never contacts the cluster while evaluating: given a throwaway kubeconfig whose entries are kind-named (Tilt only runs `local()` freely for contexts it recognises as local) and a `kubectl` that swallows `apply`, `tilt alpha tiltfile-result` evaluates rig's Tiltfile with the starter overlay included, and reports the resources it would deploy.
It runs as `rig`, as a copy under another name — the case that used to stop at load (✖ S4) — and with the data overlay, whose namespace is used but not declared. Skipped, not failed, without tilt or kubectl.
## rig's addons apply verified files, never URLs
The offline example profile must install rig's addons with no network. Each addon therefore asks `deps.sh manifest <NAME>` for a pinned manifest, verified on disk, instead of applying a URL; the check fails if a URL comes back or a manifest is asked for without a pinned sum (versions.md). Their container images still need preloading, and nothing here pretends otherwise.
## the examples are overlays that work as shipped
Parsing the Tiltfile for real is the only way to know it still evaluates, but Tilt snapshots a kubectl context before parsing, so it cannot run without a cluster. Skipped rather than failed when there is none, the same way docgen skips its graphgen section.
`examples/` is what real overlays are copied from, so every addon there must parse, and every DAG must be valid Python. What they deploy is exercised by the parse checks above, not here.

View File

@@ -34,8 +34,21 @@ The distro docker packages ship the daemon and the CLI but frequently not this,
## Node images
Node images shipped with `KIND_VERSION`, pinned by digest so a kind upgrade can never silently move the k8s version. Profiles select one via `K8S_VERSION`. Older entries are kept deliberately: running a trailing-edge control plane is part of simulating a legacy estate.
Node images shipped with `KIND_VERSION`, pinned by digest so a kind upgrade can never silently move the k8s version. `K8S_VERSION` selects one (a profile or an overlay's rig.env may set it; the default is the newest pinned). Older entries are kept deliberately, for targets that run an older Kubernetes.
## Cabinets
## Workload images are not pinned here
Public services dropped in as-is, the upstream image unmodified. The same declaration installs on compose or in the cluster, so a dependency is named once and works either way. Pinned by tag rather than digest because they are ordinary upstream images with no supply chain claim attached — bump freely, and preload them for the offline profile.
What an overlay runs is pinned by the overlay: `examples/data/rig.env` carries its postgres, redis and airflow images. This file holds what rig itself needs — the toolchain, the node images, the registry and rig's own addons — so it never says what any particular environment runs.
## The addons' manifests
metallb, cert-manager and metrics-server are installed from their upstream manifests. Those are pinned here by URL and SHA256 like the binaries, fetched through the same `DEPS_SOURCE` resolver (upstream, artifactory, baked) by `deps.sh manifest <NAME>`, verified, and applied from `vendor/manifests/` — never a URL applied directly. That is what lets the offline example profile install its addons with no network: the deps-full image carries them.
cert-manager and metrics-server publish their manifests as release assets, and GitHub reports each asset's SHA256 (`digest` in the release API); those are the pinned sums. metallb does not: its manifest is a file in the repository at the release tag, with no published sum. Its pin was taken from a download whose git blob id matched the one GitHub serves for `config/manifests/metallb-native.yaml` at that tag, and the blob id is kept beside it (`METALLB_MANIFEST_GIT_BLOB`) so the next bump is checked the same way:
```
curl -sSL "https://api.github.com/repos/metallb/metallb/contents/config/manifests/metallb-native.yaml?ref=<tag>" | jq -r .sha
(printf 'blob %d\0' "$(wc -c < metallb-native.yaml)"; cat metallb-native.yaml) | sha1sum
```
Bumping an addon's version means bumping its manifest sum in the same edit; `deps.sh manifest` refuses a mismatch.

View File

@@ -0,0 +1,98 @@
# `examples/data` — an overlay with a database, a scheduler and a worked pipeline
postgres, redis and airflow, each an upstream image run unmodified, installed as
this overlay's own addons. rig ships the mechanism that finds and runs them; which
services a workload needs is the workload's business, so they live here rather
than in rig's `ctrl/addons/`. Copy what you need into your own overlay's `addons/`.
```bash
OVERLAY=examples/data make cluster up # cluster `data`: postgres, then airflow
OVERLAY=examples/data make tilt # the items-api simulator the DAG reads
```
```
rig.env ADDONS (metallb is rig's; the rest are here), namespace, identities, image pins
addons/postgres.sh one replica on a PVC; the password generated once and kept
addons/airflow.sh one `standalone` pod on its own `airflow` database; needs postgres
addons/redis.sh only for switching airflow to CeleryExecutor (not in ADDONS)
k8s/ the items-api simulator
dags/items_to_postgres.py the worked example: API client → adapter → postgres
Tiltfile names the simulator's resource
```
Everything lands in the `data` namespace (`DATA_NAMESPACE`, and the kustomization
names it too), so resetting an app's namespace leaves the databases alone. Costs
roughly 2 GB with airflow, under 1 without. Airflow's first boot runs the whole
metadata migration, so expect a few minutes before it is ready.
## The worked example: three links kept apart
- **Metadata DB (infra).** `addons/airflow.sh` creates an `airflow` database on the
same postgres and points `SQL_ALCHEMY_CONN` there — airflow's own tables never land
in the app's database.
- **DAG delivery.** The addon turns this overlay's `dags/` into the `airflow-dags`
ConfigMap, mounted at `/opt/airflow/dags`; re-run `make cluster up` after editing a
DAG. The faster path later: a kind `extraMount` of `dags/` plus a Tilt `sync` — noted,
not built.
- **Data connection (operational logic).** `AIRFLOW_CONN_APP_DB`, composed each run
from the postgres secret, gives DAGs the app's database as the `app_db` connection.
One password reaches both URLs, with nowhere to drift.
The DAG itself calls the simulator with its own HTTP client (the wire, as the API
returns it), renames the wire's fields into the app's names in `to_app_row` — **the
adapter, which belongs to whoever owns the app's model and so lives in the overlay** —
and upserts into `items`. Hourly, no backfill, one retry, idempotent on `item_id`.
```bash
kubectl --context kind-data -n data exec deploy/airflow -- airflow dags unpause items_to_postgres
kubectl --context kind-data -n data exec deploy/airflow -- airflow dags trigger items_to_postgres
kubectl --context kind-data -n data exec deploy/postgres -- psql -U app -d app -c 'select * from items'
```
## Addons in an overlay
Each one runs from rig's `ctrl/` (rig's `addons.sh` exports `RIG_CTRL`), so it
starts with `cd "${RIG_CTRL:?...}"`, sources `./lib/config.sh` and calls
`load_config` — and sees every key this overlay's `rig.env` sets. Run them through
rig (`bash ctrl/addons.sh install`, or `make cluster up`), not directly.
## postgres — plain manifests, one replica
Plain manifests rather than a helm chart: a chart repo is a network dependency,
and an offline machine needs a path with none. The image is pinned in `rig.env`
and can be preloaded into a local registry like every other image.
One replica on a PVC. This models a dependency for local work, not a
highly-available database, and pretending otherwise on a kind node would be a
more elaborate lie rather than a more useful one.
The password is not in `rig.env`: `addons/postgres.sh` generates one on first
install and keeps it across re-runs, so re-running the addon never rotates the
credential out from under whatever is already connected.
## redis
Cache, and the broker anything queue-shaped runs on. No persistence: a broker
that loses its queue on restart is the honest local model, and a PVC here buys
nothing but a volume to clean up.
## airflow
Airflow needs a metadata database before it will start at all, so the script
refuses rather than rolls a pod that will CrashLoopBackOff while the real problem
(postgres missing from `ADDONS`) stays invisible in the logs.
One pod on `standalone`: migration, admin user, scheduler and webserver in a
single container, on LocalExecutor, which needs no broker. The official chart's
five deployments model an installation; switching this on means wanting pipelines.
redis is here for the day it moves to CeleryExecutor, and not before.
## Reaching them
Reach the databases with port-forward rather than binding more host ports:
```bash
kubectl --context kind-data -n data port-forward svc/postgres 5432:5432
kubectl --context kind-data -n data port-forward svc/airflow 8080:8080
kubectl --context kind-data -n data get secret postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d
```

View File

@@ -0,0 +1,6 @@
# The data overlay's half of the dev loop: the items-api simulator the example DAG reads.
# rig's ctrl/Tiltfile has already applied k8s/overlays/dev; paths here are relative to
# this folder. postgres and airflow are addons (make cluster up), not Tilt resources.
# Notes: README.md
k8s_resource('items-api', labels=['simulator'])

View File

@@ -1,9 +1,11 @@
#!/usr/bin/env bash
# Apache Airflow — the cluster half of the airflow cabinet (one `standalone` pod).
# Apache Airflow for this overlay: one `standalone` pod (LocalExecutor — no broker).
# Its own `airflow` database on the postgres addon; the app's data reaches DAGs as the
# `app_db` connection; DAGs from this overlay's dags/, as a ConfigMap.
# Requires the postgres addon; refuses to install without it.
# Notes: docs/notes/addons.md
# Notes: ../README.md
set -euo pipefail
cd "$(dirname "$0")/.."
cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
source ./lib/config.sh
load_config
@@ -13,16 +15,25 @@ NS="${DATA_NAMESPACE:-data}"
if ! $K get deployment -n "$NS" postgres >/dev/null 2>&1; then
echo " ! airflow needs the postgres addon, and it is not installed" >&2
echo " add it before airflow in the profile's ADDONS:" >&2
echo " add it before airflow in the overlay's ADDONS:" >&2
echo " ADDONS=\"... postgres airflow\"" >&2
exit 1
fi
# Reuse the credential postgres generated rather than storing a second copy.
# ── metadata DB: airflow's own tables, kept out of the app's database ──────
# Same postgres instance, separate database, created once (idempotent).
db_user=$($K get secret -n "$NS" postgres -o jsonpath='{.data.POSTGRES_USER}' | base64 -d)
db_pass=$($K get secret -n "$NS" postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d)
db_name=$($K get secret -n "$NS" postgres -o jsonpath='{.data.POSTGRES_DB}' | base64 -d)
psql() { $K exec -n "$NS" deploy/postgres -- psql -U "$db_user" -d "$db_name" -tAc "$1"; }
if [ "$(psql "SELECT 1 FROM pg_database WHERE datname = 'airflow'")" = 1 ]; then
echo " database 'airflow' exists"
else
psql "CREATE DATABASE airflow" >/dev/null
echo " created database 'airflow' beside '${db_name}'"
fi
# ── what is generated once and kept: re-running never rotates these ────────
if $K get secret -n "$NS" airflow >/dev/null 2>&1; then
echo " secret exists, keeping the current admin password and fernet key"
else
@@ -34,11 +45,30 @@ else
--from-literal=ADMIN_USER="${AIRFLOW_ADMIN_USER:-admin}" \
--from-literal=ADMIN_PASSWORD="$admin_password" \
--from-literal=FERNET_KEY="$fernet_key" \
--from-literal=SQL_ALCHEMY_CONN="postgresql+psycopg2://${db_user}:${db_pass}@postgres:5432/${db_name}" \
>/dev/null
echo " generated an admin password (read it back with the command below)"
fi
# ── connections: composed from what the postgres secret owns, every run ─────
# Nothing to drift: one password reaches the metadata DB and the data connection.
$K create secret generic airflow-connections -n "$NS" \
--from-literal=SQL_ALCHEMY_CONN="postgresql+psycopg2://${db_user}:${db_pass}@postgres:5432/airflow" \
--from-literal=AIRFLOW_CONN_APP_DB="postgres://${db_user}:${db_pass}@postgres:5432/${db_name}" \
--dry-run=client -o yaml | $K apply -f - >/dev/null
# ── DAG delivery: this overlay's dags/ as a ConfigMap ───────────────────────
# Edits land by re-running this addon (make cluster up). The later path — a kind
# extraMount of dags/ plus a Tilt sync — is noted in the README, not built.
# A ConfigMap volume is kubelet's ..data/..<timestamp> symlinks, and Airflow's DAG walker
# follows symlinks: without the .airflowignore it stops at "Detected recursive loop".
dags="$(_from_ctrl "$OVERLAY_DIR")/dags"
if [ -d "$dags" ]; then
$K create configmap airflow-dags -n "$NS" --from-file="$dags" \
--from-literal=.airflowignore='^\.\.' \
--dry-run=client -o yaml | $K apply -f - >/dev/null
echo " dags: $(ls "$dags" | grep -c '\.py$') file(s) from $(basename "$(_abs_from_ctrl "$OVERLAY_DIR")")/dags"
fi
echo " applying manifests"
$K apply -n "$NS" -f - >/dev/null <<YAML
apiVersion: v1
@@ -79,7 +109,10 @@ spec:
value: "false"
- name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
valueFrom:
secretKeyRef: {name: airflow, key: SQL_ALCHEMY_CONN}
secretKeyRef: {name: airflow-connections, key: SQL_ALCHEMY_CONN}
- name: AIRFLOW_CONN_APP_DB
valueFrom:
secretKeyRef: {name: airflow-connections, key: AIRFLOW_CONN_APP_DB}
- name: AIRFLOW__CORE__FERNET_KEY
valueFrom:
secretKeyRef: {name: airflow, key: FERNET_KEY}
@@ -91,6 +124,9 @@ spec:
secretKeyRef: {name: airflow, key: ADMIN_PASSWORD}
ports:
- containerPort: 8080
volumeMounts:
- name: dags
mountPath: /opt/airflow/dags
readinessProbe:
httpGet:
path: /health
@@ -99,6 +135,11 @@ spec:
initialDelaySeconds: 60
periodSeconds: 15
failureThreshold: 20
volumes:
- name: dags
configMap:
name: airflow-dags
optional: true
YAML
echo " waiting for airflow (the first boot migrates the database, so this is slow)..."

View File

@@ -1,8 +1,8 @@
#!/usr/bin/env bash
# PostgreSQL — the cluster half of the postgres cabinet: plain manifests, one replica on a PVC.
# Notes: docs/notes/addons.md
# PostgreSQL for this overlay: plain manifests, one replica on a PVC, password generated once and kept.
# Notes: ../README.md
set -euo pipefail
cd "$(dirname "$0")/.."
cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
source ./lib/config.sh
load_config

View File

@@ -1,8 +1,8 @@
#!/usr/bin/env bash
# Redis — the cluster half of the redis cabinet: cache/broker, no persistence.
# Notes: docs/notes/addons.md
# Redis for this overlay: cache and broker (Celery), no persistence.
# Notes: ../README.md
set -euo pipefail
cd "$(dirname "$0")/.."
cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
source ./lib/config.sh
load_config

View File

@@ -0,0 +1,81 @@
"""Pull items from the items API, rename them into the app's names, upsert into postgres.
Three links, kept apart on purpose:
- the API client (`fetch_items`) talks to the wire as it is — here the overlay's
own simulator, `items-api`, whose field names are the API's;
- the adapter (`to_app_row`) is the one place the wire's names become the app's:
`id` -> `item_id`, `name` -> `item_name`, `price.amount_cents` -> `price_cents`.
It belongs to whoever owns the app's model, so it lives in the overlay, not in rig;
- the load writes through the `app_db` connection (AIRFLOW_CONN_APP_DB, built by
addons/airflow.sh from the postgres secret) and is idempotent: an upsert keyed
on `item_id`, so a retry or a rerun never duplicates a row.
Operational logic is explicit and minimal: hourly, no backfill, one retry.
"""
import json
import urllib.request
from datetime import datetime, timedelta
from airflow import DAG
from airflow.operators.python import PythonOperator
ITEMS_URL = "http://items-api/v1/items"
CREATE = """
CREATE TABLE IF NOT EXISTS items (
item_id text PRIMARY KEY,
item_name text NOT NULL,
price_cents integer NOT NULL,
currency text NOT NULL,
loaded_at timestamptz NOT NULL DEFAULT now()
)
"""
UPSERT = """
INSERT INTO items (item_id, item_name, price_cents, currency)
VALUES (%(item_id)s, %(item_name)s, %(price_cents)s, %(currency)s)
ON CONFLICT (item_id) DO UPDATE
SET item_name = EXCLUDED.item_name,
price_cents = EXCLUDED.price_cents,
currency = EXCLUDED.currency,
loaded_at = now()
"""
def fetch_items():
"""The API client: the wire, as the API returns it."""
with urllib.request.urlopen(ITEMS_URL, timeout=10) as response:
return json.load(response)["items"]
def to_app_row(item):
"""The adapter: the API's names in, the app's names out."""
return {
"item_id": item["id"],
"item_name": item["name"],
"price_cents": item["price"]["amount_cents"],
"currency": item["price"]["currency"],
}
def load_items():
from airflow.providers.postgres.hooks.postgres import PostgresHook
rows = [to_app_row(item) for item in fetch_items()]
hook = PostgresHook(postgres_conn_id="app_db")
hook.run(CREATE)
for row in rows:
hook.run(UPSERT, parameters=row)
print(f"upserted {len(rows)} items")
with DAG(
dag_id="items_to_postgres",
schedule="@hourly",
start_date=datetime(2026, 1, 1),
catchup=False,
default_args={"retries": 1, "retry_delay": timedelta(minutes=1)},
tags=["example"],
) as dag:
PythonOperator(task_id="load_items", python_callable=load_items)

View File

@@ -0,0 +1,95 @@
# The simulator: a stub of the API the DAG reads, faithful to the wire (its field
# names are the API's, not the app's). Same shape as the starter's example-mock.
apiVersion: v1
kind: ConfigMap
metadata:
name: items-api-stub
data:
routes.json: |
{
"/health": {"status": 200, "body": {"status": "ok"}},
"/v1/items": {"status": 200, "body": {"items": [
{"id": "a-100", "name": "anvil", "price": {"amount_cents": 1999, "currency": "USD"}},
{"id": "b-200", "name": "bucket", "price": {"amount_cents": 450, "currency": "USD"}},
{"id": "c-300", "name": "crate", "price": {"amount_cents": 1200, "currency": "USD"}}
]}}
}
serve.py: |
import json, os
from http.server import BaseHTTPRequestHandler, HTTPServer
ROUTES = json.load(open("/etc/stub/routes.json"))
NAME = os.environ.get("STUB_NAME", "stub")
class H(BaseHTTPRequestHandler):
def do_GET(self):
r = ROUTES.get(self.path)
if r is None:
self.send_response(404)
self.end_headers()
self.wfile.write(json.dumps(
{"error": "no canned route", "stub": NAME, "path": self.path}
).encode())
return
body = json.dumps(r["body"]).encode()
self.send_response(r["status"])
self.send_header("Content-Type", "application/json")
self.send_header("X-Mocked-By", NAME)
self.end_headers()
self.wfile.write(body)
def log_message(self, fmt, *args):
print("%s %s" % (NAME, fmt % args), flush=True)
HTTPServer(("0.0.0.0", 8080), H).serve_forever()
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: items-api
labels:
app: items-api
rig.component/impl: mock
spec:
replicas: 1
selector:
matchLabels:
app: items-api
template:
metadata:
labels:
app: items-api
spec:
containers:
- name: stub
image: python:3.12-slim
command: ["python3", "/etc/stub/serve.py"]
env:
- name: STUB_NAME
value: items-api
ports:
- containerPort: 8080
volumeMounts:
- name: stub
mountPath: /etc/stub
readinessProbe:
httpGet: { path: /health, port: 8080 }
initialDelaySeconds: 2
resources:
requests: { memory: 32Mi, cpu: 10m }
limits: { memory: 64Mi }
volumes:
- name: stub
configMap:
name: items-api-stub
---
apiVersion: v1
kind: Service
metadata:
name: items-api
spec:
selector:
app: items-api
ports:
- port: 80
targetPort: 8080

View File

@@ -0,0 +1,10 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
# Beside the addons, in DATA_NAMESPACE — but no Namespace object: the addons created
# it, and a Namespace Tilt owned would be deleted by `tilt down`, taking postgres and
# airflow with it. rig's Tiltfile creates namespaces that are used and not declared.
namespace: data
resources:
- items-api.yaml

View File

@@ -0,0 +1,5 @@
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base

23
rig/examples/data/rig.env Normal file
View File

@@ -0,0 +1,23 @@
# This overlay's settings: layered over rig's defaults, under ctrl/.env and the caller.
# data — postgres, redis and airflow (upstream images, run unmodified) in their own namespace.
# Use it: OVERLAY=examples/data make cluster up Notes: README.md
# Order matters: addons install in the order listed, and airflow refuses to start
# without postgres, so postgres comes first. metallb is rig's own. Airflow runs
# LocalExecutor and needs no broker: add redis (before airflow) only to switch to Celery.
ADDONS="metallb postgres airflow"
# Namespace for the dependency containers (k8s/base/kustomization.yaml names it too).
DATA_NAMESPACE=data
# Postgres identity. The password is generated once by addons/postgres.sh and kept.
POSTGRES_DB=app
POSTGRES_USER=app
POSTGRES_STORAGE=2Gi
AIRFLOW_ADMIN_USER=admin
# Upstream images, pinned by tag; bump freely, and preload them for an offline machine.
POSTGRES_IMAGE=postgres:16-alpine
REDIS_IMAGE=redis:7-alpine
AIRFLOW_IMAGE=apache/airflow:2.10.4

View File

@@ -1,7 +1,7 @@
# EXAMPLE component image. Copy, rename, replace:
# ctrl/Dockerfile.api -> image <cluster>-api -> image: in k8s/base/api.yaml
# COPY paths are relative to the REPO ROOT (Tilt context='..'), not this directory.
# Notes: docs/notes/Dockerfile.example.md
# Dockerfile.api -> image <cluster>-api -> image: in k8s/base/api.yaml
# COPY paths are relative to the build context the overlay's Tiltfile names (context='.').
# Notes: rig's docs/notes/Dockerfile.example.md
FROM python:3.12-slim
@@ -11,7 +11,7 @@ WORKDIR /app
COPY api/requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
# Repo-root relative — see above.
# Context-relative — see above.
COPY api/ ./api/
# Match this with the containerPort in the manifest and the target of the
@@ -20,5 +20,5 @@ EXPOSE 8000
CMD ["python", "-m", "api"]
# live_update: Tiltfile's sync('../api', '/app/api') must match COPY api/ + WORKDIR /app,
# live_update: the Tiltfile's sync('api', '/app/api') must match COPY api/ + WORKDIR /app,
# or edits silently do nothing.

View File

@@ -0,0 +1,59 @@
# `examples/starter` — the overlay rig runs when you name none
An overlay is one folder, outside rig, that holds what runs: its settings, its
manifests, its images. rig reads it and never writes into it — see
[`docs/notes/overlay.md`](../../docs/notes/overlay.md) for the contract. This one
ships with rig so `make tilt` has something to deploy on a fresh clone, and so a
real overlay has a shape to be written against:
```
rig.env settings layered over rig's defaults (this one sets none)
k8s/base/ the components, as plain manifests
k8s/overlays/dev/ how this environment differs from the base — MANIFESTS_DIR's default
Tiltfile the workload's half of the dev loop; rig's ctrl/Tiltfile includes it
Dockerfile.example the shape of an image you build yourself
```
To start your own, copy this folder somewhere rig does not track —
`rig/local/<name>/`, or a repo of its own — and name it:
```bash
cp -r examples/starter local/myenv
OVERLAY=local/myenv make cluster up # or OVERLAY=local/myenv in ctrl/.env
```
The cluster, context and port block then follow the overlay's folder name
(`myenv`, `kind-myenv`), so it never collides with another.
## The two components are examples, not the system
They exist so the real manifests have a shape to be written against.
### The three states a component can be in
Switching between them should be a one-line change, never a rewrite. The DNS
name stays the same in every case, so callers never know the difference:
| state | what exists | when |
| --- | --- | --- |
| **real** | an image built from source, hot-reloaded | the one thing you are working on |
| **mock** | a stub returning canned responses (`example-mock.yaml`) | everything else — most of the estate |
| **remote** | no pod at all, just a Service (`example-remote.yaml`) | when the real system is reachable and you want it |
Most components should be **mock**. What has to be faithful is the topology —
names, ports, dependency order, who can reach whom, how it fails. The workloads
are noise, and mocking them is what makes several copies of a large estate fit
on one laptop.
## Images
`Dockerfile.example` is a commented shape, not a working build. Paths in this
folder's Tiltfile are relative to this folder, so `context='.'` is the overlay
and every `COPY` in the Dockerfile is relative to it:
```
docker_build(CLUSTER + '-api', context='.', dockerfile='Dockerfile.api')
```
The name given to `docker_build` must match `image:` in the manifest — that
string is the only thing connecting the two.

View File

@@ -0,0 +1,69 @@
# The starter overlay's half of the dev loop. rig's ctrl/Tiltfile includes this once
# it has applied k8s/overlays/dev, so every path here is relative to THIS folder.
# Facts from rig, via os.getenv: RIG_CLUSTER RIG_CONTEXT RIG_HTTP_PORT RIG_HTTPS_PORT
# RIG_TILT_PORT RIG_REGISTRY RIG_OVERLAY_DIR
# Notes: rig's docs/notes/overlay.md
CLUSTER = os.getenv('RIG_CLUSTER')
HTTP = os.getenv('RIG_HTTP_PORT')
# ── Images ─────────────────────────────────────────────────────────────────
# (nothing yet — the examples run upstream images. Add docker_build calls here.)
# ── Resources ──────────────────────────────────────────────────────────────
# (nothing yet — add k8s_resource calls here to name and order what you deploy.)
# ═══════════════════════════════════════════════════════════════════════════
# Catalogue — paste what you need, delete the rest.
# Commented out so this file runs as-is.
# ═══════════════════════════════════════════════════════════════════════════
#
# ── build an image ─────────────────────────────────────────────────────────
# context= and dockerfile= are both relative to this folder, so every COPY in the
# Dockerfile is relative to the context you name here.
#
# docker_build(
# CLUSTER + '-api', # must match `image:` in the manifest —
# context='.', # that string is the only thing
# dockerfile='Dockerfile.api', # connecting the two
# ignore=['.git', 'rig', '.venv', 'node_modules', '__pycache__'],
# live_update=[sync('api', '/app/api')],
# )
#
# ── a shared base, built once ──────────────────────────────────────────────
# Components that share code build FROM one base image instead of each carrying
# a copy of it. Tilt builds the base first when a Dockerfile's FROM names it.
#
# docker_build(CLUSTER + '-base', context='repodir/base')
# docker_build(CLUSTER + '-api', context='repodir/api') # its Dockerfile: FROM <cluster>-base
#
# ── name and order a resource ──────────────────────────────────────────────
# k8s_resource('api', resource_deps=['postgres'], labels=['app'])
# k8s_resource('gateway', resource_deps=['api', 'ui'], labels=['app'])
#
# ── reload the gateway when its config changes ─────────────────────────────
# A hash-less configMapGenerator ConfigMap never changes name, so edits do NOT
# roll the pod on their own.
#
# local_resource(
# 'gateway-reload',
# cmd='kubectl --context %s -n <namespace> rollout restart deployment/gateway' % os.getenv('RIG_CONTEXT'),
# deps=['k8s/base/Caddyfile'],
# resource_deps=['gateway'],
# auto_init=False,
# )
#
# ── manifests that need kustomize flags ────────────────────────────────────
# rig applies MANIFESTS_DIR without flags. If a secretGenerator reads above its
# kustomization root, set MANIFESTS_DIR=none in rig.env and apply them here instead;
# the flag loosens a safety check for the whole build.
#
# k8s_yaml(kustomize('k8s/overlays/dev', flags=['--load-restrictor=LoadRestrictionsNone']))
#
# ── reach a service directly, bypassing the gateway ────────────────────────
# Prefer the gateway; host ports are shared machine-wide. If you need one, take it
# from this environment's own port block.
#
# k8s_resource('postgres', port_forwards=[str(int(HTTP) + 5) + ':5432'])

View File

@@ -0,0 +1,9 @@
# This overlay's settings: layered over rig's defaults, under ctrl/.env and the caller.
# The starter sets nothing, so a rig with no overlay named behaves exactly like one with
# no overlay at all. Any key a profile could set belongs here; paths are relative to
# this folder. PROFILE and OVERLAY are refused — they are what chooses this file.
# Notes: rig's docs/notes/overlay.md
#
# ADDONS="metallb"
# K8S_VERSION=v1_36
# MANIFESTS_DIR=k8s/overlays/dev

View File

@@ -16,7 +16,7 @@ endif
help: ## list targets
@grep -hE '^[a-z][a-z-]*:.*?##' $(MAKEFILE_LIST) | sed 's/:.*##/\t/' | expand -t16
deps: ## rigdeps.sh [detect|list|verify|fetch|install] (default detect)
deps: ## rigdeps.sh [detect|list|verify|fetch|install|manifest|manifests] (default detect)
bash $(HERE)rigdeps.sh $(or $(ARGS),detect)
mem: ## rigmini.sh [status|push|all|backup|restore] (default status)

View File

@@ -8,7 +8,12 @@
# `make selftest` fails while this file differs from what rig generates.
# ── from the libraries ──
declare -- CONFIG_OVERRIDABLE=$'PROFILE CLUSTER K8S_VERSION KIND_CONFIG ADDONS\n REGISTRY_MODE INGRESS_MODE DNS_MODE TILT_PORT\n SOURCE ARCH DEPS_SOURCE HTTP_PORT HTTPS_PORT\n REGISTRY_PORT MANIFESTS_DIR'
declare -- CONFIG_OVERRIDABLE=$'PROFILE OVERLAY CLUSTER K8S_VERSION KIND_CONFIG ADDONS\n REGISTRY_MODE INGRESS_MODE DNS_MODE TILT_PORT\n SOURCE ARCH DEPS_SOURCE HTTP_PORT HTTPS_PORT\n REGISTRY_PORT MANIFESTS_DIR'
declare -- DEFAULT_OVERLAY="examples/starter"
_abs_from_ctrl ()
{
( cd "$(_from_ctrl "$1")" 2> /dev/null && pwd )
}
_config_restore ()
{
local line;
@@ -19,10 +24,25 @@ _config_restore ()
done <<< "$1";
return 0
}
_from_ctrl ()
{
case "$1" in
/*)
echo "$1"
;;
*)
echo "../$1"
;;
esac
}
default_cluster_name ()
{
local n;
n=$(basename "$(cd .. && pwd)");
if [ -n "${OVERLAY:-}" ]; then
n=$(basename "$(_abs_from_ctrl "$OVERLAY_DIR")");
else
n=$(basename "$(cd .. && pwd)");
fi;
n=$(echo "$n" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9-' '-');
n=$(echo "$n" | sed 's/^-*//; s/-*$//');
echo "${n:-rig}"
@@ -35,8 +55,11 @@ derive_port_base ()
}
render_kind_config ()
{
local host_workdir="${HOST_WORKDIR:-$(cd .. && pwd)}";
sed -e "s|\${CLUSTER}|${CLUSTER}|g" -e "s|\${NODE_IMAGE}|${NODE_IMAGE}|g" -e "s|\${HTTP_PORT}|${HTTP_PORT}|g" -e "s|\${HOST_WORKDIR}|${host_workdir}|g" "$KIND_CONFIG"
local host_workdir="${HOST_WORKDIR:-$(cd .. && pwd)}" overlay_dir="";
if [ -n "$OVERLAY_DIR" ]; then
overlay_dir=$(_abs_from_ctrl "$OVERLAY_DIR");
fi;
sed -e "s|\${CLUSTER}|${CLUSTER}|g" -e "s|\${NODE_IMAGE}|${NODE_IMAGE}|g" -e "s|\${HTTP_PORT}|${HTTP_PORT}|g" -e "s|\${HOST_WORKDIR}|${host_workdir}|g" -e "s|\${OVERLAY_DIR}|${overlay_dir}|g" "$KIND_CONFIG"
}
# ── configuration, frozen for profile 'default' ──
@@ -46,7 +69,8 @@ load_config() {
if [ -n "${!k+x}" ]; then saved+="$k=$(printf '%q' "${!k}")"$'\n'; fi
done
declare -g ADDONS=""
declare -gx AIRFLOW_IMAGE="apache/airflow:2.10.4"
declare -gx CERT_MANAGER_MANIFEST_SHA256="5f6a499b8c1857d57f560f536e0dcc830914b45c420899fe7ad0692c8624e408"
declare -gx CERT_MANAGER_MANIFEST_URL="https://github.com/cert-manager/cert-manager/releases/download/v1.21.1/cert-manager.yaml"
declare -gx CERT_MANAGER_VERSION="v1.21.1"
declare -g CLUSTER="rig"
declare -gx COMPOSE_SHA256="db1889184726840f75c4f9c001048430d4f25b3be3cb084d3ddd762bc0aed576"
@@ -71,8 +95,13 @@ load_config() {
declare -gx KUBECTL_SHA256="ebbd080e7c2e275093b55915722043257eb24004363e20acb3c4d71919f88336"
declare -gx KUBECTL_URL="https://dl.k8s.io/release/v1.36.3/bin/linux/amd64/kubectl"
declare -gx KUBECTL_VERSION="v1.36.3"
declare -g MANIFESTS_DIR="ctrl/k8s/overlays/dev"
declare -g MANIFESTS_DIR="examples/starter/k8s/overlays/dev"
declare -gx METALLB_MANIFEST_GIT_BLOB="7fbda334cc3ac0aaabdcb081af4f543feb3c2f9f"
declare -gx METALLB_MANIFEST_SHA256="b0b9be2802f10aa32d45308b4457d06cde0c70544712c8d0cf5511657ffd2b69"
declare -gx METALLB_MANIFEST_URL="https://raw.githubusercontent.com/metallb/metallb/v0.16.0/config/manifests/metallb-native.yaml"
declare -gx METALLB_VERSION="v0.16.0"
declare -gx METRICS_SERVER_MANIFEST_SHA256="1cec29a5267809306a2c6ec74a3e449abbb705b4a8beed0c8a1963910f72c79b"
declare -gx METRICS_SERVER_MANIFEST_URL="https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.9.0/components.yaml"
declare -gx METRICS_SERVER_VERSION="v0.9.0"
declare -g NODES="1"
declare -g NODE_IMAGE="kindest/node:v1.36.1@sha256:3489c7674813ba5d8b1a9977baea8a6e553784dab7b84759d1014dbd78f7ebd5"
@@ -81,14 +110,12 @@ load_config() {
declare -gx NODE_IMAGE_v1_35="kindest/node:v1.35.5@sha256:ce977ae6d65918d0b58a5f8b5e940429c2ce42fa3a5619ec2bbc60b949c0ac95"
declare -gx NODE_IMAGE_v1_36="kindest/node:v1.36.1@sha256:3489c7674813ba5d8b1a9977baea8a6e553784dab7b84759d1014dbd78f7ebd5"
declare -g NODE_MB="800"
declare -gx POSTGRES_IMAGE="postgres:16-alpine"
declare -g OVERLAY_DIR="examples/starter"
declare -gx PROFILE="default"
declare -g PROFILE_NAME="default"
declare -gx REDIS_IMAGE="redis:7-alpine"
declare -gx REGISTRY_IMAGE="registry:2"
declare -g REGISTRY_MODE="local"
declare -g REGISTRY_PORT="20313"
declare -gx STUB_IMAGE="python:3.12-slim"
declare -g TILT_PORT="20312"
declare -gx TILT_SHA256="e9672b8a18d43501f35dcfe98465969a7db0e436b36cf0c50c7e6f8d40de5fe6"
declare -gx TILT_URL="https://github.com/tilt-dev/tilt/releases/download/v0.37.6/tilt.0.37.6.linux.x86_64.tar.gz"
@@ -100,7 +127,8 @@ load_config() {
# ── ctrl/deps.sh ──
# Toolchain installer: detect the host, install pinned tools into $OUT_BIN, report
# host actions it will not perform (no sudo, no apt). Usually via `make deps`.
# Usage: deps.sh [detect [all] | list | verify [core|dev] | fetch [core|dev] [--to DIR] | install [core|dev]]
# Usage: deps.sh [detect [all] | list | verify [core|dev] | fetch [core|dev] [--to DIR] | install [core|dev]
# | manifest NAME | manifests [--to DIR]]
# Notes: docs/notes/deps.md
set -euo pipefail
@@ -817,6 +845,51 @@ require_linux
cmd="${1:-install}"
[ $# -gt 0 ] && shift
# ── manifests rig's own addons apply ───────────────────────────────────────
# Pinned (URL + SHA256), fetched through the same DEPS_SOURCE resolver as the
# binaries and verified, then applied from disk: an offline machine needs no
# network for them. Default home: vendor/manifests/ in rig's folder (gitignored).
MANIFESTS_HOME="${MANIFESTS_HOME:-$(cd .. && pwd)/vendor/manifests}"
BAKED_MANIFESTS="${BAKED_MANIFESTS:-/opt/rig/manifests}"
MANIFEST_NAMES="METALLB CERT_MANAGER METRICS_SERVER"
manifest_path() { # NAME dir
local v="${1}_VERSION"
echo "$2/$(echo "$1" | tr 'A-Z_' 'a-z-')-${!v}.yaml"
}
# Make one pinned manifest present and verified in dir; print only its path.
fetch_manifest() { # NAME dir
local name="$1" dir="$2" url_var="${1}_MANIFEST_URL" sha_var="${1}_MANIFEST_SHA256" file
if [ -z "${!url_var:-}" ] || [ -z "${!sha_var:-}" ]; then
echo "no pinned manifest for $name (${url_var} / ${sha_var} unset)" >&2
exit 1
fi
file=$(manifest_path "$name" "$dir")
if [ -f "$file" ] && [ "$($SHA "$file" | awk '{print $1}')" = "${!sha_var}" ]; then
echo "$file"
return
fi
mkdir -p "$dir"
if [ "$DEPS_SOURCE" = baked ]; then
cp "$(manifest_path "$name" "$BAKED_MANIFESTS")" "$file.tmp"
else
download "$(resolve_url "${!url_var}")" "$file.tmp"
fi
verify "$file.tmp" "${!sha_var}" "$name manifest"
mv "$file.tmp" "$file"
echo "$file"
}
fetch_manifests() { # [--to DIR]
local dest="$MANIFESTS_HOME" n
if [ "${1:-}" = --to ]; then dest="$(abspath "${2:?--to needs a directory}")"; fi
echo "fetching the addons' manifests into $dest (source: $DEPS_SOURCE)"
for n in $MANIFEST_NAMES; do
echo " $n $(fetch_manifest "$n" "$dest")"
done
}
# Baked mode copies binaries already in the image, so it needs no downloader.
need_downloads() {
require_amd64
@@ -830,9 +903,13 @@ case "$cmd" in
verify) verify_tools "${1:-dev}" ;;
fetch) need_downloads; fetch "$@" ;;
install) need_downloads; install "${1:-dev}" ;;
*) echo "usage: $0 [detect [all]|list|verify|fetch|install]" >&2
manifest) need_downloads
fetch_manifest "${1:?usage: $0 manifest <METALLB|CERT_MANAGER|METRICS_SERVER>}" "$MANIFESTS_HOME" ;;
manifests) need_downloads; fetch_manifests "$@" ;;
*) echo "usage: $0 [detect [all]|list|verify|fetch|install|manifest NAME|manifests]" >&2
echo " install [core|dev] (default dev)" >&2
echo " fetch [core|dev] [--to DIR]" >&2
echo " manifests [--to DIR] the addons' pinned manifests, verified" >&2
echo " OUT_BIN=<dir> overrides the install directory" >&2
exit 1 ;;
esac

View File

@@ -8,7 +8,12 @@
# `make selftest` fails while this file differs from what rig generates.
# ── from the libraries ──
declare -- CONFIG_OVERRIDABLE=$'PROFILE CLUSTER K8S_VERSION KIND_CONFIG ADDONS\n REGISTRY_MODE INGRESS_MODE DNS_MODE TILT_PORT\n SOURCE ARCH DEPS_SOURCE HTTP_PORT HTTPS_PORT\n REGISTRY_PORT MANIFESTS_DIR'
declare -- CONFIG_OVERRIDABLE=$'PROFILE OVERLAY CLUSTER K8S_VERSION KIND_CONFIG ADDONS\n REGISTRY_MODE INGRESS_MODE DNS_MODE TILT_PORT\n SOURCE ARCH DEPS_SOURCE HTTP_PORT HTTPS_PORT\n REGISTRY_PORT MANIFESTS_DIR'
declare -- DEFAULT_OVERLAY="examples/starter"
_abs_from_ctrl ()
{
( cd "$(_from_ctrl "$1")" 2> /dev/null && pwd )
}
_config_restore ()
{
local line;
@@ -19,10 +24,25 @@ _config_restore ()
done <<< "$1";
return 0
}
_from_ctrl ()
{
case "$1" in
/*)
echo "$1"
;;
*)
echo "../$1"
;;
esac
}
default_cluster_name ()
{
local n;
n=$(basename "$(cd .. && pwd)");
if [ -n "${OVERLAY:-}" ]; then
n=$(basename "$(_abs_from_ctrl "$OVERLAY_DIR")");
else
n=$(basename "$(cd .. && pwd)");
fi;
n=$(echo "$n" | tr '[:upper:]' '[:lower:]' | tr -c 'a-z0-9-' '-');
n=$(echo "$n" | sed 's/^-*//; s/-*$//');
echo "${n:-rig}"
@@ -35,8 +55,11 @@ derive_port_base ()
}
render_kind_config ()
{
local host_workdir="${HOST_WORKDIR:-$(cd .. && pwd)}";
sed -e "s|\${CLUSTER}|${CLUSTER}|g" -e "s|\${NODE_IMAGE}|${NODE_IMAGE}|g" -e "s|\${HTTP_PORT}|${HTTP_PORT}|g" -e "s|\${HOST_WORKDIR}|${host_workdir}|g" "$KIND_CONFIG"
local host_workdir="${HOST_WORKDIR:-$(cd .. && pwd)}" overlay_dir="";
if [ -n "$OVERLAY_DIR" ]; then
overlay_dir=$(_abs_from_ctrl "$OVERLAY_DIR");
fi;
sed -e "s|\${CLUSTER}|${CLUSTER}|g" -e "s|\${NODE_IMAGE}|${NODE_IMAGE}|g" -e "s|\${HTTP_PORT}|${HTTP_PORT}|g" -e "s|\${HOST_WORKDIR}|${host_workdir}|g" -e "s|\${OVERLAY_DIR}|${overlay_dir}|g" "$KIND_CONFIG"
}
# ── configuration, frozen for profile 'default' ──
@@ -46,7 +69,8 @@ load_config() {
if [ -n "${!k+x}" ]; then saved+="$k=$(printf '%q' "${!k}")"$'\n'; fi
done
declare -g ADDONS=""
declare -gx AIRFLOW_IMAGE="apache/airflow:2.10.4"
declare -gx CERT_MANAGER_MANIFEST_SHA256="5f6a499b8c1857d57f560f536e0dcc830914b45c420899fe7ad0692c8624e408"
declare -gx CERT_MANAGER_MANIFEST_URL="https://github.com/cert-manager/cert-manager/releases/download/v1.21.1/cert-manager.yaml"
declare -gx CERT_MANAGER_VERSION="v1.21.1"
declare -g CLUSTER="rig"
declare -gx COMPOSE_SHA256="db1889184726840f75c4f9c001048430d4f25b3be3cb084d3ddd762bc0aed576"
@@ -71,8 +95,13 @@ load_config() {
declare -gx KUBECTL_SHA256="ebbd080e7c2e275093b55915722043257eb24004363e20acb3c4d71919f88336"
declare -gx KUBECTL_URL="https://dl.k8s.io/release/v1.36.3/bin/linux/amd64/kubectl"
declare -gx KUBECTL_VERSION="v1.36.3"
declare -g MANIFESTS_DIR="ctrl/k8s/overlays/dev"
declare -g MANIFESTS_DIR="examples/starter/k8s/overlays/dev"
declare -gx METALLB_MANIFEST_GIT_BLOB="7fbda334cc3ac0aaabdcb081af4f543feb3c2f9f"
declare -gx METALLB_MANIFEST_SHA256="b0b9be2802f10aa32d45308b4457d06cde0c70544712c8d0cf5511657ffd2b69"
declare -gx METALLB_MANIFEST_URL="https://raw.githubusercontent.com/metallb/metallb/v0.16.0/config/manifests/metallb-native.yaml"
declare -gx METALLB_VERSION="v0.16.0"
declare -gx METRICS_SERVER_MANIFEST_SHA256="1cec29a5267809306a2c6ec74a3e449abbb705b4a8beed0c8a1963910f72c79b"
declare -gx METRICS_SERVER_MANIFEST_URL="https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.9.0/components.yaml"
declare -gx METRICS_SERVER_VERSION="v0.9.0"
declare -g NODES="1"
declare -g NODE_IMAGE="kindest/node:v1.36.1@sha256:3489c7674813ba5d8b1a9977baea8a6e553784dab7b84759d1014dbd78f7ebd5"
@@ -81,14 +110,12 @@ load_config() {
declare -gx NODE_IMAGE_v1_35="kindest/node:v1.35.5@sha256:ce977ae6d65918d0b58a5f8b5e940429c2ce42fa3a5619ec2bbc60b949c0ac95"
declare -gx NODE_IMAGE_v1_36="kindest/node:v1.36.1@sha256:3489c7674813ba5d8b1a9977baea8a6e553784dab7b84759d1014dbd78f7ebd5"
declare -g NODE_MB="800"
declare -gx POSTGRES_IMAGE="postgres:16-alpine"
declare -g OVERLAY_DIR="examples/starter"
declare -gx PROFILE="default"
declare -g PROFILE_NAME="default"
declare -gx REDIS_IMAGE="redis:7-alpine"
declare -gx REGISTRY_IMAGE="registry:2"
declare -g REGISTRY_MODE="local"
declare -g REGISTRY_PORT="20313"
declare -gx STUB_IMAGE="python:3.12-slim"
declare -g TILT_PORT="20312"
declare -gx TILT_SHA256="e9672b8a18d43501f35dcfe98465969a7db0e436b36cf0c50c7e6f8d40de5fe6"
declare -gx TILT_URL="https://github.com/tilt-dev/tilt/releases/download/v0.37.6/tilt.0.37.6.linux.x86_64.tar.gz"

View File

@@ -7,7 +7,7 @@
"rig_addon": "airflow",
"ports": [8080],
"volumes": ["airflow_logs"],
"depends_on": ["postgres", "redis"],
"depends_on": ["postgres"],
"env": {
"AIRFLOW_PORT": "8080",
"AIRFLOW_ADMIN_USER": "admin",
@@ -15,8 +15,8 @@
"AIRFLOW_DAGS_DIR": "./dags"
},
"notes": [
"Brings postgres and redis with it — Airflow needs a metadata database and a broker, and will not start without both.",
"LocalExecutor by default: one container, no separate worker. Switch to CeleryExecutor in service.yml when the room outgrows it.",
"Brings postgres with it — Airflow needs a metadata database and will not start without one. It gets its own `airflow` database on that instance, apart from the app's.",
"LocalExecutor by default: one container, no separate worker, no broker. redis is only needed to switch to CeleryExecutor in service.yml; add it then, not before.",
"8080 collides with almost everything. Set AIRFLOW_PORT in the room's .env."
]
}