rig major updates

rig/examples/data/README.md

@@ -0,0 +1,98 @@
+# `examples/data` — an overlay with a database, a scheduler and a worked pipeline
+
+postgres, redis and airflow, each an upstream image run unmodified, installed as
+this overlay's own addons. rig ships the mechanism that finds and runs them; which
+services a workload needs is the workload's business, so they live here rather
+than in rig's `ctrl/addons/`. Copy what you need into your own overlay's `addons/`.
+
+```bash
+OVERLAY=examples/data make cluster up     # cluster `data`: postgres, then airflow
+OVERLAY=examples/data make tilt           # the items-api simulator the DAG reads
+```
+
+```
+rig.env                    ADDONS (metallb is rig's; the rest are here), namespace, identities, image pins
+addons/postgres.sh         one replica on a PVC; the password generated once and kept
+addons/airflow.sh          one `standalone` pod on its own `airflow` database; needs postgres
+addons/redis.sh            only for switching airflow to CeleryExecutor (not in ADDONS)
+k8s/                       the items-api simulator
+dags/items_to_postgres.py  the worked example: API client → adapter → postgres
+Tiltfile                   names the simulator's resource
+```
+
+Everything lands in the `data` namespace (`DATA_NAMESPACE`, and the kustomization
+names it too), so resetting an app's namespace leaves the databases alone. Costs
+roughly 2 GB with airflow, under 1 without. Airflow's first boot runs the whole
+metadata migration, so expect a few minutes before it is ready.
+
+## The worked example: three links kept apart
+
+- **Metadata DB (infra).** `addons/airflow.sh` creates an `airflow` database on the
+  same postgres and points `SQL_ALCHEMY_CONN` there — airflow's own tables never land
+  in the app's database.
+- **DAG delivery.** The addon turns this overlay's `dags/` into the `airflow-dags`
+  ConfigMap, mounted at `/opt/airflow/dags`; re-run `make cluster up` after editing a
+  DAG. The faster path later: a kind `extraMount` of `dags/` plus a Tilt `sync` — noted,
+  not built.
+- **Data connection (operational logic).** `AIRFLOW_CONN_APP_DB`, composed each run
+  from the postgres secret, gives DAGs the app's database as the `app_db` connection.
+  One password reaches both URLs, with nowhere to drift.
+
+The DAG itself calls the simulator with its own HTTP client (the wire, as the API
+returns it), renames the wire's fields into the app's names in `to_app_row` — **the
+adapter, which belongs to whoever owns the app's model and so lives in the overlay** —
+and upserts into `items`. Hourly, no backfill, one retry, idempotent on `item_id`.
+
+```bash
+kubectl --context kind-data -n data exec deploy/airflow -- airflow dags unpause items_to_postgres
+kubectl --context kind-data -n data exec deploy/airflow -- airflow dags trigger items_to_postgres
+kubectl --context kind-data -n data exec deploy/postgres -- psql -U app -d app -c 'select * from items'
+```
+
+## Addons in an overlay
+
+Each one runs from rig's `ctrl/` (rig's `addons.sh` exports `RIG_CTRL`), so it
+starts with `cd "${RIG_CTRL:?...}"`, sources `./lib/config.sh` and calls
+`load_config` — and sees every key this overlay's `rig.env` sets. Run them through
+rig (`bash ctrl/addons.sh install`, or `make cluster up`), not directly.
+
+## postgres — plain manifests, one replica
+
+Plain manifests rather than a helm chart: a chart repo is a network dependency,
+and an offline machine needs a path with none. The image is pinned in `rig.env`
+and can be preloaded into a local registry like every other image.
+
+One replica on a PVC. This models a dependency for local work, not a
+highly-available database, and pretending otherwise on a kind node would be a
+more elaborate lie rather than a more useful one.
+
+The password is not in `rig.env`: `addons/postgres.sh` generates one on first
+install and keeps it across re-runs, so re-running the addon never rotates the
+credential out from under whatever is already connected.
+
+## redis
+
+Cache, and the broker anything queue-shaped runs on. No persistence: a broker
+that loses its queue on restart is the honest local model, and a PVC here buys
+nothing but a volume to clean up.
+
+## airflow
+
+Airflow needs a metadata database before it will start at all, so the script
+refuses rather than rolls a pod that will CrashLoopBackOff while the real problem
+(postgres missing from `ADDONS`) stays invisible in the logs.
+
+One pod on `standalone`: migration, admin user, scheduler and webserver in a
+single container, on LocalExecutor, which needs no broker. The official chart's
+five deployments model an installation; switching this on means wanting pipelines.
+redis is here for the day it moves to CeleryExecutor, and not before.
+
+## Reaching them
+
+Reach the databases with port-forward rather than binding more host ports:
+
+```bash
+kubectl --context kind-data -n data port-forward svc/postgres 5432:5432
+kubectl --context kind-data -n data port-forward svc/airflow 8080:8080
+kubectl --context kind-data -n data get secret postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d
+```

rig/examples/data/Tiltfile

@@ -0,0 +1,6 @@
+# The data overlay's half of the dev loop: the items-api simulator the example DAG reads.
+# rig's ctrl/Tiltfile has already applied k8s/overlays/dev; paths here are relative to
+# this folder. postgres and airflow are addons (make cluster up), not Tilt resources.
+# Notes: README.md
+
+k8s_resource('items-api', labels=['simulator'])

rig/examples/data/addons/airflow.sh

@@ -0,0 +1,150 @@
+#!/usr/bin/env bash
+# Apache Airflow for this overlay: one `standalone` pod (LocalExecutor — no broker).
+# Its own `airflow` database on the postgres addon; the app's data reaches DAGs as the
+# `app_db` connection; DAGs from this overlay's dags/, as a ConfigMap.
+# Requires the postgres addon; refuses to install without it.
+# Notes: ../README.md
+set -euo pipefail
+cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
+
+source ./lib/config.sh
+load_config
+
+K="kubectl --context ${KUBECONTEXT}"
+NS="${DATA_NAMESPACE:-data}"
+
+if ! $K get deployment -n "$NS" postgres >/dev/null 2>&1; then
+    echo "  ! airflow needs the postgres addon, and it is not installed" >&2
+    echo "    add it before airflow in the overlay's ADDONS:" >&2
+    echo "      ADDONS=\"... postgres airflow\"" >&2
+    exit 1
+fi
+
+# ── metadata DB: airflow's own tables, kept out of the app's database ──────
+# Same postgres instance, separate database, created once (idempotent).
+db_user=$($K get secret -n "$NS" postgres -o jsonpath='{.data.POSTGRES_USER}' | base64 -d)
+db_pass=$($K get secret -n "$NS" postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d)
+db_name=$($K get secret -n "$NS" postgres -o jsonpath='{.data.POSTGRES_DB}' | base64 -d)
+psql() { $K exec -n "$NS" deploy/postgres -- psql -U "$db_user" -d "$db_name" -tAc "$1"; }
+if [ "$(psql "SELECT 1 FROM pg_database WHERE datname = 'airflow'")" = 1 ]; then
+    echo "  database 'airflow' exists"
+else
+    psql "CREATE DATABASE airflow" >/dev/null
+    echo "  created database 'airflow' beside '${db_name}'"
+fi
+
+# ── what is generated once and kept: re-running never rotates these ────────
+if $K get secret -n "$NS" airflow >/dev/null 2>&1; then
+    echo "  secret exists, keeping the current admin password and fernet key"
+else
+    admin_password=$(head -c 18 /dev/urandom | base64 | tr -d '/+=' | head -c 24)
+    # Airflow requires a 32-byte urlsafe-base64 key; without a fixed one every
+    # restart invalidates every stored connection.
+    fernet_key=$(head -c 32 /dev/urandom | base64 | tr '+/' '-_')
+    $K create secret generic airflow -n "$NS" \
+        --from-literal=ADMIN_USER="${AIRFLOW_ADMIN_USER:-admin}" \
+        --from-literal=ADMIN_PASSWORD="$admin_password" \
+        --from-literal=FERNET_KEY="$fernet_key" \
+        >/dev/null
+    echo "  generated an admin password (read it back with the command below)"
+fi
+
+# ── connections: composed from what the postgres secret owns, every run ─────
+# Nothing to drift: one password reaches the metadata DB and the data connection.
+$K create secret generic airflow-connections -n "$NS" \
+    --from-literal=SQL_ALCHEMY_CONN="postgresql+psycopg2://${db_user}:${db_pass}@postgres:5432/airflow" \
+    --from-literal=AIRFLOW_CONN_APP_DB="postgres://${db_user}:${db_pass}@postgres:5432/${db_name}" \
+    --dry-run=client -o yaml | $K apply -f - >/dev/null
+
+# ── DAG delivery: this overlay's dags/ as a ConfigMap ───────────────────────
+# Edits land by re-running this addon (make cluster up). The later path — a kind
+# extraMount of dags/ plus a Tilt sync — is noted in the README, not built.
+# A ConfigMap volume is kubelet's ..data/..<timestamp> symlinks, and Airflow's DAG walker
+# follows symlinks: without the .airflowignore it stops at "Detected recursive loop".
+dags="$(_from_ctrl "$OVERLAY_DIR")/dags"
+if [ -d "$dags" ]; then
+    $K create configmap airflow-dags -n "$NS" --from-file="$dags" \
+        --from-literal=.airflowignore='^\.\.' \
+        --dry-run=client -o yaml | $K apply -f - >/dev/null
+    echo "  dags: $(ls "$dags" | grep -c '\.py$') file(s) from $(basename "$(_abs_from_ctrl "$OVERLAY_DIR")")/dags"
+fi
+
+echo "  applying manifests"
+$K apply -n "$NS" -f - >/dev/null <<YAML
+apiVersion: v1
+kind: Service
+metadata:
+  name: airflow
+spec:
+  selector:
+    app: airflow
+  ports:
+    - port: 8080
+      targetPort: 8080
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: airflow
+spec:
+  replicas: 1
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: airflow
+  template:
+    metadata:
+      labels:
+        app: airflow
+    spec:
+      containers:
+        - name: airflow
+          image: ${AIRFLOW_IMAGE}
+          args: ["standalone"]
+          env:
+            - name: AIRFLOW__CORE__EXECUTOR
+              value: LocalExecutor
+            - name: AIRFLOW__CORE__LOAD_EXAMPLES
+              value: "false"
+            - name: AIRFLOW__DATABASE__SQL_ALCHEMY_CONN
+              valueFrom:
+                secretKeyRef: {name: airflow-connections, key: SQL_ALCHEMY_CONN}
+            - name: AIRFLOW_CONN_APP_DB
+              valueFrom:
+                secretKeyRef: {name: airflow-connections, key: AIRFLOW_CONN_APP_DB}
+            - name: AIRFLOW__CORE__FERNET_KEY
+              valueFrom:
+                secretKeyRef: {name: airflow, key: FERNET_KEY}
+            - name: _AIRFLOW_WWW_USER_USERNAME
+              valueFrom:
+                secretKeyRef: {name: airflow, key: ADMIN_USER}
+            - name: _AIRFLOW_WWW_USER_PASSWORD
+              valueFrom:
+                secretKeyRef: {name: airflow, key: ADMIN_PASSWORD}
+          ports:
+            - containerPort: 8080
+          volumeMounts:
+            - name: dags
+              mountPath: /opt/airflow/dags
+          readinessProbe:
+            httpGet:
+              path: /health
+              port: 8080
+            # First boot runs the whole migration before it serves anything.
+            initialDelaySeconds: 60
+            periodSeconds: 15
+            failureThreshold: 20
+      volumes:
+        - name: dags
+          configMap:
+            name: airflow-dags
+            optional: true
+YAML
+
+echo "  waiting for airflow (the first boot migrates the database, so this is slow)..."
+$K rollout status deployment/airflow -n "$NS" --timeout=600s
+
+echo "  in-cluster: http://airflow.${NS}.svc.cluster.local:8080"
+echo "  reach it:   kubectl --context ${KUBECONTEXT} -n ${NS} port-forward svc/airflow 8080:8080"
+echo "  password:   kubectl --context ${KUBECONTEXT} -n ${NS} get secret airflow -o jsonpath='{.data.ADMIN_PASSWORD}' | base64 -d"

rig/examples/data/addons/postgres.sh

@@ -0,0 +1,105 @@
+#!/usr/bin/env bash
+# PostgreSQL for this overlay: plain manifests, one replica on a PVC, password generated once and kept.
+# Notes: ../README.md
+set -euo pipefail
+cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
+
+source ./lib/config.sh
+load_config
+
+K="kubectl --context ${KUBECONTEXT}"
+NS="${DATA_NAMESPACE:-data}"
+
+$K get namespace "$NS" >/dev/null 2>&1 || $K create namespace "$NS"
+
+# The password is generated once and then left alone, so re-running this does
+# not rotate the credential out from under whatever is already connected.
+if $K get secret -n "$NS" postgres >/dev/null 2>&1; then
+    echo "  secret exists, keeping the current password"
+else
+    password=$(head -c 18 /dev/urandom | base64 | tr -d '/+=' | head -c 24)
+    $K create secret generic postgres -n "$NS" \
+        --from-literal=POSTGRES_DB="${POSTGRES_DB:-postgres}" \
+        --from-literal=POSTGRES_USER="${POSTGRES_USER:-postgres}" \
+        --from-literal=POSTGRES_PASSWORD="$password" >/dev/null
+    echo "  generated a password (read it back with the command printed below)"
+fi
+
+echo "  applying manifests"
+$K apply -n "$NS" -f - >/dev/null <<YAML
+apiVersion: v1
+kind: PersistentVolumeClaim
+metadata:
+  name: postgres-data
+spec:
+  accessModes: [ReadWriteOnce]
+  resources:
+    requests:
+      storage: ${POSTGRES_STORAGE:-2Gi}
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: postgres
+spec:
+  selector:
+    app: postgres
+  ports:
+    - port: 5432
+      targetPort: 5432
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: postgres
+spec:
+  replicas: 1
+  # One volume, one writer. Rolling would start a second pod against the same
+  # PVC before the first exits, and Postgres refuses to share a data directory.
+  strategy:
+    type: Recreate
+  selector:
+    matchLabels:
+      app: postgres
+  template:
+    metadata:
+      labels:
+        app: postgres
+    spec:
+      containers:
+        - name: postgres
+          image: ${POSTGRES_IMAGE}
+          envFrom:
+            - secretRef:
+                name: postgres
+          env:
+            # The image initialises into the volume root otherwise, and a
+            # lost+found from the PVC makes it refuse to initdb.
+            - name: PGDATA
+              value: /var/lib/postgresql/data/pgdata
+          ports:
+            - containerPort: 5432
+          volumeMounts:
+            - name: data
+              mountPath: /var/lib/postgresql/data
+          readinessProbe:
+            exec:
+              command: ["sh", "-c", "pg_isready -U \$POSTGRES_USER -d \$POSTGRES_DB"]
+            initialDelaySeconds: 5
+            periodSeconds: 5
+          livenessProbe:
+            exec:
+              command: ["sh", "-c", "pg_isready -U \$POSTGRES_USER -d \$POSTGRES_DB"]
+            initialDelaySeconds: 30
+            periodSeconds: 15
+      volumes:
+        - name: data
+          persistentVolumeClaim:
+            claimName: postgres-data
+YAML
+
+echo "  waiting for postgres..."
+$K rollout status deployment/postgres -n "$NS" --timeout=240s
+
+echo "  in-cluster: postgres.${NS}.svc.cluster.local:5432"
+echo "  password:   kubectl --context ${KUBECONTEXT} -n ${NS} get secret postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d"

rig/examples/data/addons/redis.sh

@@ -0,0 +1,57 @@
+#!/usr/bin/env bash
+# Redis for this overlay: cache and broker (Celery), no persistence.
+# Notes: ../README.md
+set -euo pipefail
+cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
+
+source ./lib/config.sh
+load_config
+
+K="kubectl --context ${KUBECONTEXT}"
+NS="${DATA_NAMESPACE:-data}"
+
+$K get namespace "$NS" >/dev/null 2>&1 || $K create namespace "$NS"
+
+echo "  applying manifests"
+$K apply -n "$NS" -f - >/dev/null <<YAML
+apiVersion: v1
+kind: Service
+metadata:
+  name: redis
+spec:
+  selector:
+    app: redis
+  ports:
+    - port: 6379
+      targetPort: 6379
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: redis
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: redis
+  template:
+    metadata:
+      labels:
+        app: redis
+    spec:
+      containers:
+        - name: redis
+          image: ${REDIS_IMAGE}
+          ports:
+            - containerPort: 6379
+          readinessProbe:
+            exec:
+              command: ["redis-cli", "ping"]
+            initialDelaySeconds: 3
+            periodSeconds: 5
+YAML
+
+echo "  waiting for redis..."
+$K rollout status deployment/redis -n "$NS" --timeout=180s
+
+echo "  in-cluster: redis://redis.${NS}.svc.cluster.local:6379/0"

rig/examples/data/dags/items_to_postgres.py

@@ -0,0 +1,81 @@
+"""Pull items from the items API, rename them into the app's names, upsert into postgres.
+
+Three links, kept apart on purpose:
+
+- the API client (`fetch_items`) talks to the wire as it is — here the overlay's
+  own simulator, `items-api`, whose field names are the API's;
+- the adapter (`to_app_row`) is the one place the wire's names become the app's:
+  `id` -> `item_id`, `name` -> `item_name`, `price.amount_cents` -> `price_cents`.
+  It belongs to whoever owns the app's model, so it lives in the overlay, not in rig;
+- the load writes through the `app_db` connection (AIRFLOW_CONN_APP_DB, built by
+  addons/airflow.sh from the postgres secret) and is idempotent: an upsert keyed
+  on `item_id`, so a retry or a rerun never duplicates a row.
+
+Operational logic is explicit and minimal: hourly, no backfill, one retry.
+"""
+import json
+import urllib.request
+from datetime import datetime, timedelta
+
+from airflow import DAG
+from airflow.operators.python import PythonOperator
+
+ITEMS_URL = "http://items-api/v1/items"
+
+CREATE = """
+CREATE TABLE IF NOT EXISTS items (
+    item_id     text PRIMARY KEY,
+    item_name   text NOT NULL,
+    price_cents integer NOT NULL,
+    currency    text NOT NULL,
+    loaded_at   timestamptz NOT NULL DEFAULT now()
+)
+"""
+
+UPSERT = """
+INSERT INTO items (item_id, item_name, price_cents, currency)
+VALUES (%(item_id)s, %(item_name)s, %(price_cents)s, %(currency)s)
+ON CONFLICT (item_id) DO UPDATE
+   SET item_name = EXCLUDED.item_name,
+       price_cents = EXCLUDED.price_cents,
+       currency = EXCLUDED.currency,
+       loaded_at = now()
+"""
+
+
+def fetch_items():
+    """The API client: the wire, as the API returns it."""
+    with urllib.request.urlopen(ITEMS_URL, timeout=10) as response:
+        return json.load(response)["items"]
+
+
+def to_app_row(item):
+    """The adapter: the API's names in, the app's names out."""
+    return {
+        "item_id": item["id"],
+        "item_name": item["name"],
+        "price_cents": item["price"]["amount_cents"],
+        "currency": item["price"]["currency"],
+    }
+
+
+def load_items():
+    from airflow.providers.postgres.hooks.postgres import PostgresHook
+
+    rows = [to_app_row(item) for item in fetch_items()]
+    hook = PostgresHook(postgres_conn_id="app_db")
+    hook.run(CREATE)
+    for row in rows:
+        hook.run(UPSERT, parameters=row)
+    print(f"upserted {len(rows)} items")
+
+
+with DAG(
+    dag_id="items_to_postgres",
+    schedule="@hourly",
+    start_date=datetime(2026, 1, 1),
+    catchup=False,
+    default_args={"retries": 1, "retry_delay": timedelta(minutes=1)},
+    tags=["example"],
+) as dag:
+    PythonOperator(task_id="load_items", python_callable=load_items)

rig/examples/data/k8s/base/items-api.yaml

@@ -0,0 +1,95 @@
+# The simulator: a stub of the API the DAG reads, faithful to the wire (its field
+# names are the API's, not the app's). Same shape as the starter's example-mock.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: items-api-stub
+data:
+  routes.json: |
+    {
+      "/health":   {"status": 200, "body": {"status": "ok"}},
+      "/v1/items": {"status": 200, "body": {"items": [
+        {"id": "a-100", "name": "anvil",  "price": {"amount_cents": 1999, "currency": "USD"}},
+        {"id": "b-200", "name": "bucket", "price": {"amount_cents": 450,  "currency": "USD"}},
+        {"id": "c-300", "name": "crate",  "price": {"amount_cents": 1200, "currency": "USD"}}
+      ]}}
+    }
+  serve.py: |
+    import json, os
+    from http.server import BaseHTTPRequestHandler, HTTPServer
+
+    ROUTES = json.load(open("/etc/stub/routes.json"))
+    NAME = os.environ.get("STUB_NAME", "stub")
+
+    class H(BaseHTTPRequestHandler):
+        def do_GET(self):
+            r = ROUTES.get(self.path)
+            if r is None:
+                self.send_response(404)
+                self.end_headers()
+                self.wfile.write(json.dumps(
+                    {"error": "no canned route", "stub": NAME, "path": self.path}
+                ).encode())
+                return
+            body = json.dumps(r["body"]).encode()
+            self.send_response(r["status"])
+            self.send_header("Content-Type", "application/json")
+            self.send_header("X-Mocked-By", NAME)
+            self.end_headers()
+            self.wfile.write(body)
+
+        def log_message(self, fmt, *args):
+            print("%s %s" % (NAME, fmt % args), flush=True)
+
+    HTTPServer(("0.0.0.0", 8080), H).serve_forever()
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: items-api
+  labels:
+    app: items-api
+    rig.component/impl: mock
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: items-api
+  template:
+    metadata:
+      labels:
+        app: items-api
+    spec:
+      containers:
+        - name: stub
+          image: python:3.12-slim
+          command: ["python3", "/etc/stub/serve.py"]
+          env:
+            - name: STUB_NAME
+              value: items-api
+          ports:
+            - containerPort: 8080
+          volumeMounts:
+            - name: stub
+              mountPath: /etc/stub
+          readinessProbe:
+            httpGet: { path: /health, port: 8080 }
+            initialDelaySeconds: 2
+          resources:
+            requests: { memory: 32Mi, cpu: 10m }
+            limits:   { memory: 64Mi }
+      volumes:
+        - name: stub
+          configMap:
+            name: items-api-stub
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: items-api
+spec:
+  selector:
+    app: items-api
+  ports:
+    - port: 80
+      targetPort: 8080

rig/examples/data/k8s/base/kustomization.yaml

@@ -0,0 +1,10 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+# Beside the addons, in DATA_NAMESPACE — but no Namespace object: the addons created
+# it, and a Namespace Tilt owned would be deleted by `tilt down`, taking postgres and
+# airflow with it. rig's Tiltfile creates namespaces that are used and not declared.
+namespace: data
+
+resources:
+  - items-api.yaml

rig/examples/data/k8s/overlays/dev/kustomization.yaml

@@ -0,0 +1,5 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - ../../base

rig/examples/data/rig.env

@@ -0,0 +1,23 @@
+# This overlay's settings: layered over rig's defaults, under ctrl/.env and the caller.
+# data — postgres, redis and airflow (upstream images, run unmodified) in their own namespace.
+# Use it: OVERLAY=examples/data make cluster up    Notes: README.md
+
+# Order matters: addons install in the order listed, and airflow refuses to start
+# without postgres, so postgres comes first. metallb is rig's own. Airflow runs
+# LocalExecutor and needs no broker: add redis (before airflow) only to switch to Celery.
+ADDONS="metallb postgres airflow"
+
+# Namespace for the dependency containers (k8s/base/kustomization.yaml names it too).
+DATA_NAMESPACE=data
+
+# Postgres identity. The password is generated once by addons/postgres.sh and kept.
+POSTGRES_DB=app
+POSTGRES_USER=app
+POSTGRES_STORAGE=2Gi
+
+AIRFLOW_ADMIN_USER=admin
+
+# Upstream images, pinned by tag; bump freely, and preload them for an offline machine.
+POSTGRES_IMAGE=postgres:16-alpine
+REDIS_IMAGE=redis:7-alpine
+AIRFLOW_IMAGE=apache/airflow:2.10.4

rig/examples/starter/Dockerfile.example

@@ -0,0 +1,24 @@
+# EXAMPLE component image. Copy, rename, replace:
+#   Dockerfile.api  ->  image <cluster>-api  ->  image: in k8s/base/api.yaml
+# COPY paths are relative to the build context the overlay's Tiltfile names (context='.').
+# Notes: rig's docs/notes/Dockerfile.example.md
+
+FROM python:3.12-slim
+
+WORKDIR /app
+
+# Dependencies first, in their own layer, so a source edit does not reinstall them.
+COPY api/requirements.txt ./
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Context-relative — see above.
+COPY api/ ./api/
+
+# Match this with the containerPort in the manifest and the target of the
+# Service in front of it.
+EXPOSE 8000
+
+CMD ["python", "-m", "api"]
+
+# live_update: the Tiltfile's sync('api', '/app/api') must match COPY api/ + WORKDIR /app,
+# or edits silently do nothing.

rig/examples/starter/README.md

@@ -0,0 +1,59 @@
+# `examples/starter` — the overlay rig runs when you name none
+
+An overlay is one folder, outside rig, that holds what runs: its settings, its
+manifests, its images. rig reads it and never writes into it — see
+[`docs/notes/overlay.md`](../../docs/notes/overlay.md) for the contract. This one
+ships with rig so `make tilt` has something to deploy on a fresh clone, and so a
+real overlay has a shape to be written against:
+
+```
+rig.env              settings layered over rig's defaults (this one sets none)
+k8s/base/            the components, as plain manifests
+k8s/overlays/dev/    how this environment differs from the base — MANIFESTS_DIR's default
+Tiltfile             the workload's half of the dev loop; rig's ctrl/Tiltfile includes it
+Dockerfile.example   the shape of an image you build yourself
+```
+
+To start your own, copy this folder somewhere rig does not track —
+`rig/local/<name>/`, or a repo of its own — and name it:
+
+```bash
+cp -r examples/starter local/myenv
+OVERLAY=local/myenv make cluster up      # or OVERLAY=local/myenv in ctrl/.env
+```
+
+The cluster, context and port block then follow the overlay's folder name
+(`myenv`, `kind-myenv`), so it never collides with another.
+
+## The two components are examples, not the system
+
+They exist so the real manifests have a shape to be written against.
+
+### The three states a component can be in
+
+Switching between them should be a one-line change, never a rewrite. The DNS
+name stays the same in every case, so callers never know the difference:
+
+| state | what exists | when |
+| --- | --- | --- |
+| **real** | an image built from source, hot-reloaded | the one thing you are working on |
+| **mock** | a stub returning canned responses (`example-mock.yaml`) | everything else — most of the estate |
+| **remote** | no pod at all, just a Service (`example-remote.yaml`) | when the real system is reachable and you want it |
+
+Most components should be **mock**. What has to be faithful is the topology —
+names, ports, dependency order, who can reach whom, how it fails. The workloads
+are noise, and mocking them is what makes several copies of a large estate fit
+on one laptop.
+
+## Images
+
+`Dockerfile.example` is a commented shape, not a working build. Paths in this
+folder's Tiltfile are relative to this folder, so `context='.'` is the overlay
+and every `COPY` in the Dockerfile is relative to it:
+
+```
+docker_build(CLUSTER + '-api', context='.', dockerfile='Dockerfile.api')
+```
+
+The name given to `docker_build` must match `image:` in the manifest — that
+string is the only thing connecting the two.

rig/examples/starter/Tiltfile

@@ -0,0 +1,69 @@
+# The starter overlay's half of the dev loop. rig's ctrl/Tiltfile includes this once
+# it has applied k8s/overlays/dev, so every path here is relative to THIS folder.
+# Facts from rig, via os.getenv: RIG_CLUSTER RIG_CONTEXT RIG_HTTP_PORT RIG_HTTPS_PORT
+#   RIG_TILT_PORT RIG_REGISTRY RIG_OVERLAY_DIR
+# Notes: rig's docs/notes/overlay.md
+
+CLUSTER = os.getenv('RIG_CLUSTER')
+HTTP    = os.getenv('RIG_HTTP_PORT')
+
+# ── Images ─────────────────────────────────────────────────────────────────
+# (nothing yet — the examples run upstream images. Add docker_build calls here.)
+
+
+# ── Resources ──────────────────────────────────────────────────────────────
+# (nothing yet — add k8s_resource calls here to name and order what you deploy.)
+
+
+# ═══════════════════════════════════════════════════════════════════════════
+# Catalogue — paste what you need, delete the rest.
+# Commented out so this file runs as-is.
+# ═══════════════════════════════════════════════════════════════════════════
+#
+# ── build an image ─────────────────────────────────────────────────────────
+# context= and dockerfile= are both relative to this folder, so every COPY in the
+# Dockerfile is relative to the context you name here.
+#
+# docker_build(
+#     CLUSTER + '-api',                  # must match `image:` in the manifest —
+#     context='.',                       #   that string is the only thing
+#     dockerfile='Dockerfile.api',       #   connecting the two
+#     ignore=['.git', 'rig', '.venv', 'node_modules', '__pycache__'],
+#     live_update=[sync('api', '/app/api')],
+# )
+#
+# ── a shared base, built once ──────────────────────────────────────────────
+# Components that share code build FROM one base image instead of each carrying
+# a copy of it. Tilt builds the base first when a Dockerfile's FROM names it.
+#
+# docker_build(CLUSTER + '-base', context='repodir/base')
+# docker_build(CLUSTER + '-api', context='repodir/api')   # its Dockerfile: FROM <cluster>-base
+#
+# ── name and order a resource ──────────────────────────────────────────────
+# k8s_resource('api', resource_deps=['postgres'], labels=['app'])
+# k8s_resource('gateway', resource_deps=['api', 'ui'], labels=['app'])
+#
+# ── reload the gateway when its config changes ─────────────────────────────
+# A hash-less configMapGenerator ConfigMap never changes name, so edits do NOT
+# roll the pod on their own.
+#
+# local_resource(
+#     'gateway-reload',
+#     cmd='kubectl --context %s -n <namespace> rollout restart deployment/gateway' % os.getenv('RIG_CONTEXT'),
+#     deps=['k8s/base/Caddyfile'],
+#     resource_deps=['gateway'],
+#     auto_init=False,
+# )
+#
+# ── manifests that need kustomize flags ────────────────────────────────────
+# rig applies MANIFESTS_DIR without flags. If a secretGenerator reads above its
+# kustomization root, set MANIFESTS_DIR=none in rig.env and apply them here instead;
+# the flag loosens a safety check for the whole build.
+#
+# k8s_yaml(kustomize('k8s/overlays/dev', flags=['--load-restrictor=LoadRestrictionsNone']))
+#
+# ── reach a service directly, bypassing the gateway ────────────────────────
+# Prefer the gateway; host ports are shared machine-wide. If you need one, take it
+# from this environment's own port block.
+#
+# k8s_resource('postgres', port_forwards=[str(int(HTTP) + 5) + ':5432'])

rig/examples/starter/k8s/base/example-mock.yaml

@@ -0,0 +1,104 @@
+# EXAMPLE — a mocked component. Copy, rename, replace.
+#
+# A stub that answers on the right name and port with canned responses. No image
+# to build: the script is mounted from the ConfigMap, so changing the behaviour
+# is a kubectl apply, not a rebuild.
+#
+# Deliberately boring and readable. This is onboarding material — someone should
+# be able to read the generated object and recognise what it is.
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: example-service-stub
+data:
+  # Canned responses by path. Add entries as the contract becomes clear;
+  # anything unmatched returns 404 so a missing route is visible, not silent.
+  routes.json: |
+    {
+      "/health":     {"status": 200, "body": {"status": "ok"}},
+      "/v1/example": {"status": 200, "body": {"items": [], "mocked": true}}
+    }
+  serve.py: |
+    import json, os
+    from http.server import BaseHTTPRequestHandler, HTTPServer
+
+    ROUTES = json.load(open("/etc/stub/routes.json"))
+    NAME = os.environ.get("STUB_NAME", "stub")
+
+    class H(BaseHTTPRequestHandler):
+        def do_GET(self):
+            r = ROUTES.get(self.path)
+            if r is None:
+                self.send_response(404)
+                self.end_headers()
+                # Say which stub rejected it — with everything mocked, "404"
+                # alone tells you nothing about where the call actually landed.
+                self.wfile.write(json.dumps(
+                    {"error": "no canned route", "stub": NAME, "path": self.path}
+                ).encode())
+                return
+            body = json.dumps(r["body"]).encode()
+            self.send_response(r["status"])
+            self.send_header("Content-Type", "application/json")
+            self.send_header("X-Mocked-By", NAME)
+            self.end_headers()
+            self.wfile.write(body)
+
+        def log_message(self, fmt, *args):
+            print("%s %s" % (NAME, fmt % args), flush=True)
+
+    HTTPServer(("0.0.0.0", 8080), H).serve_forever()
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: example-service
+  labels:
+    app: example-service
+    rig.component/impl: mock      # so `kubectl get deploy -L rig.component/impl`
+                                  # shows at a glance what is real and what is not
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: example-service
+  template:
+    metadata:
+      labels:
+        app: example-service
+    spec:
+      containers:
+        - name: stub
+          image: python:3.12-slim
+          command: ["python3", "/etc/stub/serve.py"]
+          env:
+            - name: STUB_NAME
+              value: example-service
+          ports:
+            - containerPort: 8080
+          volumeMounts:
+            - name: stub
+              mountPath: /etc/stub
+          readinessProbe:
+            httpGet: { path: /health, port: 8080 }
+            initialDelaySeconds: 2
+          # Small enough that a whole estate of these fits alongside the real
+          # thing you are working on.
+          resources:
+            requests: { memory: 32Mi, cpu: 10m }
+            limits:   { memory: 64Mi }
+      volumes:
+        - name: stub
+          configMap:
+            name: example-service-stub
+---
+apiVersion: v1
+kind: Service
+metadata:
+  name: example-service
+spec:
+  selector:
+    app: example-service
+  ports:
+    - port: 80
+      targetPort: 8080

rig/examples/starter/k8s/base/example-remote.yaml

@@ -0,0 +1,46 @@
+# EXAMPLE — a component that is NOT simulated, pointed at the real system.
+#
+# This is the payoff of keeping the topology honest: there is no pod here at
+# all, yet `example-remote.<namespace>.svc.cluster.local` resolves exactly as it
+# does when the same component is mocked. Callers are identical in both cases,
+# so moving a dependency from mocked to real is a one-line change and nothing
+# downstream is touched.
+#
+# Use this when the real system is reachable and you want it in the loop.
+# Note that reachability depends on where you are running: systems restricted to
+# a managed workspace will not resolve from a laptop at all, which is the whole
+# reason most components should stay mocked.
+apiVersion: v1
+kind: Service
+metadata:
+  name: example-remote
+  labels:
+    rig.component/impl: remote
+spec:
+  type: ExternalName
+  externalName: real-system.internal.example.com
+---
+# If the real system has no DNS name — only an IP, which is common for legacy
+# hosts — ExternalName cannot express it. Use a bare Service plus manual
+# Endpoints instead, and delete the block above.
+#
+# apiVersion: v1
+# kind: Service
+# metadata:
+#   name: example-remote
+#   labels:
+#     rig.component/impl: remote
+# spec:
+#   ports:
+#     - port: 80
+#       targetPort: 8080
+# ---
+# apiVersion: v1
+# kind: Endpoints
+# metadata:
+#   name: example-remote      # must match the Service name exactly
+# subsets:
+#   - addresses:
+#       - ip: 10.0.0.42
+#     ports:
+#       - port: 8080

rig/examples/starter/k8s/base/kustomization.yaml

@@ -0,0 +1,11 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+# The namespace every component lands in. The overlay overrides it, so a rig
+# modelling two estates can apply the same base twice under different names.
+namespace: rig
+
+resources:
+  - namespace.yaml
+  - example-mock.yaml
+  - example-remote.yaml

rig/examples/starter/k8s/base/namespace.yaml

@@ -0,0 +1,4 @@
+apiVersion: v1
+kind: Namespace
+metadata:
+  name: rig

rig/examples/starter/k8s/overlays/dev/kustomization.yaml

@@ -0,0 +1,22 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
+kind: Kustomization
+
+resources:
+  - ../../base
+
+# The dev overlay is where a rig says how its estate differs from the base —
+# which components are real, which are mocked, which point at a live system.
+# Kept empty on purpose: the base already boots, and an overlay full of examples
+# is harder to read than one that starts blank.
+#
+# The shape a patch takes, for when the first one is needed:
+#
+# patches:
+#   - target: {kind: Service, name: example-service}
+#     patch: |
+#       - op: replace
+#         path: /spec/type
+#         value: NodePort
+#       - op: add
+#         path: /spec/ports/0/nodePort
+#         value: 30080

rig/examples/starter/rig.env

@@ -0,0 +1,9 @@
+# This overlay's settings: layered over rig's defaults, under ctrl/.env and the caller.
+# The starter sets nothing, so a rig with no overlay named behaves exactly like one with
+# no overlay at all. Any key a profile could set belongs here; paths are relative to
+# this folder. PROFILE and OVERLAY are refused — they are what chooses this file.
+# Notes: rig's docs/notes/overlay.md
+#
+# ADDONS="metallb"
+# K8S_VERSION=v1_36
+# MANIFESTS_DIR=k8s/overlays/dev