postgres, redis and airflow, each an upstream image run unmodified, installed as this overlay's own addons. rig ships the mechanism that finds and runs them; which services a workload needs is the workload's business, so they live here rather than in rig's ctrl/addons/. Copy what you need into your own overlay's addons/.

OVERLAY=examples/data make cluster up     # cluster `data`: postgres, then airflow
OVERLAY=examples/data make tilt           # the items-api simulator the DAG reads

rig.env                    ADDONS (metallb is rig's; the rest are here), namespace, identities, image pins
addons/postgres.sh         one replica on a PVC; the password generated once and kept
addons/airflow.sh          one `standalone` pod on its own `airflow` database; needs postgres
addons/redis.sh            only for switching airflow to CeleryExecutor (not in ADDONS)
k8s/                       the items-api simulator
dags/items_to_postgres.py  the worked example: API client → adapter → postgres
Tiltfile                   names the simulator's resource

Everything lands in the data namespace (DATA_NAMESPACE, and the kustomization names it too), so resetting an app's namespace leaves the databases alone. Costs roughly 2 GB with airflow, under 1 without. Airflow's first boot runs the whole metadata migration, so expect a few minutes before it is ready.

The worked example: three links kept apart

Metadata DB (infra). addons/airflow.sh creates an airflow database on the same postgres and points SQL_ALCHEMY_CONN there — airflow's own tables never land in the app's database.
DAG delivery. The addon turns this overlay's dags/ into the airflow-dags ConfigMap, mounted at /opt/airflow/dags; re-run make cluster up after editing a DAG. The faster path later: a kind extraMount of dags/ plus a Tilt sync — noted, not built.
Data connection (operational logic). AIRFLOW_CONN_APP_DB, composed each run from the postgres secret, gives DAGs the app's database as the app_db connection. One password reaches both URLs, with nowhere to drift.

The DAG itself calls the simulator with its own HTTP client (the wire, as the API returns it), renames the wire's fields into the app's names in to_app_row — the adapter, which belongs to whoever owns the app's model and so lives in the overlay — and upserts into items. Hourly, no backfill, one retry, idempotent on item_id.

kubectl --context kind-data -n data exec deploy/airflow -- airflow dags unpause items_to_postgres
kubectl --context kind-data -n data exec deploy/airflow -- airflow dags trigger items_to_postgres
kubectl --context kind-data -n data exec deploy/postgres -- psql -U app -d app -c 'select * from items'

Addons in an overlay

Each one runs from rig's ctrl/ (rig's addons.sh exports RIG_CTRL), so it starts with cd "${RIG_CTRL:?...}", sources ./lib/config.sh and calls load_config — and sees every key this overlay's rig.env sets. Run them through rig (bash ctrl/addons.sh install, or make cluster up), not directly.

postgres — plain manifests, one replica

Plain manifests rather than a helm chart: a chart repo is a network dependency, and an offline machine needs a path with none. The image is pinned in rig.env and can be preloaded into a local registry like every other image.

One replica on a PVC. This models a dependency for local work, not a highly-available database, and pretending otherwise on a kind node would be a more elaborate lie rather than a more useful one.

The password is not in rig.env: addons/postgres.sh generates one on first install and keeps it across re-runs, so re-running the addon never rotates the credential out from under whatever is already connected.

redis

Cache, and the broker anything queue-shaped runs on. No persistence: a broker that loses its queue on restart is the honest local model, and a PVC here buys nothing but a volume to clean up.

airflow

Airflow needs a metadata database before it will start at all, so the script refuses rather than rolls a pod that will CrashLoopBackOff while the real problem (postgres missing from ADDONS) stays invisible in the logs.

One pod on standalone: migration, admin user, scheduler and webserver in a single container, on LocalExecutor, which needs no broker. The official chart's five deployments model an installation; switching this on means wanting pipelines. redis is here for the day it moves to CeleryExecutor, and not before.

Reaching them

Reach the databases with port-forward rather than binding more host ports:

kubectl --context kind-data -n data port-forward svc/postgres 5432:5432
kubectl --context kind-data -n data port-forward svc/airflow 8080:8080
kubectl --context kind-data -n data get secret postgres -o jsonpath='{.data.POSTGRES_PASSWORD}' | base64 -d

README.md

examples/data — an overlay with a database, a scheduler and a worked pipeline