mariano/soleprint
Public Access
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

simpler check and deps messages

Browse Source
This commit is contained in:
buenosairesam
2026-09-17 15:01:48 -03:00
parent 1dc9d38c80
commit 565cecfb50
49 changed files with 1442 additions and 1426 deletions
Download Patch File Download Diff File Expand all files Collapse all files

32
rig/ctrl/.env.example
View File

@@ -1,50 +1,38 @@
# Machine-local config. Copy to ctrl/.env (gitignored) and edit.
# Cluster SHAPE: an optional profile in ctrl/env.d/ — see the *.env.example there.
# The architecture MODEL lives in arch/<name>.json — not here either.
# Cluster SHAPE: an optional profile in ctrl/env.d/. Architecture MODEL: arch/<name>.json.
# Notes: docs/notes/env.md
# A profile in ctrl/env.d/ to build. Empty means rig's built-in defaults, which
# need no profile at all. Copy an env.d/*.env.example to <name>.env to add one.
PROFILE=
# Cluster name; the kubectl context becomes kind-<CLUSTER>.
# LEAVE THIS UNSET unless you need a name that differs from the directory —
# it defaults to this folder's name, which is what makes the folder copyable:
# copy it, rename it, and you get a separate environment with no edits.
# LEAVE UNSET: it defaults to this folder's name, which keeps the folder copyable.
# CLUSTER=
# Host ports. LEAVE UNSET — they derive from the directory name so several
# environments coexist without negotiating (see ctrl/ports.sh). `make check`
# shows this environment's block; `bash ctrl/ports.sh persist` writes it here so it stops
# being derived and becomes fixed. Set a value only to override.
# Host ports. LEAVE UNSET — derived from the directory name (see ctrl/ports.sh).
# `bash ctrl/ports.sh persist` pins them here; set a value only to override.
# HTTP_PORT=
# HTTPS_PORT=
# TILT_PORT=
# REGISTRY_PORT=
# Where the application manifests live. The real ones are expected to be
# versioned separately from this installer — they change on a different cadence,
# by different people. Repoint this at their repo and rig stops owning them:
# Where the application manifests live; repoint at their own repo, e.g.
# MANIFESTS_DIR=../platform-manifests/overlays/dev
MANIFESTS_DIR=ctrl/k8s/overlays/dev
# Where the installer fetches the pinned binaries from.
# upstream GitHub releases / dl.k8s.io (needs internet)
# artifactory a generic repo — what a locked-down client usually allows
# baked already inside the installer image; no network at all
# Where the installer fetches the pinned binaries from:
# upstream (needs internet) | artifactory (generic repo) | baked (in the image)
DEPS_SOURCE=upstream
DEPS_ARTIFACTORY_URL=
# --- Registry -------------------------------------------------------------
# Mode comes from the profile (REGISTRY_MODE). These are the secrets it needs.
# Required for mirror/remote:
# Mode comes from the profile (REGISTRY_MODE). Secrets required for mirror/remote:
REGISTRY_REMOTE_URL=
REGISTRY_USER=
REGISTRY_PASSWORD=
# Corporate root CA, if Artifactory is fronted by an internal CA (it usually is).
# Trust has to reach THREE places and nothing does it for you: the host docker
# daemon, every kind node's containerd, and any in-cluster client. registry.sh
# handles the first two; check.sh reports when it's configured but not trusted.
# Corporate root CA, if Artifactory is fronted by an internal CA.
# Symptom when missing: x509: certificate signed by unknown authority
REGISTRY_CA_FILE=

26
rig/ctrl/Dockerfile.deps
View File

@@ -1,33 +1,17 @@
# The toolchain installer image. It does NOT run the cluster — it installs a toolchain
# onto the host and gets out of the way.
#
# This exists to kill a bootstrap paradox: a plain bash installer needs curl, jq
# and sha256sum to already be present, and a minimal Debian has none of them.
# It carries its own toolchain, so the only host prerequisite is Docker.
#
# Two variants from one file:
# Toolchain installer image: installs the pinned toolchain onto the host; Docker is the only prerequisite.
# docker build -f ctrl/Dockerfile.deps --target deps -t <slug>-deps .
# docker build -f ctrl/Dockerfile.deps --target deps-full -t <slug>-deps:full .
#
# deps-full bakes every pinned binary in at build time. `docker save` it and
# you have the whole installer as one file to carry into an air-gapped network.
# Notes: docs/notes/Dockerfile.deps.md
FROM debian:trixie-slim AS deps
# ca-certificates + curl: fetch and verify. graphviz + python3: render diagrams
# and validate the arch model, so the host never needs an apt package.
#
# docker-cli, NOT docker.io: we only ever talk to the host's daemon through the
# mounted socket, and under --no-install-recommends the docker.io package ships
# docker-init without the actual `docker` binary.
# curl: fetch and verify; graphviz + python3: diagrams. docker-cli, NOT docker.io
# (which lacks the `docker` binary under --no-install-recommends).
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates curl jq graphviz python3 docker-cli \
&& rm -rf /var/lib/apt/lists/*
# The installer is the generated standalone kit, not deps.sh plus the files it
# reads. A kit is one file with its pins frozen in and is proven to run with
# nothing else from rig present — which is exactly what an image needs, and
# `make standalone` keeps it current. Pins are the same in every profile's kit.
# The installer is the generated one-file standalone kit, pins frozen in.
ARG PROFILE=default
WORKDIR /work
COPY standalone/${PROFILE}/rigdeps.sh /work/rigdeps.sh

40
rig/ctrl/Dockerfile.example
View File

@@ -1,33 +1,13 @@
# EXAMPLE — a component image. Copy, rename, replace.
#
# Named like the manifest it feeds and the resource it becomes:
#
# ctrl/Dockerfile.api -> image <cluster>-api -> image: in k8s/base/api.yaml
#
# That image string is the ONLY thing connecting the three. Nothing checks it;
# a typo shows up as a pod stuck in ImagePullBackOff pulling from the public
# index, which reads like a network problem and is not one.
#
# ── the one that catches everyone ──────────────────────────────────────────
# The Tiltfile passes two paths with DIFFERENT bases, in adjacent arguments:
#
# context='..' the REPO ROOT (the Tiltfile is in ctrl/)
# dockerfile='Dockerfile.api' relative to the TILTFILE, so ctrl/Dockerfile.api
#
# So every COPY below is resolved against the repo root, NOT against this file's
# directory. A file sitting right beside this one is still reached as `ctrl/`:
#
# COPY ctrl/nginx.conf /etc/nginx/conf.d/default.conf # correct
# COPY nginx.conf /etc/nginx/conf.d/default.conf # fails — no such file
#
# Nothing warns you. The build just cannot find a file that is visibly there.
# EXAMPLE component image. Copy, rename, replace:
# ctrl/Dockerfile.api -> image <cluster>-api -> image: in k8s/base/api.yaml
# COPY paths are relative to the REPO ROOT (Tilt context='..'), not this directory.
# Notes: docs/notes/Dockerfile.example.md
FROM python:3.12-slim
WORKDIR /app
# Dependencies first, in their own layer: they change far less often than the
# code, so a source edit does not reinstall them on every rebuild.
# Dependencies first, in their own layer, so a source edit does not reinstall them.
COPY api/requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
@@ -40,11 +20,5 @@ EXPOSE 8000
CMD ["python", "-m", "api"]
# ── live_update ────────────────────────────────────────────────────────────
# The sync in the Tiltfile's docker_build must land where this image expects it:
#
# live_update=[sync('../api', '/app/api')]
#
# matches `COPY api/ ./api/` with WORKDIR /app. If the two disagree, Tilt syncs
# into a path nothing reads and the container keeps serving the built copy —
# edits appear to do nothing, with no error anywhere.
# live_update: Tiltfile's sync('../api', '/app/api') must match COPY api/ + WORKDIR /app,
# or edits silently do nothing.

76
rig/ctrl/Tiltfile
View File

@@ -1,28 +1,10 @@
# The dev loop. `make tilt` from the project root, or `cd ctrl && tilt up`.
#
# This file ships with rig and works unedited: rig's own k8s/base already boots,
# so `make tilt` comes up with a running cluster and no editing at all. What it
# deploys is two EXAMPLES — replace them, and add your own images and resources
# in the two marked sections near the bottom. The catalogue after them has the
# blocks to paste, with the parts that are easy to get wrong already commented.
#
# rig supplies this file; it does not own it. Nothing in rig reads it back, and
# nothing here is regenerated — edit it freely, the way you would edit
# k8s/base/example-mock.yaml. rig owns the machine, you own the workload.
#
# Nothing below is hardcoded to this directory, deliberately. Every other
# project here writes its slug into the Tiltfile five or six times by hand, so a
# copy of the project deploys into the original's cluster until someone
# remembers to edit all of them. A rig is meant to be copied and renamed, so it
# asks instead.
# Works unedited; replace the two EXAMPLES and add yours in the marked sections.
# rig supplies this file but does not own it, and nothing here names this directory.
# Notes: docs/notes/Tiltfile.md
# ── who we are, and on which ports ─────────────────────────────────────────
# One question to rig, answered by ctrl/ports.sh, which resolves it through
# lib/config.sh — the same path every other rig script takes. That is the point:
# the cluster name is NOT the bare directory name (it is lowercased and reduced
# to a DNS label), and the ports honour anything pinned in ctrl/.env. Recomputing
# either of those here in Starlark is how two copies end up disagreeing about
# which cluster they are talking to.
# Asked of ctrl/ports.sh (via lib/config.sh), never recomputed here in Starlark.
_facts = str(local('bash ports.sh active', quiet=True)).split()
CLUSTER = _facts[0]
CTX = _facts[1]
@@ -31,22 +13,13 @@ HTTPS = _facts[3]
TILT = _facts[4]
REGISTRY = _facts[5]
# Where the manifests live. rig's own are the default; point MANIFESTS_DIR in
# ctrl/.env at an overlay versioned somewhere else and rig stops owning them —
# see k8s/README.md. Real manifests usually change on a different cadence, by
# different people, under different review.
#
# The value is REPO-ROOT relative, because that is the root everything else in
# rig is expressed against. This file runs in ctrl/, so prefix rather than
# assume: '../' + 'ctrl/k8s/overlays/dev' and '../' + '../platform/overlays/dev'
# are both right, where stripping a leading 'ctrl/' would only fix the first.
# Where the manifests live (MANIFESTS_DIR, see k8s/README.md).
# The value is REPO-ROOT relative and this file runs in ctrl/, so prefix '../'.
MANIFESTS = '../' + _facts[6]
# ── refuse to deploy into the wrong cluster ────────────────────────────────
# Tilt snapshots the kubectl context at startup, BEFORE parsing this file, so it
# cannot be switched from here — only refused. `make tilt` passes --context for
# you; this catches a bare `tilt up` after some other project moved the global
# context.
# Tilt fixes the context before parsing this file, so it can only be refused here.
# `make tilt` passes --context; this catches a bare `tilt up`.
allow_k8s_contexts(CTX)
if k8s_context() != CTX:
fail("Wrong kubectl context: '%s'. This is %s — run: make tilt, or tilt up --context %s"
@@ -58,10 +31,8 @@ local('kubectl --context %s create namespace %s --dry-run=client -o yaml | kubec
% (CTX, CLUSTER, CTX), quiet=True)
# ── images go to this environment's own registry ───────────────────────────
# Fail closed. Tilt can usually infer the kind registry on its own, but "usually"
# is an inference, and when it misses, an unqualified name like 'app' quietly
# means docker.io/library/app — a push to the public index instead of the
# registry two lines away. rig runs that registry; name it.
# Fail closed: name the registry rather than let Tilt infer it, or a miss pushes
# an unqualified image to docker.io.
default_registry('localhost:' + REGISTRY)
k8s_yaml(kustomize(MANIFESTS))
@@ -82,21 +53,12 @@ k8s_resource(
# ═══════════════════════════════════════════════════════════════════════════
# Catalogue — paste what you need, delete the rest.
#
# These are the shapes that recur across every project here, with the reasoning
# kept next to them. They are comments so this file runs as-is.
# Commented out so this file runs as-is.
# ═══════════════════════════════════════════════════════════════════════════
#
# ── build an image ─────────────────────────────────────────────────────────
# The one genuinely non-obvious thing in the whole corpus: `context` and
# `dockerfile` are relative to DIFFERENT directories, in adjacent arguments,
# and nothing warns you.
#
# context= the REPO ROOT — this file is in ctrl/, so '..'
# dockerfile= relative to THIS file — so 'Dockerfile.api' is ctrl/Dockerfile.api
#
# Every COPY inside those Dockerfiles is therefore repo-root relative: a file
# sitting BESIDE the Dockerfile is still reached as `COPY ctrl/nginx.conf`.
# context= is the REPO ROOT ('..'); dockerfile= is relative to THIS file (ctrl/).
# So every COPY is repo-root relative, even for files beside the Dockerfile.
#
# docker_build(
# CLUSTER + '-api', # must match `image:` in the manifest —
@@ -111,10 +73,8 @@ k8s_resource(
# k8s_resource('gateway', resource_deps=['api', 'ui'], labels=['app'])
#
# ── reload the gateway when its config changes ─────────────────────────────
# A Caddyfile arriving via configMapGenerator with disableNameSuffixHash does
# NOT roll the pod — the ConfigMap name never changes, so nothing tells the
# Deployment anything happened. Without this you edit the routes and watch
# nothing take effect.
# A hash-less configMapGenerator ConfigMap never changes name, so edits do NOT
# roll the pod on their own.
#
# local_resource(
# 'gateway-reload',
@@ -131,9 +91,7 @@ k8s_resource(
# k8s_yaml(kustomize(MANIFESTS, flags=['--load-restrictor=LoadRestrictionsNone']))
#
# ── reach a service directly, bypassing the gateway ────────────────────────
# For a DB client or an admin UI. Prefer routing through the gateway: host ports
# are a single shared namespace across every project on this machine, which is
# why rig derives a block per environment in the first place. If you do need
# one, take it from this environment's own block rather than picking a number.
# Prefer the gateway; host ports are shared machine-wide. If you need one, take it
# from this environment's own port block.
#
# k8s_resource('postgres', port_forwards=[str(int(HTTP) + 5) + ':5432'])

5
rig/ctrl/addons.sh
View File

@@ -1,9 +1,8 @@
#!/usr/bin/env bash
# Install the addons the active profile asked for, in the order listed.
# Each addon is its own idempotent script in ctrl/addons/ — adding one is adding
# a file, not editing a dispatcher.
#
# One idempotent script per addon in ctrl/addons/.
# Usage: addons.sh install | list
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")"

12
rig/ctrl/addons/airflow.sh
View File

@@ -1,13 +1,7 @@
#!/usr/bin/env bash
# Apache Airflow — the cluster half of the airflow cabinet.
#
# Airflow needs a metadata database before it will start at all, so this refuses
# rather than rolls a pod that will CrashLoopBackOff while the real problem
# (postgres missing from ADDONS) stays invisible in the logs.
#
# One pod on `standalone`, matching the compose cabinet: migration, admin user,
# scheduler and webserver in a single container. The official chart's five
# deployments model an installation; switching this on means wanting pipelines.
# Apache Airflow — the cluster half of the airflow cabinet (one `standalone` pod).
# Requires the postgres addon; refuses to install without it.
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."

8
rig/ctrl/addons/cert-manager.sh
View File

@@ -1,10 +1,6 @@
#!/usr/bin/env bash
# cert-manager plus a self-signed cluster issuer.
#
# In a regulated estate almost everything is TLS, so the interesting question
# during onboarding is "does this service present a cert my client trusts" — not
# "can I reach a public ACME server". A local CA answers that offline, which is
# also what makes the air-gapped profile usable.
# cert-manager plus a self-signed cluster issuer (offline local CA).
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."

17
rig/ctrl/addons/metallb.sh
View File

@@ -1,15 +1,7 @@
#!/usr/bin/env bash
# MetalLB — makes `Service type: LoadBalancer` actually get an address.
#
# Why it matters here: real manifests use LoadBalancer, because a real cluster
# has one. On a bare kind cluster those Services sit at EXTERNAL-IP <pending>
# forever with no error anywhere — the deployment looks fine and simply is not
# reachable. Without this, every such Service has to be edited to NodePort,
# which means the local manifests stop matching the ones being modelled.
#
# The address pool is derived from the kind Docker network at install time, not
# hardcoded: Docker picks that subnet, it differs between machines, and a pool
# outside it is silently unroutable.
# The pool is derived from the kind Docker network at install time.
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."
@@ -57,10 +49,7 @@ else
$K apply -f "https://raw.githubusercontent.com/metallb/metallb/${METALLB_VERSION}/config/manifests/metallb-native.yaml"
fi
# `kubectl wait` on a selector errors out immediately when nothing matches yet,
# and right after apply the ReplicaSet has not created the pod — so it loses a
# race it looks like it should win. `rollout status` waits for the Deployment
# itself and handles the not-yet-created case.
# `rollout status`, not `kubectl wait`: wait errors out while the pod doesn't exist yet.
echo " waiting for the controller..."
$K rollout status deployment/controller -n metallb-system --timeout=240s
$K rollout status daemonset/speaker -n metallb-system --timeout=240s

8
rig/ctrl/addons/metrics-server.sh
View File

@@ -1,10 +1,6 @@
#!/usr/bin/env bash
# metrics-server — makes `kubectl top` work.
#
# kind nodes serve kubelet metrics over a self-signed cert, so the standard
# manifest never becomes ready without --kubelet-insecure-tls. That is fine here
# (it is a local cluster) and is the single most common reason metrics-server
# sits at 0/1 on kind.
# metrics-server — makes `kubectl top` work (patched with --kubelet-insecure-tls for kind).
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."

18
rig/ctrl/addons/postgres.sh
View File

@@ -1,20 +1,6 @@
#!/usr/bin/env bash
# PostgreSQL — the cluster half of the postgres cabinet.
#
# A cabinet is a public service dropped into the environment as-is — the
# upstream image, unmodified, reachable at a known address. This is the cluster
# half of it; the compose half is a `service.yml` beside a `cabinet.json`. The
# declaration is made once and both paths read it, so nothing is remembered
# twice.
#
# Plain manifests rather than a helm chart, matching the other addons: a chart
# repo is a network dependency, and the offline example profile exists precisely so
# there is a path with none. The image is pinned in ctrl/versions.env and can be
# preloaded into a local registry like every other image here.
#
# One replica on a PVC. This models a dependency for local work, not a
# highly-available database, and pretending otherwise on a kind node would be a
# more elaborate lie rather than a more useful one.
# PostgreSQL — the cluster half of the postgres cabinet: plain manifests, one replica on a PVC.
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."

7
rig/ctrl/addons/redis.sh
View File

@@ -1,9 +1,6 @@
#!/usr/bin/env bash
# Redis — the cluster half of the redis cabinet.
#
# Cache, and the broker anything queue-shaped runs on. No persistence: a broker
# that loses its queue on restart is the honest local model, and a PVC here buys
# nothing but a volume to clean up.
# Redis — the cluster half of the redis cabinet: cache/broker, no persistence.
# Notes: docs/notes/addons.md
set -euo pipefail
cd "$(dirname "$0")/.."

230
rig/ctrl/check.sh
View File

@@ -1,12 +1,7 @@
#!/usr/bin/env bash
# Readiness check: is this machine ready to run rig?
#
# Reports and instructs; never silently fixes anything. Everything it finds is
# either already fine, or something a human has to decide on.
#
# Runs ctrl/deps.sh host detection in a container when Docker is the only thing
# installed, or directly when the toolchain is already present. Then adds the
# checks that need this repo's config: profile sanity, CA trust, port clashes.
# Readiness check: is this machine ready to run rig? Reports and instructs; never fixes.
# Usage: check.sh [all | mem [status|push|all|backup|restore]] (all = every detail)
# Notes: docs/notes/check.md
set -euo pipefail
cd "$(dirname "$0")"
@@ -17,51 +12,25 @@ if [ "${1:-}" = mem ]; then
exec bash ./mem.sh "${@:-status}"
fi
DEPS_IMAGE="${DEPS_IMAGE:-$(basename "$(cd .. && pwd)")-deps}"
# Compact by default: facts only with `all`; problems (!) always print.
VERBOSE=""
if [ "${1:-}" = all ]; then VERBOSE=1; fi
fact() { if [ -n "$VERBOSE" ]; then echo "$@"; fi; }
# Host detection. Prefer running it bare — it needs no dependencies beyond
# coreutils — and fall back to the container only if this shell can't.
bash ./deps.sh detect
# ── repo-level checks ──────────────────────────────────────────────────────
bash ./deps.sh detect ${VERBOSE:+all}
source ./lib/config.sh
load_config
echo
echo "config"
echo " profile ${PROFILE_NAME} (nodes=${NODES})"
echo " cluster ${CLUSTER} (context ${KUBECONTEXT})"
echo " ingress ${INGRESS_MODE}"
if [ ! -f ./.env ]; then
echo " ! ctrl/.env missing — copy it: cp ctrl/.env.example ctrl/.env"
fi
# ── memory ─────────────────────────────────────────────────────────────────
#
# A profile on a box that is already full is the most common first failure, and
# it presents as pods stuck Pending rather than anything that says "memory".
# Warns; never blocks. Whether to try anyway is the user's call.
# A /proc/meminfo field in MB, 0 if absent. MEMINFO and OVERCOMMIT_FILE exist
# only so the tight and does-not-fit branches can be exercised against another
# machine's real numbers; in normal use they are the kernel's own files.
# A /proc/meminfo field in MB, 0 if absent. MEMINFO/OVERCOMMIT_FILE override for testing.
mb_of() {
awk -v k="$1:" '$1 == k { printf "%d", $2 / 1024; found = 1 }
END { if (!found) printf "0" }' "${MEMINFO:-/proc/meminfo}"
}
# NODE_MB — what one node costs — comes from load_config (lib/config.sh), where
# its measurement is recorded. It lives there, not here, because the memory tool
# and every standalone kit need the same number: a copy of it is how rigmini.sh
# came to say 2 GB per node long after rig had measured 800 MB.
# NODE_MB (cost of one node) comes from load_config in lib/config.sh; do not copy it here.
# Every running container's working set in MB, tagged with the kind cluster it
# belongs to ('-' when it is not kind). docker stats reports usage minus page
# cache, which is what actually competes — cache is handed back under pressure.
# Counting only kind would hide the usual culprit on a managed workspace, where
# the memory is held by other containers entirely.
# Every running container's working set in MB, tagged with its kind cluster ('-' if none).
container_mb() {
docker info >/dev/null 2>&1 || return 0
awk -F'\t' '
@@ -87,6 +56,25 @@ container_mb() {
<(docker stats --no-stream --format '{{.Name}}\t{{.MemUsage}}' 2>/dev/null)
}
port_busy() {
if command -v ss >/dev/null 2>&1; then
ss -ltn "sport = :$1" 2>/dev/null | grep -q LISTEN && return 0 || return 1
fi
# iproute2 is absent from a minimal Debian, so fall back to procfs rather
# than silently reporting everything as free.
local hex; hex=$(printf ':%04X' "$1")
grep -qi "^ *[0-9]*: [0-9A-F]*$hex " /proc/net/tcp /proc/net/tcp6 2>/dev/null
}
echo
echo "rig"
echo " cluster ${CLUSTER} (${KUBECONTEXT}) profile ${PROFILE_NAME}, ${NODES} node(s), registry ${REGISTRY_MODE}"
fact " ingress ${INGRESS_MODE}"
if [ ! -f ./.env ]; then
fact " .env none — built-in defaults (cp ctrl/.env.example ctrl/.env to set values)"
fi
# ── memory: does this cluster fit right now? Warns; never blocks. ──────────
total_mb=$(mb_of MemTotal)
avail_mb=$(mb_of MemAvailable)
swap_used_mb=$(( $(mb_of SwapTotal) - $(mb_of SwapFree) ))
@@ -94,141 +82,105 @@ overcommit=$(cat "${OVERCOMMIT_FILE:-/proc/sys/vm/overcommit_memory}" 2>/dev/nul
need_mb=$(( NODES * NODE_MB ))
rows=$(container_mb)
# Once this environment's own cluster is running, its real footprint is already
# out of MemAvailable and the per-node estimate stops being relevant. Subtracting
# the measurement from the estimate would count the same memory twice, and a
# running cluster that happens to sit under 800 MB would still "need" the gap.
# If our cluster is already up, its memory is already out of MemAvailable: need nothing more.
ours_mb=$(awk -F'\t' -v c="$CLUSTER" '$2 == c { s += $1 } END { print s + 0 }' <<< "$rows")
still_mb=$(( ours_mb > 0 ? 0 : need_mb ))
headroom=$(( avail_mb - still_mb ))
echo
echo "memory"
printf " this profile ~%d MB %s node(s) x %d MB — the cluster alone, your workload on top\n" \
"$need_mb" "$NODES" "$NODE_MB"
if [ "$ours_mb" -gt 0 ]; then
printf " already held %d MB by '%s', which is up\n" "$ours_mb" "$CLUSTER"
fi
printf " available %d MB of %d MB\n" "$avail_mb" "$total_mb"
# The biggest things holding memory right now, other than this cluster: kind
# clusters summed per cluster, everything else by container name.
# The biggest things holding memory, other than this cluster: kind clusters summed, the rest by name.
others=$(awk -F'\t' -v c="$CLUSTER" '
$2 != c && $2 != "-" && $2 != "" { k["kind cluster \x27" $2 "\x27"] += $1 }
$2 == "-" { k["container \x27" $3 "\x27"] += $1 }
END { for (n in k) printf "%d\t%s\n", k[n], n }' <<< "$rows" | sort -rn)
if [ -n "$others" ]; then
echo " held elsewhere:"
head -6 <<< "$others" | awk -F'\t' '{ printf " %6d MB %s\n", $1, $2 }'
n_others=$(wc -l <<< "$others")
if [ "$n_others" -gt 6 ]; then
echo " ... and $((n_others - 6)) more"
fi
fi
headroom=$(( avail_mb - still_mb ))
if [ "$still_mb" -eq 0 ]; then
if [ "$headroom" -ge 512 ]; then
printf " fits — already up; %d MB headroom for what you deploy\n" "$headroom"
else
printf " ! already up, but only %d MB headroom for anything you deploy\n" "$headroom"
fi
if [ "$still_mb" -eq 0 ] && [ "$headroom" -ge 512 ]; then
printf " memory up, holding %d MB — %d MB headroom for what you deploy\n" "$ours_mb" "$headroom"
elif [ "$still_mb" -eq 0 ]; then
printf " ! memory up, but only %d MB headroom for anything you deploy\n" "$headroom"
elif [ "$headroom" -ge 512 ]; then
printf " fits — %d MB headroom for what you deploy\n" "$headroom"
printf " memory fits — ~%d MB for %s node(s), %d MB headroom\n" "$need_mb" "$NODES" "$headroom"
elif [ "$headroom" -ge 0 ]; then
printf " ! fits, but only %d MB headroom for anything you deploy\n" "$headroom"
printf " ! memory fits, but only %d MB headroom (~%d MB for %s node(s))\n" "$headroom" "$need_mb" "$NODES"
else
printf " ! does not fit right now: ~%d MB needed, %d MB available\n" "$still_mb" "$avail_mb"
printf " ! memory does not fit: ~%d MB needed, %d MB available\n" "$still_mb" "$avail_mb"
# Two failures with opposite fixes, and telling them apart is the point.
if [ "$still_mb" -le "$total_mb" ]; then
echo " The machine is big enough; something else is holding memory (above)."
echo " Stopping that is what helps — a bigger VM would not."
echo " something else holds it (below) — stopping that helps, a bigger VM would not."
if grep -q 'kind cluster' <<< "$others"; then
echo " 'make cluster free' stops the other kind clusters. It stops, never deletes."
echo " 'make cluster free' stops the other kind clusters. It stops, never deletes."
fi
else
echo " The machine itself is too small: ~${still_mb} MB needed, ${total_mb} MB total."
echo " the machine itself is too small: ${total_mb} MB total."
fi
fi
if [ -n "$others" ] && { [ -n "$VERBOSE" ] || [ "$headroom" -lt 512 ]; }; then
echo " held elsewhere:"
head -6 <<< "$others" | awk -F'\t' '{ printf " %6d MB %s\n", $1, $2 }'
n_others=$(wc -l <<< "$others")
if [ "$n_others" -gt 6 ]; then
echo " ... and $((n_others - 6)) more"
fi
fi
if [ "$swap_used_mb" -gt 0 ]; then
printf " ! %d MB already in swap — available memory does not count it, so expect a\n" "$swap_used_mb"
echo " cluster here to be slow well before it fails"
fact " ${swap_used_mb} MB already in swap, which 'available' does not count: expect slow before failing"
fi
if [ "$overcommit" = "1" ]; then
echo " ! overcommit=1: allocations never fail here, so read 'fits' as a ceiling."
echo " A cluster that starts cleanly can still lose processes to the OOM killer."
fact " overcommit=1: allocations never fail, so read 'fits' as a ceiling (OOM killer settles up)"
fi
# The CA reaches three places and only one of them is ours. Report the other two.
if [ -n "${REGISTRY_CA_FILE:-}" ]; then
echo
echo "registry CA"
if [ ! -r "$REGISTRY_CA_FILE" ]; then
echo " ! REGISTRY_CA_FILE not readable: $REGISTRY_CA_FILE"
else
echo " file $REGISTRY_CA_FILE"
host="${REGISTRY_REMOTE_URL#*://}"; host="${host%%/*}"
if [ -n "$host" ] && [ ! -f "/etc/docker/certs.d/${host}/ca.crt" ]; then
echo " ! the HOST docker daemon does not trust it yet:"
echo " sudo mkdir -p /etc/docker/certs.d/${host}"
echo " sudo cp ${REGISTRY_CA_FILE} /etc/docker/certs.d/${host}/ca.crt"
echo " (kind nodes are handled by registry.sh; in-cluster clients are the workload's job)"
fi
fi
fi
# Host ports this environment will try to bind. Checked before cluster creation
# because docker reports a clash halfway through, as an opaque
# "failed to bind host port ...: address already in use".
echo
echo "ports (block derived from the directory name; pin it: bash ctrl/ports.sh persist)"
port_busy() {
if command -v ss >/dev/null 2>&1; then
ss -ltn "sport = :$1" 2>/dev/null | grep -q LISTEN && return 0 || return 1
fi
# iproute2 is absent from a minimal Debian, so fall back to procfs rather
# than silently reporting everything as free.
local hex; hex=$(printf ':%04X' "$1")
grep -qi "^ *[0-9]*: [0-9A-F]*$hex " /proc/net/tcp /proc/net/tcp6 2>/dev/null
}
# A port held by THIS environment's own cluster is not a clash — it is the thing
# working. Reporting it as a problem every time the cluster is up would train
# people to ignore this section, which is the opposite of the point.
# Extract with a second grep rather than `tr -d ':->'`: in tr, ':->' is the
# character RANGE ':' to '>', which does not contain '-', so the trailing dash
# survives and nothing ever matches.
# ── ports: checked before creation; docker reports a clash only halfway through. ──
# Ports held by our own cluster are not clashes. Second grep, not `tr -d ':->'` (a tr range).
ours=$(docker ps --filter "label=io.x-k8s.kind.cluster=${CLUSTER}" \
--format '{{.Ports}}' 2>/dev/null | tr ',' '\n' \
| grep -oE ':[0-9]+->' | grep -oE '[0-9]+' || true)
clash=0
clash=0 list="" mine=0
for entry in "HTTP:${HTTP_PORT}" "HTTPS:${HTTPS_PORT}" \
"TILT:${TILT_PORT}" "REGISTRY:${REGISTRY_PORT}"; do
name="${entry%%:*}"; p="${entry#*:}"
[ -n "$p" ] || continue
list+="$p "
if ! port_busy "$p"; then
printf " %-9s %-6s free\n" "$name" "$p"
fact "$(printf " %-9s %-6s free" "$name" "$p")"
elif echo "$ours" | grep -qx "$p"; then
printf " %-9s %-6s in use by this environment's cluster\n" "$name" "$p"
mine=1
fact "$(printf " %-9s %-6s in use by this environment's cluster" "$name" "$p")"
else
printf " ! %-9s %-6s IN USE by something else\n" "$name" "$p"
printf " ! ports %s %s IN USE by something else\n" "$name" "$p"
clash=1
fi
done
if [ "$clash" -eq 1 ]; then
echo " override the clashing one in ctrl/.env, e.g. HTTP_PORT=21080"
echo " (or rename this directory — the whole block follows the name)"
echo " override it in ctrl/.env (e.g. HTTP_PORT=21080), or rename this directory"
elif [ "$mine" -eq 1 ]; then
echo " ports ${list% } held by this cluster"
else
echo " ports ${list% } free"
fi
fact " derived from the directory name; pin them: bash ctrl/ports.sh persist"
# What `make cluster up` wires in beside the cluster. Both are set up by it —
# listed here only so there is nothing to run just to look.
echo
echo "registry"
bash ./registry.sh status | sed 's/^/ /'
# ── what `make cluster up` wires in beside the cluster ─────────────────────
REG_NAME="${CLUSTER}-registry"
if state=$(docker inspect -f '{{.State.Status}}' "$REG_NAME" 2>/dev/null); then
echo " registry localhost:${REGISTRY_PORT} ($state)"
else
fact " registry no container yet — 'make cluster up' starts it"
fi
echo " addons ${ADDONS:-none}"
fact " available: $(ls addons/*.sh 2>/dev/null | xargs -n1 basename | sed 's/\.sh$//' | tr '\n' ' ')"
echo
echo "addons"
bash ./addons.sh list | sed 's/^/ /'
# The CA reaches three places and only one of them is ours. Report the other two.
if [ -n "${REGISTRY_CA_FILE:-}" ]; then
if [ ! -r "$REGISTRY_CA_FILE" ]; then
echo " ! CA REGISTRY_CA_FILE not readable: $REGISTRY_CA_FILE"
else
fact " CA $REGISTRY_CA_FILE"
host="${REGISTRY_REMOTE_URL#*://}"; host="${host%%/*}"
if [ -n "$host" ] && [ ! -f "/etc/docker/certs.d/${host}/ca.crt" ]; then
echo " ! CA the HOST docker daemon does not trust it yet:"
echo " sudo mkdir -p /etc/docker/certs.d/${host}"
echo " sudo cp ${REGISTRY_CA_FILE} /etc/docker/certs.d/${host}/ca.crt"
echo " (kind nodes are handled by registry.sh; in-cluster clients are the workload's job)"
fi
fi
fi

14
rig/ctrl/cluster.sh
View File

@@ -1,17 +1,7 @@
#!/usr/bin/env bash
# Cluster lifecycle, plus what else is running on this machine.
#
# `list` and `free` live here rather than in a separate script because a
# near-identical second name (cluster / clusters) is a trap — you reach for one
# and get the other. One target, one file, unambiguous subcommands.
#
# "Idempotent" here means CONVERGENT, not "exits early if the cluster exists".
# That distinction matters: an interrupted first run can leave a cluster created
# but not finished, and returning early on the re-run would strand it there.
# The create step is conditional; every step after it always runs, and each one
# is individually idempotent.
#
# Cluster lifecycle (convergent, not exit-early), plus what else runs on this machine.
# Usage: cluster.sh up | down | reset | list | free
# Notes: docs/notes/cluster.md
set -euo pipefail
cd "$(dirname "$0")"

276
rig/ctrl/deps.sh
View File

@@ -1,38 +1,18 @@
#!/usr/bin/env bash
# rig:standalone rigdeps detect
# Toolchain installer: detect the host, install a pinned toolchain onto it, then
# report what it could not do.
#
# It never runs the cluster, never uses sudo or apt, and writes only into
# $OUT_BIN (default ~/.local/bin). Everything that would touch the host proper —
# systemd, inotify limits, .wslconfig, docker group — is REPORTED for a human to
# decide on, never performed. That is what makes it safe to run on a machine that
# already has a working setup.
#
# Usage (normally via `make deps`, or directly):
# deps.sh detect # report host facts only, change nothing
# deps.sh list # the pinned versions
# deps.sh verify [core|dev] # run what is installed and see if it works
# deps.sh fetch [core|dev] [--to DIR] # download + verify into DIR
# deps.sh install [core|dev] # detect, fetch, install, report
#
# Tiers: 'core' is kubectl + jq (talk to a cluster); 'dev' adds kind and tilt
# Default is dev.
#
# Runs both inside the installer container and bare on a host. Inside the
# container, host files are read through $HOST_ROOT (mount / as :ro); bare, it
# falls back to /.
# Toolchain installer: detect the host, install pinned tools into $OUT_BIN, report
# host actions it will not perform (no sudo, no apt). Usually via `make deps`.
# Usage: deps.sh [detect [all] | list | verify [core|dev] | fetch [core|dev] [--to DIR] | install [core|dev]]
# Notes: docs/notes/deps.md
set -euo pipefail
# Keep the caller's cwd so a relative --to resolves where the user expects,
# not against ctrl/ once we've moved.
# Keep the caller's cwd so a relative --to resolves there, not against ctrl/.
INVOKED_FROM="$PWD"
cd "$(dirname "$0")"
# Pins arrive through load_config like every other setting, not by sourcing
# versions.env here. That is what lets `make standalone` freeze them into a
# one-file installer: configuration has exactly one way in.
# Pins arrive through load_config, not by sourcing versions.env, so `make
# standalone` can freeze them in.
source ./lib/config.sh
load_config
@@ -53,12 +33,11 @@ BAKED_BIN="${BAKED_BIN:-/opt/rig/bin}"
# Collected by detect(), printed by report_manual() at the very end.
MANUAL=()
# Host FILES (/etc/..., /mnt/c/...) must be read through the mount. Kernel-level
# facts (kernel version, meminfo, inotify) are shared with the container, so the
# container's own view is already the host's.
# A /proc/meminfo field in MB, 0 if the field is absent. MEMINFO exists so the
# tight and does-not-fit branches can be exercised against a real machine's
# numbers from somewhere else; in normal use it is always /proc/meminfo.
# Facts print only with VERBOSE (`detect all`); problems (! and -) always print.
fact() { if [ -n "${VERBOSE:-}" ]; then echo "$@"; fi; }
# Host FILES are read through $HOST_ROOT; kernel facts are shared with the container.
# A /proc/meminfo field in MB, 0 if absent. MEMINFO overrides the source for testing.
mb_of() {
awk -v k="$1:" '$1 == k { printf "%d", $2 / 1024; found = 1 }
END { if (!found) printf "0" }' "${MEMINFO:-/proc/meminfo}"
@@ -83,9 +62,7 @@ arch() {
esac
}
# The pins above are amd64. Rather than download something that cannot execute
# and let it fail as "cannot execute binary file: Exec format error", say so
# here and hand over the commands that produce the right checksums.
# Pins are amd64 only: refuse elsewhere and print how to get the right checksums.
require_amd64() {
local a; a=$(arch)
[ "$a" = "amd64" ] && return 0
@@ -138,9 +115,7 @@ pick_sha() {
}
# ── package manager, for the instructions only ─────────────────────────────
# This never runs a package manager. It names one so the reported action is
# something you can paste, on the distro you are actually on — an apt line on
# Amazon Linux 2 is a wrong answer dressed up as help.
# Never runs one; names the right one so reported actions are pasteable.
pkg_install_cmd() {
local pkg="$1"
@@ -160,9 +135,7 @@ docker_pkg() {
# ── detect ─────────────────────────────────────────────────────────────────
# Windows outside WSL — Git Bash, MSYS, Cygwin — looks close enough to work and
# then fails in a pile of confusing ways: no /proc, no docker socket, none of
# the tooling. Detectable, so name it instead.
# Windows outside WSL (Git Bash, MSYS, Cygwin) fails confusingly; name it instead.
require_linux() {
case "$(uname -s)" in
MINGW*|MSYS*|CYGWIN*)
@@ -187,36 +160,30 @@ is_wsl() { grep -qi microsoft /proc/version 2>/dev/null; }
detect() {
echo "host"
echo " kernel $(uname -r)"
echo " arch $(arch) ($(uname -m))"
fact " kernel $(uname -r)"
local osr; osr=$(host_file /etc/os-release)
[ -r "$osr" ] && echo " distro $(sed -n 's/^PRETTY_NAME="\(.*\)"/\1/p' "$osr")"
local osr distro=""; osr=$(host_file /etc/os-release)
[ -r "$osr" ] && distro=$(sed -n 's/^PRETTY_NAME="\(.*\)"/\1/p' "$osr")
echo " distro ${distro:-unknown} $(arch), $(if is_wsl; then echo WSL; else echo native linux; fi)"
# In MB. Whole gigabytes lose nearly half a GB on exactly the machines where
# it matters: 1874 MB available used to print as "1 GB". Facts only — whether
# that is enough depends on the profile, which check.sh knows and this does not.
# In MB (whole GB rounds away too much). Facts only; check.sh judges sufficiency.
local total_mb avail_mb swap_total_mb swap_used_mb om
total_mb=$(mb_of MemTotal)
avail_mb=$(mb_of MemAvailable)
swap_total_mb=$(mb_of SwapTotal)
swap_used_mb=$(( swap_total_mb - $(mb_of SwapFree) ))
printf " memory %d MB total, %d MB available\n" "$total_mb" "$avail_mb"
if [ "$swap_total_mb" -gt 0 ]; then
printf " swap %d MB used of %d MB\n" "$swap_used_mb" "$swap_total_mb"
fi
printf " memory %d MB total, %d MB available%s\n" "$total_mb" "$avail_mb" \
"$(if [ "$swap_used_mb" -gt 0 ]; then echo ", $swap_used_mb MB in swap"; fi)"
# How the kernel answers an allocation it cannot really satisfy. With 1 it
# always says yes and settles up later with the OOM killer, so a cluster that
# starts cleanly can still lose processes afterwards.
# Overcommit mode: with 1 the OOM killer settles up later, after a clean start.
om=$(cat "${OVERCOMMIT_FILE:-/proc/sys/vm/overcommit_memory}" 2>/dev/null || echo '?')
case "$om" in
0) echo " overcommit 0 heuristic — allocations are granted on a guess" ;;
1) echo " overcommit 1 always — every allocation succeeds; the OOM killer is the only limit" ;;
2) echo " overcommit 2 strict — an allocation fails honestly instead of killing later" ;;
0) fact " overcommit 0 heuristic — allocations are granted on a guess" ;;
1) fact " overcommit 1 always — every allocation succeeds; the OOM killer is the only limit" ;;
2) fact " overcommit 2 strict — an allocation fails honestly instead of killing later" ;;
esac
echo " install to $OUT_BIN"
fact " install to $OUT_BIN"
detect_libc
detect_prereqs
detect_wsl
@@ -228,18 +195,13 @@ detect() {
detect_wsl() {
if ! is_wsl; then
echo " platform native linux"
return
fi
echo " platform WSL"
# systemd is off by default in WSL, and the ingress/DNS paths that use a
# host service need it. Enabling it requires a Windows-side restart, which
# cannot be issued from inside the distro.
# systemd is off by default in WSL; enabling it needs a Windows-side restart.
local wc; wc=$(host_file /etc/wsl.conf)
if [ -r "$wc" ] && grep -qE '^\s*systemd\s*=\s*true' "$wc"; then
echo " systemd enabled in wsl.conf"
fact " systemd enabled in wsl.conf"
else
echo " ! systemd not enabled in /etc/wsl.conf"
MANUAL+=("Enable systemd — add to /etc/wsl.conf:
@@ -251,15 +213,15 @@ detect_wsl() {
# WSL regenerates /etc/resolv.conf on every boot, which silently reverts any
# local DNS setup.
if [ -r "$wc" ] && grep -qE '^\s*generateResolvConf\s*=\s*false' "$wc"; then
echo " resolv.conf pinned (generateResolvConf=false)"
fact " resolv.conf pinned (generateResolvConf=false)"
else
echo " - resolv.conf is WSL-generated; DNS_MODE=dnsmasq would be reverted on reboot"
fact " - resolv.conf is WSL-generated; DNS_MODE=dnsmasq would be reverted on reboot"
fi
local wcfg
wcfg=$(ls "$HOST_ROOT"/mnt/c/Users/*/.wslconfig 2>/dev/null | head -1 || true)
if [ -n "$wcfg" ] && grep -qE '^\s*memory\s*=' "$wcfg"; then
echo " wslconfig memory set: $(grep -E '^\s*memory\s*=' "$wcfg" | tr -d ' ')"
fact " wslconfig memory set: $(grep -E '^\s*memory\s*=' "$wcfg" | tr -d ' ')"
else
MANUAL+=("Cap/raise the WSL VM memory — see what is set versus what booted:
make check mem
@@ -267,11 +229,8 @@ detect_wsl() {
fi
}
# Not a path check: /mnt is an ordinary mount point and an ext4 disk mounted
# there is perfectly fine. What matters is the filesystem. The Windows drives
# arrive as 9p (WSL2) or drvfs (WSL1); network and fuse mounts behave the same
# way. None of them deliver inotify events, so anything watching files goes
# quiet without saying why.
# Filesystem types that deliver no inotify events (9p, drvfs, network, fuse).
# Checks the fs type, not the path.
watch_hostile_fs() {
local dir="$1" fstype
fstype=$(findmnt -no FSTYPE --target "$dir" 2>/dev/null || true)
@@ -292,28 +251,22 @@ detect_filesystem() {
on a $fstype mount, and everything else is slower:
cp -r \"$root\" ~/ && cd ~/$(basename "$root")")
else
echo " filesystem $root ($(findmnt -no FSTYPE --target "$root" 2>/dev/null || echo local))"
fact " filesystem $root ($(findmnt -no FSTYPE --target "$root" 2>/dev/null || echo local))"
fi
}
# tilt is the one binary here that needs a recent glibc. MEASURED, not guessed:
# tilt 0.37.6 on Amazon Linux 2 (glibc 2.26) fails with
#
# /lib64/libc.so.6: version `GLIBC_2.34' not found (required by .../tilt)
#
# which names a symbol rather than the problem. Amazon Linux 2 is a stock
# WorkSpaces bundle, so this is the likely case, not an exotic one. Report the
# version now; `verify` catches the actual failure after installing.
# tilt needs glibc >= 2.34 (measured on Amazon Linux 2). Report the version here;
# `verify` catches the actual failure after installing.
detect_libc() {
local v=""
if command -v ldd >/dev/null 2>&1; then
v=$(ldd --version 2>/dev/null | head -1 | grep -oE '[0-9]+\.[0-9]+$' || true)
fi
if [ -z "$v" ]; then
echo " libc unknown (no ldd) — 'verify' is the real test"
fact " libc unknown (no ldd) — 'verify' is the real test"
return 0
fi
echo " libc glibc $v"
fact " libc glibc $v"
if [ "$(printf '%s\n2.34\n' "$v" | sort -V | head -1)" != "2.34" ]; then
echo " ! older than glibc 2.34, which tilt needs. kubectl, kind, jq and"
echo " ctlptl are static or libc-only and work here; tilt will not start."
@@ -322,25 +275,23 @@ detect_libc() {
return 0
}
# What this script needs to do its own job. Reported here so `detect` answers
# "will install work?" instead of leaving you to find out one download in.
# Amazon Linux 2 ships without tar, which is exactly the surprise this catches.
# What this script itself needs, so `detect` answers "will install work?".
detect_prereqs() {
local missing=""
if command -v curl >/dev/null 2>&1; then echo " download curl"
elif command -v wget >/dev/null 2>&1; then echo " download wget"
if command -v curl >/dev/null 2>&1; then fact " download curl"
elif command -v wget >/dev/null 2>&1; then fact " download wget"
else echo " ! no curl and no wget — nothing can be downloaded"; missing+=" curl"
fi
if command -v sha256sum >/dev/null 2>&1 || command -v shasum >/dev/null 2>&1; then
echo " checksums ok"
fact " checksums ok"
else
echo " ! no sha256sum or shasum — downloads could not be verified"
missing+=" coreutils"
fi
if command -v tar >/dev/null 2>&1 && command -v gzip >/dev/null 2>&1; then
echo " archives tar + gzip"
fact " archives tar + gzip"
else
echo " ! no tar/gzip — tilt and ctlptl ship as tarballs, so the dev tier"
echo " cannot be unpacked. The core tier is two bare binaries and is fine."
@@ -355,10 +306,7 @@ detect_prereqs() {
}
detect_docker() {
# Reachability of the daemon is the real question, and the CLI is only how
# we ask it. Note that when this runs inside the installer container, Docker
# necessarily exists on the host — otherwise nothing would be executing —
# so a missing CLI in here is an installer packaging bug, not a host problem.
# Daemon reachability is the real question; the CLI is only how we ask.
if ! command -v docker >/dev/null 2>&1; then
if [ -S /var/run/docker.sock ]; then
echo " docker socket present (no cli in this context)"
@@ -377,12 +325,9 @@ detect_docker() {
echo " docker $(docker version --format '{{.Server.Version}}' 2>/dev/null)"
local n
n=$(docker ps --filter "label=io.x-k8s.kind.cluster" --format '{{.Names}}' 2>/dev/null | wc -l)
# Must be an `if`, not `[ ] && echo`: as the last statement in this
# function the latter returns 1 when the count is zero, and `set -e`
# then kills the caller. That is the fresh-machine case — no clusters
# yet — so the bug only ever shows up where it does most harm.
# Must be an `if`, not `[ ] && echo`: a zero count would return 1 under set -e.
if [ "$n" -gt 0 ]; then
echo " - $n kind node container(s) already running; see 'make cluster list'"
echo " kind $n node container(s) running — 'make cluster list'"
fi
else
echo " ! docker cli present but the daemon is unreachable"
@@ -397,7 +342,7 @@ detect_inotify() {
local w i
w=$(cat /proc/sys/fs/inotify/max_user_watches 2>/dev/null || echo 0)
i=$(cat /proc/sys/fs/inotify/max_user_instances 2>/dev/null || echo 0)
echo " inotify watches=$w instances=$i"
fact " inotify watches=$w instances=$i"
if [ "$w" -lt 524288 ] || [ "$i" -lt 512 ]; then
echo " ! inotify limits are low — Tilt will silently stop noticing file changes"
@@ -457,18 +402,13 @@ fetch_tgz() {
echo " fetching $name"
download "$(resolve_url "$url")" "$tmp"
verify "$tmp" "$sha" "$name"
# --no-same-owner: extracting as root would otherwise restore the uid/gid
# baked into the archive (some ship as uid 1001), leaving a binary the host
# user does not own.
# --no-same-owner: as root, tar would restore the archive's uid/gid.
tar -xzf "$tmp" -C "$dest" --strip-components="$strip" --no-same-owner "$inner"
rm -f "$tmp"
chmod +x "$dest/$name"
}
# The installer runs as root so it can reach the docker socket, which means
# everything it writes into a mounted volume lands root-owned and unusable from
# the host. Hand it back to whoever owns the mount point (the host user created
# that directory before mounting it).
# The installer runs as root; hand files in a mounted dir back to the mount point's owner.
fix_ownership() {
local dir="$1"
[ -d "$dir" ] || return 0
@@ -480,38 +420,14 @@ fix_ownership() {
chown -R "$owner" "$dir" 2>/dev/null || true
}
# Two tiers, because not every machine should get cluster tooling.
#
# core kubectl, jq — talk to a cluster someone else runs. Nothing that
# creates one. Appropriate on a managed or corporate-issued machine
# where development tools are not wanted by default.
# dev core plus kind and tilt — build clusters and hot-reload into them.
#
# The split exists because "install the toolchain" is not one decision: on a
# managed workspace the right answer is kubectl and nothing else.
# core: talk to a cluster someone else runs. dev: core plus tools that build clusters.
CORE_TOOLS="kubectl jq"
# No helm: every addon installs with `kubectl apply -f <url>`, so nothing here
# has ever invoked it. Add it back the day something actually needs a chart.
#
# ctlptl is 'dev' rather than 'core' for the same reason kind is: core is "talk
# to a cluster someone else runs", and ctlptl builds them. It earns its place
# because it is what wires a cluster to a local registry — without one, an
# unqualified image name resolves to docker.io/library/<name> and there is
# nothing structural stopping a push there.
#
# docker-compose is 'dev' for the same reason, and is here because the distro
# docker packages ship the daemon and CLI but frequently not the compose
# plugin — so `docker compose up` fails with "unknown command" on an otherwise
# working Docker, and nothing about that message names the missing piece.
# No helm (nothing uses a chart). ctlptl wires in a local registry; compose is often
# missing from distro docker packages.
DEV_TOOLS="kind tilt ctlptl docker-compose"
# ── what is already on this machine ───────────────────────────────────────
#
# A tool already on PATH at its pinned version is left where it is. Without
# this, install downloads a second copy into OUT_BIN and then reports the first
# one as shadowed — noise, and wrong, when both are the same version. That is
# the normal state of any machine someone set up by hand, whatever directory
# they happened to choose.
# A tool already on PATH at its pinned version is left where it is.
pin_of() {
case "$1" in
@@ -524,9 +440,7 @@ pin_of() {
esac
}
# The version string a binary reports. Each tool spells the question
# differently, and kubectl has to be told --client or it goes looking for a
# server to ask.
# The version string a binary reports (kubectl needs --client).
reported_version() {
local tool="$1" path="$2"
case "$tool" in
@@ -536,13 +450,8 @@ reported_version() {
esac
}
# Does the binary at PATH report PIN? Matched as a whole version token, so
# 0.37.6 never matches 10.37.60, with the leading v optional either side: kind
# says v0.32.0, jq says jq-1.8.2, and tilt says v0.37.6 against a pin of 0.37.6.
#
# Bash's own regex rather than grep, deliberately. grep is not the same program
# on every machine — some builds reject patterns that others accept — and a
# failed grep inside a count reads exactly like a zero.
# Does the binary at PATH report PIN? Whole-token match, leading v optional.
# Bash regex rather than grep, deliberately.
version_matches() {
local tool="$1" path="$2" pin="$3" out v re
out=$(reported_version "$tool" "$path") || return 1
@@ -552,10 +461,8 @@ version_matches() {
[[ $out =~ $re ]]
}
# DEPS_ONLY narrows a fetch to the tools it names. Unset means the whole tier,
# which is what an explicit `deps.sh fetch` always gets: "download these into
# DIR" must not quietly skip something because this machine happens to have it.
# Only install() sets it, to what detect_toolchain found missing or mismatched.
# DEPS_ONLY narrows a fetch to the tools it names; unset means the whole tier.
# Only install() sets it.
want() { [ -z "${DEPS_ONLY:-}" ] || [[ " $DEPS_ONLY " == *" $1 "* ]]; }
# Every tool in the tier with its state, probed once and reported once. What
@@ -564,20 +471,18 @@ TOOLCHAIN_NEED=""
detect_toolchain() {
local tier="${TIER:-dev}" b pin path found
TOOLCHAIN_NEED=""
local n=0
echo
echo "toolchain (pinned, tier '$tier')"
fact "toolchain (pinned, tier '$tier')"
for b in $(tier_tools "$tier"); do
n=$((n + 1))
pin=$(pin_of "$b")
path=$(command -v "$b" 2>/dev/null || true)
# compose is the one tool that is normally NOT a binary on PATH. It is a
# docker CLI plugin, so a machine where `docker compose` works perfectly
# has no `docker-compose` to find — and probing only PATH would report it
# missing and re-download a copy that is already there. That is the exact
# noise the version-aware skip exists to prevent, so ask docker instead.
# compose is normally a docker CLI plugin, not on PATH: ask docker instead.
if [ "$b" = docker-compose ] && [ -z "$path" ]; then
if found=$(docker compose version --short 2>/dev/null) && [ -n "$found" ]; then
if [ "${found#v}" = "${pin#v}" ]; then
printf " %-8s %-9s %s\n" "$b" "$pin" "docker cli plugin"
fact "$(printf " %-8s %-9s %s" "$b" "$pin" "docker cli plugin")"
else
printf " ! %-8s wants %s, the docker cli plugin reports '%s'\n" \
"$b" "$pin" "$found"
@@ -590,7 +495,7 @@ detect_toolchain() {
printf " - %-8s %-9s not found\n" "$b" "$pin"
TOOLCHAIN_NEED+="$b "
elif version_matches "$b" "$path" "$pin"; then
printf " %-8s %-9s %s\n" "$b" "$pin" "$path"
fact "$(printf " %-8s %-9s %s" "$b" "$pin" "$path")"
else
found=$(reported_version "$b" "$path" 2>/dev/null | head -1 || true)
printf " ! %-8s wants %s, %s reports '%s'\n" "$b" "$pin" "$path" "$found"
@@ -598,9 +503,10 @@ detect_toolchain() {
fi
done
if [ -z "$TOOLCHAIN_NEED" ]; then
echo " every pinned tool is already on PATH — nothing to fetch"
if [ -n "${VERBOSE:-}" ]; then echo " all $n on PATH — nothing to fetch"
else echo "toolchain all $n pinned tools on PATH (tier $tier)"; fi
else
echo " 'make deps' fetches only: ${TOOLCHAIN_NEED% }"
echo "toolchain 'make deps' fetches only: ${TOOLCHAIN_NEED% }"
fi
}
@@ -641,8 +547,7 @@ fetch() {
fi
fix_ownership "$dest"
# kind writes the kubeconfig as root too; hand that back as well when it's
# a mounted host directory rather than container-local state.
# kind writes the kubeconfig as root too; hand that back as well.
fix_ownership "${KUBE_DIR:-/out/kube}"
}
@@ -664,14 +569,8 @@ report_manual() {
done
}
# Installing into a directory that sits early in PATH silently replaces whatever
# the machine was already using — which on a shared or client machine can break
# unrelated work (kubectl more than one minor away from a cluster is the common
# one). Say so; never decide it for them.
# Downloading a verified binary proves it is the right file, not that this
# machine can run it. On an old distro tilt fails here, with a linker error
# about a missing symbol, and finding that out now beats finding out during a
# first cluster build.
# A verified download proves the right file, not that this machine can run it
# (old glibc breaks tilt). Run each one now.
verify_tools() {
local tier="${1:-dev}" b bin out rc broke=0
echo "checking that each one actually runs"
@@ -681,11 +580,7 @@ verify_tools() {
printf ' %-14s not installed\n' "$b"
continue
fi
# Not piped into `head`. With `pipefail` set, a tool that prints more
# than one line gets SIGPIPE when head closes the pipe, and the
# pipeline reports 141 — so a working kubectl was announced as "does
# not run here", with its own correct version string as the evidence.
# Take the first line afterwards, from the string.
# Not piped into `head`: under pipefail, SIGPIPE (141) looked like failure.
rc=0
case "$b" in
kubectl) out=$("$bin" version --client 2>&1) || rc=$? ;;
@@ -763,18 +658,12 @@ warn_shadowing() {
OUT_BIN=\$PWD/def/bin make deps # then put that dir first in PATH")
}
# A copy in OUT_BIN only gives you `docker-compose`. That hyphenated form is the
# retired v1 spelling; every compose file written in the last few years assumes
# `docker compose`, which resolves plugins BY NAME out of a plugin directory.
# So the binary is fetched like any other and then linked, in your own home —
# no root, and nothing outside it.
# Link the fetched docker-compose into ~/.docker/cli-plugins so `docker compose` works.
install_compose_plugin() {
local src="$OUT_BIN/docker-compose" dir="$HOME/.docker/cli-plugins"
[ -x "$src" ] || return 0
mkdir -p "$dir"
# Something else already owns that name — docker-desktop and some distro
# packages install a real file there. Overwriting it would take the plugin
# away from whatever put it there, so say so and let the user decide.
# A real file there belongs to something else (docker-desktop, distro): don't overwrite.
if [ -e "$dir/docker-compose" ] && [ ! -L "$dir/docker-compose" ]; then
MANUAL+=("Something already installs the compose plugin at
$dir/docker-compose
@@ -805,16 +694,12 @@ install() {
if [ "$tier" = "core" ]; then
echo " (no kind/tilt — 'make deps dev' adds them)"
fi
# Only when compose was one of the things fetched: linking a binary
# that is already satisfied elsewhere on PATH would point the plugin at
# a copy rig did not install.
# Only when compose was fetched, never at a copy rig did not install.
case " $TOOLCHAIN_NEED " in
*" docker-compose "*) install_compose_plugin ;;
esac
# Only worth saying when something actually landed in OUT_BIN. When every
# tool was satisfied elsewhere, OUT_BIN may reasonably be off PATH, and
# telling the user to add it would be advice to fix nothing.
# PATH advice only when something actually landed in OUT_BIN.
case ":${PATH}:" in
*":$OUT_BIN:"*) ;;
*) MANUAL+=("Put the toolchain on your PATH — add to ~/.bashrc:
@@ -830,10 +715,7 @@ install() {
require_linux
# Read the command, THEN shift — and shift only if there is something there.
# A bare `shift` with no positional parameters returns 1, and under `set -e`
# that ended the script before a single line was printed: running this with no
# arguments at all, the documented default, did nothing and said nothing.
# Shift only if there is an argument: a bare `shift` returns 1 under set -e.
cmd="${1:-install}"
[ $# -gt 0 ] && shift
@@ -845,12 +727,12 @@ need_downloads() {
}
case "$cmd" in
detect) detect; report_manual ;;
detect) if [ "${1:-}" = all ]; then VERBOSE=1; fi; detect; report_manual ;;
list) list ;;
verify) verify_tools "${1:-dev}" ;;
fetch) need_downloads; fetch "$@" ;;
install) need_downloads; install "${1:-dev}" ;;
*) echo "usage: $0 [detect|list|verify|fetch|install]" >&2
*) echo "usage: $0 [detect [all]|list|verify|fetch|install]" >&2
echo " install [core|dev] (default dev)" >&2
echo " fetch [core|dev] [--to DIR]" >&2
echo " OUT_BIN=<dir> overrides the install directory" >&2

13
rig/ctrl/docs.sh
View File

@@ -1,16 +1,7 @@
#!/usr/bin/env bash
# Documentation: render the diagrams, and serve the pages.
#
# The docs are the instructions for building the cluster, so they must work
# BEFORE anything else exists. That rules out serving them from the cluster, and
# it rules out python -m http.server too — a minimal Debian has no python3. What
# it does have, by definition, is Docker: the single prerequisite rig already
# demands. So a throwaway nginx container serves a read-only bind mount.
#
# Rendered SVGs are committed alongside their .dot sources for the same reason:
# the pages have to read on a machine with no Graphviz installed.
#
# Documentation: render the diagrams, and serve the pages from a throwaway nginx container.
# Usage: docs.sh serve | graphs
# Notes: docs/notes/docs.md
set -euo pipefail
cd "$(dirname "$0")"

20
rig/ctrl/env.d/client.env.example
View File

@@ -1,10 +1,6 @@
# EXAMPLE PROFILE. rig needs none of these: with no profile it runs on its
# built-in defaults (lib/config.sh). To use this one, copy it to client.env in this
# directory and name it — PROFILE=client in ctrl/.env, or on the command line. It
# then overlays the defaults; anything it does not set, they still supply.
#
# client — images through a pull-through cache of the corporate registry, with
# TLS and metrics addons. More nodes or port mappings: edit k8s/kind-config.yaml.tpl.
# EXAMPLE PROFILE (optional): copy to client.env, then PROFILE=client; overlays the defaults.
# client — images via a pull-through cache of the corporate registry, TLS and metrics addons.
# Notes: docs/notes/env.md
PROFILE_NAME=client
K8S_VERSION=v1_36
@@ -13,14 +9,8 @@ REGISTRY_MODE=mirror
INGRESS_MODE=hostport
DNS_MODE=hosts
# Ports derive from the directory name by default (see ctrl/ports.sh), so
# several environments run side by side.
#
# Opt in to the real ports below only when this is the ONLY environment and
# nothing else owns :80. They fail to bind otherwise, and docker reports it as an
# opaque "failed to bind host port 0.0.0.0:80/tcp: address already in use"
# halfway through cluster creation. `make check` checks before you spend the
# time. Uncommenting also means only one environment can exist at a time.
# Ports derive from the directory name by default (see ctrl/ports.sh).
# Real ports only if this is the ONLY environment and nothing owns :80; `make check` tests it.
# HTTP_PORT=80
# HTTPS_PORT=443

24
rig/ctrl/env.d/data.env.example
View File

@@ -1,16 +1,6 @@
# EXAMPLE PROFILE. rig needs none of these: with no profile it runs on its
# built-in defaults (lib/config.sh). To use this one, copy it to data.env in this
# directory and name it — PROFILE=data in ctrl/.env, or on the command line. It
# then overlays the defaults; anything it does not set, they still supply.
#
# data — databases and a scheduler for an environment that needs them: postgres,
# redis and airflow, each an upstream image run unmodified.
#
# Everything lands in the `data` namespace (DATA_NAMESPACE to move it), so
# `make cluster reset` on the app namespace leaves the databases alone.
#
# Costs roughly 2-3 GB with airflow, under 1 without. Airflow's first boot runs
# the whole metadata migration, so expect a few minutes before it is ready.
# EXAMPLE PROFILE (optional): copy to data.env, then PROFILE=data; overlays the defaults.
# data — postgres, redis and airflow (upstream images) in the `data` namespace.
# Notes: docs/notes/env.md
PROFILE_NAME=data
K8S_VERSION=v1_36
@@ -25,16 +15,12 @@ DNS_MODE=hosts
# Namespace for the dependency containers.
DATA_NAMESPACE=data
# Postgres identity. The password is not here: postgres.sh generates one on
# first install and keeps it across re-runs, so re-running the addon never
# rotates the credential out from under whatever is already connected.
# Postgres identity. The password is generated once by postgres.sh and kept.
POSTGRES_DB=app
POSTGRES_USER=app
POSTGRES_STORAGE=2Gi
AIRFLOW_ADMIN_USER=admin
# Ports derive from the directory name by default — see ctrl/ports.sh. Reach
# the databases with port-forward rather than binding more host ports:
# Reach the databases with port-forward, e.g.
# kubectl -n data port-forward svc/postgres 5432:5432
# kubectl -n data port-forward svc/airflow 8080:8080

13
rig/ctrl/env.d/offline.env.example
View File

@@ -1,13 +1,6 @@
# EXAMPLE PROFILE. rig needs none of these: with no profile it runs on its
# built-in defaults (lib/config.sh). To use this one, copy it to offline.env in this
# directory and name it — PROFILE=offline in ctrl/.env, or on the command line. It
# then overlays the defaults; anything it does not set, they still supply.
#
# offline — air-gapped. Everything comes from a local registry that was loaded
# ahead of time; nothing reaches the internet. Pair with the deps-full image
# (DEPS_SOURCE=baked) so the toolchain install is offline too.
#
# The heavier addons are left out to keep first boot viable.
# EXAMPLE PROFILE (optional): copy to offline.env, then PROFILE=offline; overlays the defaults.
# offline — air-gapped: images from a preloaded local registry; pair with DEPS_SOURCE=baked.
# Notes: docs/notes/env.md
PROFILE_NAME=offline
K8S_VERSION=v1_36

26
rig/ctrl/k8s/kind-config.yaml.tpl
View File

@@ -1,24 +1,12 @@
# The cluster. One node by default — add nodes or port mappings by editing this
# file, then `make cluster reset`.
#
# A TEMPLATE rather than a plain kind-config.yaml because a rig is copied and
# renamed to make a second environment, and both the cluster name and the host
# port follow the directory. A checked-in literal would make every copy collide
# on both. ctrl/cluster.sh renders it with sed — not envsubst, which is
# gettext-base and absent from a minimal Debian, and rig's whole premise is that
# Docker is the only prerequisite.
#
# Substituted by ctrl/cluster.sh: CLUSTER, NODE_IMAGE, HTTP_PORT, HOST_WORKDIR
# (named without the ${...} braces so this line survives the substitution)
# The node count is READ BACK from this file by lib/config.sh, so this YAML is
# the source of truth for it — there is no second place to update.
# The cluster. Add nodes or port mappings here, then `make cluster reset`.
# ctrl/cluster.sh substitutes (sed): CLUSTER, NODE_IMAGE, HTTP_PORT, HOST_WORKDIR
# lib/config.sh reads the node count back from this file.
# Notes: docs/notes/kind-config.md
kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
name: ${CLUSTER}
# Point containerd at a certs.d directory. registry.sh drops per-host hosts.toml
# files in there afterwards, so switching registry mode never requires
# recreating the cluster.
# containerd reads per-host registry config from certs.d (written by registry.sh).
containerdConfigPatches:
- |-
[plugins."io.containerd.grpc.v1.cri".registry]
@@ -27,9 +15,7 @@ containerdConfigPatches:
nodes:
- role: control-plane
image: ${NODE_IMAGE}
# One NodePort bridged to the host; an in-cluster gateway owns it. There is
# deliberately no ingress controller — they pin a narrow window of k8s
# versions, and running a trailing-edge control plane is the point.
# One NodePort bridged to the host, owned by an in-cluster gateway (no ingress controller).
extraPortMappings:
- containerPort: 30080
hostPort: ${HTTP_PORT}

130
rig/ctrl/lib/config.sh
View File

@@ -1,38 +1,16 @@
# Shared config loading. Sourced, never executed.
#
# The ecosystem convention is that scripts are standalone with no shared log
# library — that still holds. This file is not a logging lib; it is the single
# definition of how the config layers compose, which every script has to agree
# on exactly. Precedence, weakest first:
#
# built-in defaults below; fill only what nothing else set
# ctrl/versions.env pinned toolchain + image digests (committed)
# ctrl/env.d/<profile> addons, registry — OPTIONAL, examples ship as *.env.example
# ctrl/.env machine-local values and secrets (gitignored)
# the caller's env `make cluster up PROFILE=<name>` (always wins)
#
# That last rule is why this is more than a few `source` lines: .env sets
# PROFILE, so without snapshotting it would silently override the PROFILE the
# user just typed on the command line.
#
# Shared config loading: how the config layers compose. Sourced, never executed.
# Precedence, weakest first: defaults < versions.env < env.d/<profile> < .env < caller's env.
# Run from ctrl/.
# Notes: docs/notes/config.md
# Values a user can reasonably override per-invocation. Anything set in the
# environment when load_config runs is restored after the files are read.
# NODES is deliberately NOT here: it is read back out of the kind config below,
# so the file is the one place that decides it.
#
# REGISTRY_PORT and MANIFESTS_DIR were missing here while ctrl/.env set them, so
# the caller's env silently LOST to the file for those two — the one precedence
# rule this header states. Both are now listed; the other twelve are unchanged.
# Per-invocation overrides: restored after the files are read, so the caller wins.
# NODES is deliberately not here (read from the kind config).
CONFIG_OVERRIDABLE="PROFILE CLUSTER K8S_VERSION KIND_CONFIG ADDONS
REGISTRY_MODE INGRESS_MODE DNS_MODE TILT_PORT
SOURCE ARCH DEPS_SOURCE HTTP_PORT HTTPS_PORT
REGISTRY_PORT MANIFESTS_DIR"
# The containing folder's name, reduced to something kind accepts as a cluster
# name (a DNS label: lowercase alphanumerics and dashes). Run from ctrl/, so the
# repo root is the parent.
# The repo folder's name, reduced to a DNS label kind accepts as a cluster name.
default_cluster_name() {
local n
n=$(basename "$(cd .. && pwd)")
@@ -41,9 +19,7 @@ default_cluster_name() {
echo "${n:-rig}"
}
# Base of this environment's 10-port block. cksum is used rather than $RANDOM or
# bash hashing because it is POSIX and returns the same value on every machine,
# which is what makes the block reproducible instead of merely unique.
# Base of this environment's 10-port block; cksum so it is the same on every machine.
derive_port_base() {
local h; h=$(printf '%s' "$1" | cksum | awk '{print $1}')
echo $((20000 + (h % 200) * 10))
@@ -61,19 +37,14 @@ load_config() {
set -a
source ./versions.env
# RIG_PORTABLE skips the machine-local layer. config_snapshot sets it, so a
# generated standalone kit never carries this machine's .env — which holds
# local values and, by its own description, secrets.
# RIG_PORTABLE skips the machine-local .env (set by config_snapshot for kits).
if [ -z "${RIG_PORTABLE:-}" ] && [ -f ./.env ]; then source ./.env; fi
set +a
# Re-apply overrides now so PROFILE is the caller's before we pick the file.
_config_restore "$saved"
# A profile is an optional overlay, never a prerequisite. rig assumes no
# configuration: with no profile named — or no env.d/ at all — it runs on the
# built-in defaults below. What IS an error is naming a profile that does not
# exist, because a typo must not quietly fall back to something else.
# A profile is optional; naming one that does not exist is an error.
local profile="${PROFILE:-}"
if [ -n "$profile" ] && [ "$profile" != default ]; then
if [ ! -f "./env.d/${profile}.env" ]; then
@@ -104,27 +75,19 @@ load_config() {
K8S_VERSION="${K8S_VERSION#NODE_IMAGE_}"
fi
# Identity follows the FOLDER, so copying this directory somewhere else and
# renaming it yields a distinct environment with no further edits. Without
# this, two copies would share one cluster and `make cluster down` in either
# would destroy the other's.
# Identity follows the folder, so a renamed copy is a distinct environment.
CLUSTER="${CLUSTER:-$(default_cluster_name)}"
KUBECONTEXT="kind-${CLUSTER}"
# Host ports are a single shared namespace, so unlike the cluster name they
# cannot just follow the directory — they have to be spread out. Anything
# already set (ctrl/.env, a profile, the command line) wins; only the gaps
# are filled. See ports.sh for the reasoning.
# Host ports: fill only the gaps from the derived block; anything already set wins.
local base; base=$(derive_port_base "$CLUSTER")
HTTP_PORT="${HTTP_PORT:-$base}"
HTTPS_PORT="${HTTPS_PORT:-$((base + 1))}"
TILT_PORT="${TILT_PORT:-$((base + 2))}"
REGISTRY_PORT="${REGISTRY_PORT:-$((base + 3))}"
# Where the workload's manifests live, repo-root relative. Defaulted here so
# it is always resolved rather than sometimes-set: it is the seam that lets
# the real manifests be versioned away from the installer, and a consumer
# should not have to know whether anyone filled it in. See k8s/README.md.
# Where the workload's manifests live, repo-root relative; always resolved.
# See k8s/README.md.
MANIFESTS_DIR="${MANIFESTS_DIR:-ctrl/k8s/overlays/dev}"
# Profiles name a k8s minor (v1_36); versions.env holds the pinned digest.
@@ -148,25 +111,13 @@ load_config() {
# check.sh and the memory tool size their budget on NODES.
NODES=$(grep -c '^ - role:' "$KIND_CONFIG")
# What one node costs, measured rather than guessed. On 2026-09-11 a minimal
# control-plane node ran at 620 MiB idle and ~728 MiB with a small mock, plus
# 16 MiB for the local registry — ~745 MiB of working set. 800 rounds that up,
# and agrees with the 800 MB observed independently on a larger rig. Worker
# nodes carry no etcd or apiserver and are lighter, so for a multi-node shape
# this errs high. It is the cluster alone: whatever you deploy comes on top.
#
# Here rather than in check.sh because the memory tool and every standalone
# kit need the same figure.
# Measured MB per node (cluster alone, errs high for workers); shared by
# check.sh, the memory tool and standalone kits.
NODE_MB=800
}
# Render the kind config to stdout. sed rather than envsubst: envsubst is
# gettext-base, absent from a minimal Debian, and Docker is meant to be the only
# prerequisite. The variable list is explicit so a template cannot quietly start
# depending on something the caller does not set.
#
# hostPath entries are resolved by the HOST dockerd, so HOST_WORKDIR must stay a
# host path even when this runs inside the installer container.
# Render the kind config to stdout with sed (not envsubst) over an explicit variable list.
# HOST_WORKDIR must be a host path: the host dockerd resolves hostPath entries.
render_kind_config() {
local host_workdir="${HOST_WORKDIR:-$(cd .. && pwd)}"
sed -e "s|\${CLUSTER}|${CLUSTER}|g" \
@@ -189,14 +140,10 @@ _config_restore() {
}
# ── what a standalone kit needs to know ────────────────────────────────────
# Two questions the kit generator (ctrl/standalone.sh) asks, so that it never has
# to know how configuration is stored. Where profiles live, which files are
# layered and what is derived are this file's business and can change freely;
# the generator only calls these.
# The questions ctrl/standalone.sh asks, so it never knows how config is stored.
# Every configuration rig can be run as, one per line: each profile file, or —
# when there are none — `default`, the built-in configuration load_config uses
# when no profile is named. Never empty, because rig never needs a profile.
# Every configuration rig can run as, one per line: each profile, or `default`
# when there are none. Never empty.
config_profiles() {
local f found=""
for f in ./env.d/*.env; do
@@ -206,20 +153,8 @@ config_profiles() {
[ -n "$found" ] || echo default
}
# The resolved configuration, as `declare -p` lines — exactly what load_config
# leaves behind, minus the machine-local layer. A kit freezes this in place of
# load_config, so it carries rig's decisions and not this machine's secrets.
#
# config_snapshot <profile> that profile, as any machine would resolve it
# config_snapshot --current what THIS machine runs: every overridable key as
# resolved here, handed back in as if typed on the
# command line, over the same portable resolution.
# Values derived from those choices follow them;
# anything else the local layer set — credentials —
# is not carried. config_left_out names it.
#
# Found by difference, not by a list: whatever load_config sets today, it sets.
# A list here would be one more place to forget a variable.
# What load_config sets, minus the machine-local layer, as `declare -p` lines.
# Usage: config_snapshot <profile> | --current (found by difference, not a list)
config_snapshot() {
local _rig_snap_choices
if [ "$1" = --current ]; then
@@ -257,10 +192,7 @@ config_snapshot() {
# The profile this machine runs, as load_config resolves it here.
config_current_profile() { ( load_config >/dev/null && echo "$PROFILE_NAME" ); }
# What an export of this machine's configuration does NOT carry, by name only:
# keys the machine-local layer sets that are not choices a caller may override.
# They are this machine's own — registry and mirror credentials, mostly — so the
# target has to be told to supply them. Values are never printed.
# Names (never values) of .env keys an export does not carry, e.g. credentials.
config_left_out() {
[ -f ./.env ] || return 0
local k
@@ -272,20 +204,8 @@ config_left_out() {
done
}
# A replacement for load_config with a resolution frozen in (a profile, or
# --current — see config_snapshot), printed
# as a function definition for a standalone kit to carry. The generator embeds
# whatever this prints and interprets none of it, so what "frozen" means stays
# rig's decision.
#
# It keeps load_config's one stated rule: the caller's env wins for anything in
# CONFIG_OVERRIDABLE. A kit therefore behaves like rig — `OUT_BIN=... rigdeps.sh`
# still works — rather than like a copy with everything pinned.
#
# What freezing does give up, knowingly: values DERIVED from an overridable one
# are fixed at generation. Override CLUSTER and the ports stay the ones derived
# for the original name. Re-deriving would mean carrying the layering itself,
# which is exactly what a kit exists not to need.
# Print a load_config with a resolution frozen in, for a standalone kit to carry.
# The caller's env still wins; derived values (e.g. ports) stay fixed.
config_freeze() {
local snap
snap=$(config_snapshot "$1") || return 1

121
rig/ctrl/mem.sh
View File

@@ -1,32 +1,8 @@
#!/usr/bin/env bash
# rig:standalone rigmini status
# How much memory this machine will actually give you before something dies —
# rig's memory tool, and (generated from this file) the standalone rigmini.sh.
#
# There are two numbers and they are rarely the same. `status` reports what the
# machine ADVERTISES and what is quietly capping it. `push` finds what it will
# SURVIVE, by allocating until it stops. `all` does both and weighs the result
# against what this profile's cluster needs.
#
# The gap between them is the whole reason this exists. Under WSL the cap lives
# in .wslconfig; in a container or a managed workspace it is a cgroup limit, and
# there /proc/meminfo reports the HOST's memory while the kernel kills you at a
# fraction of it. A script that only read MemTotal would confidently report 32 GB
# on a box that OOMs at 2.
#
# Runs on native Linux and under WSL. On WSL the memory you see is a VM
# allocation that can be raised, and the commonest failure is raising it without
# restarting — so status compares what .wslconfig says with what actually booted.
#
# Reports and instructs. It never raises a limit, frees anything or installs a
# package. The one write it can make is `backup`, which copies .wslconfig beside
# itself, so that `restore` has something to put back after a hand edit.
#
# Usage:
# mem.sh status what it has, what caps it
# mem.sh push [--to GB] [--to-oom] climb until it stops
# mem.sh all [--budget GB] both, then the verdict
# mem.sh backup | restore .wslconfig, WSL only
# rig's memory tool (also generated as rigmini.sh): what the machine advertises vs. what it survives.
# Usage: mem.sh status | push [--to GB] [--to-oom] | all [--budget GB] | backup | restore (WSL)
# Notes: docs/notes/mem.md
set -euo pipefail
cd "$(dirname "$0")"
source ./lib/config.sh
@@ -42,9 +18,7 @@ BUDGET_EXPLICIT=no # whether --budget was given, which retires the guess belo
# ── platform ───────────────────────────────────────────────────────────────
# Windows outside WSL — Git Bash, MSYS, Cygwin — looks close enough to work and
# then fails in a pile of confusing ways: no /proc, no docker socket, none of
# the tooling. Detectable, so name it instead.
# Refuse Git Bash / MSYS / Cygwin and kernels without /proc, with a clear message.
require_linux() {
case "$(uname -s)" in
MINGW*|MSYS*|CYGWIN*)
@@ -99,9 +73,7 @@ avail_meminfo_mb() {
fi
}
# Where a cgroup records this cgroup's own limit and usage. Set once by
# find_cgroup, because every later reading needs both and hunting for the files
# on each call would be the slow part of the poll loop.
# This cgroup's limit/usage files, set once by find_cgroup (cheap for the poll loop).
CG_MAX_FILE=""
CG_CUR_FILE=""
CG_VERSION=""
@@ -109,10 +81,8 @@ CG_VERSION=""
find_cgroup() {
local rel
# Inside a container the cgroup namespace makes the top of the tree BE the
# container's own cgroup, so the unqualified path is already the right one.
# On a host it is the root cgroup, which is never limited — hence the second
# attempt via /proc/self/cgroup, which names the slice this shell is in.
# Top of tree first (right inside a container), then this shell's own slice
# from /proc/self/cgroup (right on a host).
if [ -r /sys/fs/cgroup/memory.max ]; then
CG_VERSION=v2
CG_MAX_FILE=/sys/fs/cgroup/memory.max
@@ -141,10 +111,7 @@ find_cgroup() {
return 0
}
# The cap in MB, or "" when there is none worth reporting. v2 spells unlimited
# "max"; v1 spells it as a number near 2^63, which is why this compares against
# MemTotal rather than testing for a magic value — a "limit" above the machine's
# own memory is not a limit, however it is written.
# The cap in MB, or "" when unlimited ("max", or any value >= MemTotal).
cgroup_cap_mb() {
local raw cap
[ -n "$CG_MAX_FILE" ] && [ -r "$CG_MAX_FILE" ] || { echo ""; return 0; }
@@ -185,10 +152,7 @@ effective_ceiling_mb() {
echo "$c"
}
# How much room is left RIGHT NOW, from whichever accounting actually governs.
# In a capped container /proc/meminfo describes the host and is worse than
# useless for this — it would report tens of gigabytes free on a box that is one
# allocation from being killed.
# Room left right now: cgroup cap minus usage when capped, else MemAvailable.
headroom_mb() {
local cap used
cap=$(cgroup_cap_mb)
@@ -202,9 +166,7 @@ headroom_mb() {
# ── status ─────────────────────────────────────────────────────────────────
# /mnt/c/Users can hold several real accounts — a renamed login leaves the old
# directory behind — so picking the first alphabetically is a coin toss. Ask
# Windows, then fall back to whichever profile actually owns a config.
# Ask Windows for %USERPROFILE%; fall back to whichever profile owns a .wslconfig.
wslconfig_path() {
local profile winpath found
profile=$(cmd.exe /c "echo %USERPROFILE%" 2>/dev/null | tr -d "\r\n" || true)
@@ -260,9 +222,7 @@ status() {
echo " ulimit -v unlimited"
fi
# overcommit_memory=0 is the default heuristic: a large allocation is
# granted on a guess, and the reckoning arrives later as an OOM kill rather
# than as a failed malloc. It is why `push` touches every page it asks for.
# Overcommit mode decides whether limits show as failed mallocs or OOM kills.
local om or_
om=$(cat /proc/sys/vm/overcommit_memory 2>/dev/null || echo '?')
or_=$(cat /proc/sys/vm/overcommit_ratio 2>/dev/null || echo '?')
@@ -335,9 +295,7 @@ status() {
echo " ! docker cli present but the daemon is unreachable"
fi
# WSL keeps its cap on the Windows side, in a file this shell can read but
# not usefully apply — the change costs a full VM restart. Report it, and
# report the commonest mistake, which is editing it and not restarting.
# WSL: report the .wslconfig cap and whether it was applied (needs wsl --shutdown).
if is_wsl; then
local cfg conf conf_mb n
cfg=$(wslconfig_path)
@@ -397,7 +355,7 @@ require_wsl() {
# backup and restore act on the file, so unlike status they must not guess.
wslconfig_required() {
local cfg; cfg=$(wslconfig_required)
local cfg; cfg=$(wslconfig_path)
if [ -z "$cfg" ]; then
echo "cannot tell which Windows profile owns .wslconfig. Candidates:" >&2
ls -d /mnt/c/Users/*/ 2>/dev/null \
@@ -429,8 +387,7 @@ backup() {
local cfg dest
cfg=$(wslconfig_required)
[ -r "$cfg" ] || { echo "nothing to back up: $cfg does not exist" >&2; exit 1; }
# Timestamped and never overwritten: a backup that can destroy itself on a
# second run is not a backup.
# Timestamped, never overwritten.
dest="${cfg}.$(date +%Y%m%d-%H%M%S).bak"
cp "$cfg" "$dest"
echo "backed up $dest"
@@ -449,9 +406,7 @@ restore() {
echo " -> $cfg"
echo
# Newest is the right default — undo the last edit — but if you backed up
# *after* editing, the state you want is older. Show the rest so a no-op
# restore is obviously a no-op rather than a mystery.
# Restores the newest; list the others in case an older one is wanted.
count=$(ls "$cfg".*.bak 2>/dev/null | wc -l)
if [ "$count" -gt 1 ]; then
echo "$count backups exist, newest first:"
@@ -495,14 +450,9 @@ cleanup() {
return 0
}
# The child allocates and stops itself; the parent only watches. That split is
# the point: under --to-oom the allocating process is expected to be killed, and
# something has to survive to say how far it got.
# Runs as a child that may be OOM-killed; the parent survives to report.
allocator() {
# Raise our own OOM score to the maximum so the kernel picks THIS process
# first. Raising needs no privilege (only lowering does). Without it, the
# kernel is free to choose your shell, your ssh session or dockerd — on a
# box you are still using, that is not an acceptable coin toss.
# Make this process the preferred OOM victim (raising needs no privilege).
echo 1000 > "/proc/$BASHPID/oom_score_adj" 2>/dev/null || true
local arr=() held=0 i=0 rss swapped avail first_swap=0
@@ -511,16 +461,7 @@ allocator() {
swap_used_start=$(( $(mb SwapTotal) - $(mb SwapFree) ))
while :; do
# Written STRAIGHT INTO the array element. The obvious spelling —
# build one chunk and `arr+=("$chunk")` — costs three copies per step,
# not one: the template stays resident, expanding "$chunk" makes a
# temporary word, and the append makes the element. A 128 MB step then
# needs 384 MB transiently, and on a small box it is killed on the
# first append while reporting a third of the true ceiling.
#
# printf -v into a subscript also means every page is written, so it is
# resident rather than merely promised — the only kind of allocation
# that measures anything under heuristic overcommit.
# Write straight into the element (one copy, not three) and touch every page.
printf -v "arr[$i]" '%*s' "$bytes" ''
i=$((i + 1)); held=$((held + STEP_MB))
@@ -533,9 +474,7 @@ allocator() {
"$held" "$rss" "$avail" "$swapped"
printf '%s %s %s %s\n' "$held" "$rss" "$avail" "$swapped" >> "$STATE"
# Worth calling out separately from the ceiling: this is where the box
# stops being fast and starts being unusable, which for a scheduler is
# a different and earlier problem than being killed.
# First swap is reported separately: slow comes before killed.
if [ "$swapped" -gt 0 ] && [ "$first_swap" -eq 0 ]; then
first_swap=$held
echo " - first swap page at ${held} MB — past here it works but crawls"
@@ -556,30 +495,20 @@ push() {
total=$(mb MemTotal)
ceiling=$(effective_ceiling_mb)
# A step is worth about a sixty-fourth of the ceiling: enough resolution to
# find the edge, few enough lines to read, and small enough that the
# transient cost of one allocation never dominates a small box. A fixed
# size cannot do all three — 128 MB is fine on 16 GB and absurd on 512 MB.
# Default step: ceiling/64, clamped to 4..256 MB.
if [ "$STEP_EXPLICIT" = no ]; then
STEP_MB=$(( ceiling / 64 ))
[ "$STEP_MB" -lt 4 ] && STEP_MB=4
[ "$STEP_MB" -gt 256 ] && STEP_MB=256
fi
# Stop with a cushion rather than riding it to the kill. How big a cushion
# depends on what it is protecting. Under a cgroup cap, running out kills
# only this container's own processes, so it need cover no more than the
# shell that prints the result — and a 512 MB cushion on a 1 GB box would
# halve the answer. On a host there is everything else to protect, and the
# OOM killer does not promise to pick the process that caused the problem.
# Stop with a cushion: 64 MB under a cgroup cap, 512 MB on a host, or 5% of ceiling if larger.
if [ -n "$(cgroup_cap_mb)" ]; then FLOOR_MB=64; else FLOOR_MB=512; fi
[ $(( ceiling / 20 )) -gt "$FLOOR_MB" ] && FLOOR_MB=$(( ceiling / 20 ))
STATE=$(mktemp "${TMPDIR:-/tmp}/rigmini.XXXXXX")
trap cleanup EXIT
# INT kills the child and lets the summary below print anyway, so an
# impatient Ctrl-C still tells you how far it got — and, more importantly,
# still gives the memory back.
# Ctrl-C kills the child, frees the memory, and still prints the summary.
trap 'echo; echo " interrupted"; echo "stop interrupted" >> "$STATE"; [ -n "$CHILD" ] && kill -KILL "$CHILD" 2>/dev/null || true' INT
echo "push"
@@ -652,11 +581,7 @@ push() {
fi ;;
esac
# The gap between the claim and the measurement is the finding — but only
# when the BOX chose where to stop. An empty $stop means the child was ended
# rather than deciding to end; anything else (--to, the floor) is a stop we
# asked for, and flagging those as short of the ceiling would put a warning
# on every deliberately small run.
# Warn about claimed-vs-measured gap only when the box, not us, chose the stop.
local got="${rss:-$held}"
echo
if [ -z "$stop" ] && [ "$got" -lt $(( ceiling * 70 / 100 )) ]; then

46
rig/ctrl/ports.sh
View File

@@ -1,25 +1,8 @@
#!/usr/bin/env bash
# Give each environment its own block of host ports.
#
# New versions of a system mean new clusters on ONE machine, not new machines.
# Cluster name, kubectl context, registry container and image tag already derive
# from the directory name, so two copies never collide there — but host ports are
# a single shared namespace and would.
#
# The block is derived from the directory name: stateless, stable, and requiring
# no coordination between copies that know nothing about each other.
#
# base = 20000 + (hash(slug) % 200) * 10
# +0 HTTP +1 HTTPS +2 TILT +3 REGISTRY (+4..9 reserved)
#
# 20000+ deliberately avoids the ports something is already likely to hold: 80,
# 443, 3000, 5432, 8000, 8080.
#
# Derivation is a default, not a decision. On first use the resolved block is
# written into ctrl/.env, so it becomes pinned, visible and editable rather than
# a number that appears from nowhere. Anything already in ctrl/.env wins.
#
# Give each environment its own block of host ports, derived from the directory name.
# base = 20000 + (hash(slug) % 200) * 10; +0 HTTP +1 HTTPS +2 TILT +3 REGISTRY
# Usage: ports.sh show | active | derive | persist
# Notes: docs/notes/ports.md
set -euo pipefail
cd "$(dirname "$0")"
@@ -38,28 +21,9 @@ derive() {
DERIVED_REGISTRY=$((base + 3))
}
# The resolved facts a consumer outside bash needs, machine-readable:
#
# Resolved facts for consumers outside bash, space-separated, positional:
# CLUSTER KUBECONTEXT HTTP HTTPS TILT REGISTRY MANIFESTS_DIR
#
# Identity and ports together, because they are one fact set — the header above
# says so: both derive from the directory name so that copies never collide. A
# consumer needs all of them or none, and fetching them separately is how two
# end up disagreeing. MANIFESTS_DIR rides along because the one consumer that
# needs the addressing is the one that needs to know what to deploy.
#
# Space-separated, so MANIFESTS_DIR must not contain spaces. Everything else in
# rig already assumes that of paths — kind, docker and kubectl all do.
#
# `derive` answers a DIFFERENT question — what the directory name alone implies
# — and deliberately ignores ctrl/.env. Configuring anything from it would
# silently contradict this file's own rule that "anything already in ctrl/.env
# wins". `active` is what anything downstream should read.
#
# Why this exists at all: the cluster name is not the bare directory name.
# default_cluster_name() lowercases it and replaces every character outside
# [a-z0-9-], because it has to be a DNS label. Re-deriving that in another
# language is how a copy in `My_Project/` ends up guarding the wrong context.
# Read this, not `derive` (which ignores ctrl/.env).
active() {
load_config
echo "$CLUSTER $KUBECONTEXT $HTTP_PORT $HTTPS_PORT $TILT_PORT $REGISTRY_PORT $MANIFESTS_DIR"

38
rig/ctrl/registry.sh
View File

@@ -1,28 +1,8 @@
#!/usr/bin/env bash
# Registry plumbing. THIS is the seam — not a tool.
#
# Four modes, selected by REGISTRY_MODE in the active profile:
#
# none Tilt builds straight into the node. No registry at all — and so no
# guard against an outward push: an unqualified image name means
# docker.io/library/<name>, and only Tilt's kind detection stands
# between that and a real push. Throwaway use only; every profile
# here now defaults to `local` instead.
# local a registry:2 container wired into the cluster.
# mirror the same container, but configured as a pull-through CACHE of the
# corporate registry. What a locked-down client actually looks like:
# images originate from corp, you don't hammer it, and you keep
# working when the VPN drops.
# remote no local container; pull straight from the corporate registry using
# an imagePullSecret.
#
# Deliberately a script rather than a tool. ctlptl collapses the `local` wiring
# into one line, but its Registry spec only accepts name/port/image/listenAddress
# — there is no way to set REGISTRY_PROXY_REMOTEURL, so it cannot express
# `mirror` at all. Keeping the seam here is what keeps the corporate registry
# swappable.
#
# Registry plumbing: REGISTRY_MODE none | local | mirror | remote. A script, not ctlptl,
# because ctlptl cannot express `mirror`.
# Usage: registry.sh up | down | status
# Notes: docs/notes/registry.md
set -euo pipefail
cd "$(dirname "$0")"
@@ -34,17 +14,7 @@ REG_PORT="${REGISTRY_PORT:-5005}"
K="kubectl --context ${KUBECONTEXT}"
# ── CA trust ───────────────────────────────────────────────────────────────
# A corporate registry is almost always fronted by an internal CA, and trust has
# to reach three separate places. Nothing does this for you, and the symptom when
# it's missing is an opaque:
# x509: certificate signed by unknown authority
#
# 1. the host docker daemon — /etc/docker/certs.d/<host>/ca.crt (needs root)
# 2. every kind node's containerd — nodes do NOT inherit host trust
# 3. anything doing HTTPS from inside the cluster, in its own trust store
#
# We handle (2) here because it's ours to handle. (1) is reported by check.sh
# since it needs root. (3) belongs to the workload.
# Copy REGISTRY_CA_FILE into every kind node's trust store (nodes don't inherit host trust).
install_ca_into_nodes() {
[ -n "${REGISTRY_CA_FILE:-}" ] || return 0

101
rig/ctrl/selftest.sh
View File

@@ -1,20 +1,8 @@
#!/usr/bin/env bash
# What rig has settled, written down as assertions.
#
# These are documentation that runs. Each check is ONE decision that has already
# been made, with the reason above it — not coverage, and deliberately not an
# exhaustive sweep of use cases. rig's own index says a rule without its reason
# gets overridden the first time it is inconvenient; a rule nobody can restate
# is worse. So the test says what was decided, and failing it should read as
# "you are about to undo this" rather than "something broke".
#
# Scope, on purpose:
# - no cluster, no docker, no network. It must be cheap enough to actually run.
# - it asserts about RIG. `make check` asserts about the MACHINE and never
# fails; this exits 1, the way `make standalone check` does.
# - what actually deploys is not testable here. `tilt ci` stays a manual step.
#
# What rig has settled, written down as assertions: one decision per check.
# No cluster, docker or network; exits 1 on failure (unlike `make check`).
# Usage: make selftest (or: bash ctrl/selftest.sh)
# Notes: docs/notes/selftest.md
set -uo pipefail # NOT -e: one failing check must not abort the rest
cd "$(dirname "$0")"
@@ -43,10 +31,7 @@ resolved() {
note "rig needs no profile"
# rig assumes no configuration. A profile is an overlay on built-in defaults, so
# a rig with no env.d/ at all must resolve, report, and still generate a kit —
# and naming a profile that does not exist must still be an error, because a
# typo that silently fell back to the defaults would be worse than a failure.
# No env.d/ must still resolve and generate a kit; an unknown profile stays an error.
NP="$(mktemp -d)"
cp -r .. "$NP/rig"; rm -rf "$NP/rig/ctrl/env.d"; sed -i '/^PROFILE=/d' "$NP/rig/ctrl/.env" 2>/dev/null
check "no env.d: config resolves" "default" \
@@ -63,11 +48,7 @@ rm -rf "$NP"
note "the ports.sh active contract"
# ports.sh active is read POSITIONALLY by two other files — the Makefile takes
# $(word 2) and $(word 5), the Tiltfile takes _facts[0]..[6]. Insert a field in
# the middle and nothing errors: Tilt simply guards on the wrong context or
# binds the wrong port. The field count and order are the contract, so they are
# pinned here rather than left to whoever edits ports.sh next.
# ports.sh active is read positionally by the Makefile and Tiltfile: pin field count and order.
FACTS="$(bash ports.sh active)"
check "active: exactly 7 fields" "7" "$(printf '%s' "$FACTS" | wc -w)"
read -r F_CLUSTER F_CTX F_HTTP F_HTTPS F_TILT F_REG F_MANIFESTS <<< "$FACTS"
@@ -83,16 +64,7 @@ check "derive: still 4 fields, not 7" "4" "$(bash ports.sh derive | wc -w)"
note "the caller's env beats the files"
# lib/config.sh states one precedence rule: versions.env < env.d/<profile> <
# ctrl/.env < the caller's env. It is enforced by CONFIG_OVERRIDABLE, a
# hand-maintained list — and a key missing from it loses to the file SILENTLY.
# REGISTRY_PORT and MANIFESTS_DIR were both missing on 2026-09-13 and were found
# by accident.
#
# So this loop is generated FROM the list: add a key to CONFIG_OVERRIDABLE and
# this test starts asking about it without anyone remembering to come here.
# Three keys name something that must exist and are validated at load, so they
# get a real alternative rather than a sentinel.
# Every key in CONFIG_OVERRIDABLE must lose to the caller's env; the loop follows the list.
test_value() {
case "$1" in
# Picked from what exists, never named: rig must not need any particular
@@ -122,16 +94,8 @@ done
note "one derivation, not three"
# The Makefile used to compute the cluster name itself and sed TILT_PORT out of
# ctrl/.env — a second derivation of values lib/config.sh already owns, which
# could disagree with it after `ports.sh persist`. It now reads ports.sh
# active. Nothing structurally prevents the sed coming back, so the agreement is
# asserted against the real `make -n` output rather than against the source.
# --no-print-directory and a grep, not `tail -1`: run from `make selftest` this
# is a RECURSIVE make, and the "Entering/Leaving directory" lines go to STDOUT.
# tail -1 then reads "make[1]: Leaving directory ..." and both checks below fail
# — but only when invoked through make, never when the script is run directly.
# A test that passes one way and fails the other is worse than no test.
# The Makefile must take context/port from ports.sh active, checked on real `make -n` output.
# --no-print-directory + grep, not tail -1: under `make selftest` this is a recursive make.
MK="$(cd .. && make --no-print-directory -n tilt 2>/dev/null | grep -m1 'tilt ')"
check "Makefile: --context comes from active" "$F_CTX" \
"$(printf '%s' "$MK" | sed -n 's/.*--context \([^ ]*\).*/\1/p')"
@@ -140,10 +104,7 @@ check "Makefile: --port comes from active" "$F_TILT" \
note "identity follows the folder, safely"
# The cluster name is NOT the bare directory name: kind needs a DNS label, so
# default_cluster_name lowercases it and replaces everything outside [a-z0-9-].
# Re-deriving that anywhere else is how a copy ends up guarding the wrong
# context — which is exactly why the Tiltfile asks instead of computing.
# The cluster name is the folder name made a DNS label, derived only in lib/config.sh.
TMP="$(mktemp -d)"
trap 'rm -rf "$TMP"' EXIT
mkdir -p "$TMP/My_Proj"
@@ -159,52 +120,33 @@ check "a renamed copy gets a DIFFERENT block" "different" \
note "ports are stable across versions"
# Not a change-detector. The block is derived, never stored, so if the
# derivation shifts then every EXISTING environment's ports move underneath it —
# a running cluster keeps its old ports while rig starts reporting new ones, and
# `ports.sh show` stops describing reality. Anchored to three known names.
# Ports are derived, never stored: a changed derivation moves every existing env's ports.
check "derive_port_base rig" "20310" "$(derive_port_base rig)"
check "derive_port_base foo" "21690" "$(derive_port_base foo)"
check "derive_port_base my-proj" "21030" "$(derive_port_base my-proj)"
note "rig stays standalone"
# rig sits inside a host project's tree but must be copyable straight out of it:
# no imports, no paths, no assumption the host is there. This grep is the whole
# test of that claim, and until now it lived only in prose and in whoever
# remembered to run it.
#
# The pattern is assembled from fragments so this file does not match ITSELF.
# Writing it literally would fail forever; excluding this file instead would put
# a blind spot in the one check that guards the boundary.
# rig must be copyable out of its host project: no references to the host.
# The pattern is assembled from fragments so this file does not match itself.
HOST_PAT="$(printf '%s' 'sole' 'print' '|\b' 'sp' 'r\b')"
check "no host-project references" "0" \
"$(cd .. && grep -rIl -iE "$HOST_PAT" . --exclude-dir=def 2>/dev/null | wc -l)"
note "the Tiltfile hardcodes nothing"
# Every other Tiltfile on this machine writes its slug in five or six times by
# hand, so a copied project deploys into the original's cluster until someone
# edits all of them. rig's asks ports.sh. A literal kind-<name> here would mean
# that has been undone.
# The Tiltfile asks ports.sh for its context; a literal kind-<name> would undo that.
check "no literal kind-<name>" "0" "$(grep -cE "['\"]kind-[a-z0-9]" Tiltfile)"
check "guards on the variable" "1" "$(grep -c 'allow_k8s_contexts(CTX)' Tiltfile)"
check "asks ports.sh for facts" "1" "$(grep -c "local('bash ports.sh active'" Tiltfile)"
note "standalone kits are generated, current, and call only real verbs"
# The kits under standalone/<profile>/ are rig flattened into single files, one
# per profile. A kit left behind by a change to rig is exactly the drift they
# replaced — rigmini.sh once said 2 GB per node long after rig measured 800 MB —
# so a stale kit fails here rather than waiting to be noticed on another machine.
# A kit left stale by a change to rig fails here, not on another machine.
check "every kit matches what rig generates now" "yes" \
"$(bash standalone.sh check >/dev/null 2>&1 && echo yes || echo "no — run make standalone")"
# Each kit's Makefile exists so nothing wrapping these scripts has to GUESS how to
# call them. A generated Makefile once did guess: `rigmini.sh on`, not a verb,
# and a bare `rigdeps.sh` for "check and report", which installs. So every
# target's default verb must be one its script's own dispatch accepts — read
# from that dispatch, not from a list here that could drift from it.
# Every kit Makefile target must call a verb its script's own dispatch accepts.
verbs_of() {
sed -n '/^case "\$cmd" in/,/^esac/p' "$1" | grep -oE '^ [a-z]+\)' | tr -d ' )'
}
@@ -224,12 +166,8 @@ for mk in ../standalone/*/Makefile; do
done
check "there is a kit for every profile" "$(config_profiles | wc -l)" "$kits"
# An export is "take the setup I have here somewhere else", so it carries this
# machine's CHOICES — profile, ports, manifest dir — and never its credentials:
# ctrl/.env can hold registry and mirror logins next to those choices. The
# committed per-profile kits carry neither, since they must be the same on any
# machine. Proven with sentinel values in a scratch copy, because the real
# ctrl/.env may have those keys empty — and an empty value proves nothing.
# An export carries this machine's choices but never its credentials; committed kits carry neither.
# Proven with sentinel values in a scratch copy, since the real ctrl/.env may leave them empty.
SX="$TMP/export-proof"; mkdir -p "$SX"; cp -r .. "$SX/rig"
cat >> "$SX/rig/ctrl/.env" <<'EOF'
REGISTRY_USER=selftest-sentinel-user
@@ -250,10 +188,7 @@ check "export: refuses to write inside the repository" "yes" \
note "optional — needs tilt and this rig's cluster"
# Parsing the Tiltfile for real is the only way to know it still evaluates, but
# Tilt snapshots a kubectl context before parsing, so it cannot run without a
# cluster. Skipped rather than failed when there is none, the same way docgen
# skips its graphgen section.
# Tilt needs a cluster context to parse the Tiltfile, so this is skipped without one.
if ! command -v tilt >/dev/null; then
printf ' skip tilt is not installed\n'
elif ! kubectl config get-contexts -o name 2>/dev/null | grep -qx "$F_CTX"; then

52
rig/ctrl/standalone.sh
View File

@@ -1,52 +1,8 @@
#!/usr/bin/env bash
# Generate the standalone kits: single-file versions of rig's own tools, one
# folder per profile, for machines the full rig is not going to.
#
# A kit is a pure function of rig as it is right now. It gains nothing rig lacks
# and loses nothing rig has — improve rig, regenerate, and every kit follows.
# Nothing in standalone/<profile>/ is ever edited by hand.
#
# What this file does NOT know, on purpose: which tools rig has, what they are
# called, how its libraries are split, where configuration lives or what it
# contains. Rig will change shape — scripts get split, renamed and grow new
# libraries — and a generator that encoded today's layout would quietly produce
# a wrong kit the first time it did. So this works from a contract a script opts
# into, and from nothing else:
#
# 1. A marker comment, alone on a line near the top, declares an entry point:
# (hash) rig:standalone <kit-name> <default-verb>
# The default verb must only REPORT: it is run as a smoke test.
# 2. Every `source` an entry point makes names a .sh file by a path that
# resolves relative to the entry point. Libraries may source further
# libraries however they like — bash follows those itself.
# 3. Configuration enters through `load_config`, and the libraries provide
# `config_profiles`, `config_freeze <profile|--current>` — which prints a
# replacement load_config with that resolution frozen in — and, for an
# export, `config_current_profile` and `config_left_out`. How config is layered,
# stored, derived or frozen is rig's business; this only asks, and embeds
# the answer without interpreting it.
#
# Bash does the resolving, not a parser here. Libraries are sourced in a clean
# shell and read back with `declare -f` and `declare -p`, so any structure bash
# can load, this can flatten.
#
# And every kit is PROVEN to stand alone before it is written: no `source` left,
# no path into rig's tree in its code, `bash -n` clean, and its default verb run
# in an empty directory with nothing from rig present. A shape this has never
# seen either passes that, or generation stops and names the kit, the file, the
# line and what is wrong. It never writes a kit that only looks finished.
#
# Usage:
# standalone.sh write generate every kit into standalone/<profile>/
# standalone.sh check generate into a scratch dir and fail if any kit differs
# standalone.sh export DIR ONE kit for the configuration this machine runs —
# its profile plus the choices in its local config,
# WITHOUT its credentials — written outside the repo.
#
# write and check are what gets committed: one kit per profile, identical on any
# machine. export is the other question — "take the setup I have here somewhere
# else" — so it reflects this machine, and for exactly that reason it never lands
# in the repository.
# Generate standalone kits: rig's tools flattened into single files, one folder per profile.
# Usage: standalone.sh write|check generate (or diff) standalone/<profile>/
# standalone.sh export DIR one kit for this machine's config, no credentials, outside the repo
# Notes: docs/notes/standalone.md
set -euo pipefail
cd "$(dirname "$0")"

50
rig/ctrl/versions.env
View File

@@ -1,24 +1,7 @@
# Pinned toolchain — the single manifest ctrl/deps.sh installs from.
# Every entry is a single binary; none of them needs an apt repo.
# kubectl fully static
# kind libc only
# tilt libc + libstdc++ + libgcc (present in base Debian)
# jq upstream static build (Debian's is linked against libjq/libonig)
#
# Checksums are the upstream-published SHA256 of the linux/amd64 artifact.
#
# To bump: change the version, then take the checksum from the release's own
# published list — never hand-edit or hand-copy one from a download you did.
# For anything hosted on GitHub releases that is:
#
# curl -sSL https://github.com/<org>/<repo>/releases/download/<tag>/checksums.txt \
# | grep linux.x86_64
#
# (kubectl publishes its own instead: <KUBECTL_URL>.sha256.)
#
# There was a `ctrl/versions-refresh.sh` named here that has never existed. If
# bumping stops being rare enough to do by hand, write it — but a comment
# pointing at a missing script is worse than no comment.
# Pinned toolchain (linux/amd64, upstream SHA256) — the manifest ctrl/deps.sh installs from.
# To bump, take the checksum from the release's own list, e.g.
# curl -sSL https://github.com/<org>/<repo>/releases/download/<tag>/checksums.txt | grep linux.x86_64
# Notes: docs/notes/versions.md
KIND_VERSION=v0.32.0
KIND_SHA256=50030de23cf40a18505f20426f6a8506bedf13c6e509244bd1fa9463721b0f54
@@ -32,10 +15,7 @@ TILT_VERSION=0.37.6
TILT_SHA256=e9672b8a18d43501f35dcfe98465969a7db0e436b36cf0c50c7e6f8d40de5fe6
TILT_URL=https://github.com/tilt-dev/tilt/releases/download/v${TILT_VERSION}/tilt.${TILT_VERSION}.linux.x86_64.tar.gz
# ctlptl — creates a kind cluster WITH a local registry wired in, which is what
# keeps images off docker.io (an unqualified name means docker.io/library/<name>).
# Same publisher and same archive shape as tilt: binary at the archive root, so
# fetch_tgz handles it with strip=0 and no special case.
# ctlptl — kind cluster with a local registry wired in (keeps images off docker.io).
CTLPTL_VERSION=0.9.4
CTLPTL_SHA256=c63a1ec28e60bc3faf6becb76f53355c5cf5e0143dafdd27ad85db5584fa6b1e
CTLPTL_URL=https://github.com/tilt-dev/ctlptl/releases/download/v${CTLPTL_VERSION}/ctlptl.${CTLPTL_VERSION}.linux.x86_64.tar.gz
@@ -44,19 +24,14 @@ JQ_VERSION=1.8.2
JQ_SHA256=b1c22172dd303f3be49e935aa56aa48a8b7a46e0bc838b4997d3bb451495870f
JQ_URL=https://github.com/jqlang/jq/releases/download/jq-${JQ_VERSION}/jq-linux-amd64
# docker compose — the distro docker packages ship the daemon and the CLI but
# frequently not this, so `docker compose up` fails with "unknown command" on an
# otherwise working Docker. It is a CLI plugin, found by NAME in a plugin
# directory, so a copy in the bin dir alone only gives you the retired
# `docker-compose` v1 spelling; deps.sh links it into ~/.docker/cli-plugins.
# docker compose — often missing from distro packages; deps.sh links it into
# ~/.docker/cli-plugins.
COMPOSE_VERSION=5.5.1
COMPOSE_SHA256=db1889184726840f75c4f9c001048430d4f25b3be3cb084d3ddd762bc0aed576
COMPOSE_URL=https://github.com/docker/compose/releases/download/v${COMPOSE_VERSION}/docker-compose-linux-x86_64
# Node images shipped with KIND_VERSION above, pinned by digest so a kind upgrade
# can never silently move the k8s version. Profiles select one via K8S_VERSION.
# Older entries are kept deliberately: running a trailing-edge control plane is
# part of simulating a legacy estate.
# Node images for KIND_VERSION, pinned by digest; profiles pick one via K8S_VERSION.
# Older entries are kept deliberately (legacy-estate simulation).
NODE_IMAGE_v1_36=kindest/node:v1.36.1@sha256:3489c7674813ba5d8b1a9977baea8a6e553784dab7b84759d1014dbd78f7ebd5
NODE_IMAGE_v1_35=kindest/node:v1.35.5@sha256:ce977ae6d65918d0b58a5f8b5e940429c2ce42fa3a5619ec2bbc60b949c0ac95
NODE_IMAGE_v1_34=kindest/node:v1.34.8@sha256:02722c2dedddcfc00febf5d27fbeb9b7b2c14294c82109ff4a85d89ac9ba3256
@@ -72,11 +47,8 @@ CERT_MANAGER_VERSION=v1.21.1
METRICS_SERVER_VERSION=v0.9.0
METALLB_VERSION=v0.16.0
# Cabinets — public services dropped in as-is, the upstream image unmodified.
# The same declaration installs on compose or in the cluster, so a dependency is
# named once and works either way. Pinned by tag rather than
# digest because they are ordinary upstream images with no supply chain claim
# attached — bump freely, and preload them for the offline profile.
# Cabinets — unmodified upstream images, usable on compose or in the cluster.
# Pinned by tag; bump freely, and preload them for the offline profile.
POSTGRES_IMAGE=postgres:16-alpine
REDIS_IMAGE=redis:7-alpine
AIRFLOW_IMAGE=apache/airflow:2.10.4