rig major updates
This commit is contained in:
171
rig/docs/notes/overlay.md
Normal file
171
rig/docs/notes/overlay.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# Overlays: what runs lives outside rig
|
||||
|
||||
## Why
|
||||
|
||||
rig is the machine: the toolchain, the cluster, the registry, the port block, the
|
||||
dev loop's plumbing. What runs on it — the services, their manifests, their
|
||||
images, their settings — belongs to whoever owns that work, changes at a
|
||||
different rate, and often cannot be shared at all. Keeping both in one tree meant
|
||||
editing rig's own files to use it, and then carrying those edits into every copy.
|
||||
|
||||
So the use case is one folder, the **overlay**, kept outside rig's version
|
||||
control. rig reads it; rig never writes into it and never knows what is in it.
|
||||
The dependency points one way: an overlay knows about rig, rig knows about
|
||||
overlays in general and about none in particular.
|
||||
|
||||
## Where an overlay lives
|
||||
|
||||
```
|
||||
rig/local/<name>/ gitignored by rig: an overlay with no version control of its own,
|
||||
or a clone of its own repo
|
||||
anywhere/<name>/ a repo of its own, named by path
|
||||
<project>/ a project folder that carries rig at <project>/rig/ (the vendored
|
||||
layout, below)
|
||||
```
|
||||
|
||||
Name it with `OVERLAY` — in `ctrl/.env` for this machine, or per call:
|
||||
|
||||
```bash
|
||||
OVERLAY=local/myenv make cluster up
|
||||
```
|
||||
|
||||
Relative paths are relative to rig's folder. With none named, rig uses its own
|
||||
`examples/starter`, which sets nothing, so a plain rig behaves as it always did.
|
||||
An overlay that is named and missing is an error; it never falls back.
|
||||
|
||||
## What rig reads from it
|
||||
|
||||
Every piece is optional.
|
||||
|
||||
| in the overlay | what rig does with it |
|
||||
| --- | --- |
|
||||
| `rig.env` | a config layer (below). Any key a profile could set. |
|
||||
| `k8s/overlays/dev/` | the default `MANIFESTS_DIR`: rig's Tiltfile applies it with kustomize |
|
||||
| `kind-config.yaml.tpl` | the default `KIND_CONFIG`: the cluster's shape, rendered like rig's own |
|
||||
| `addons/<name>.sh` | an addon, found before rig's `ctrl/addons/<name>.sh` of the same name |
|
||||
| `Tiltfile` | the workload's half of the dev loop, included by rig's `ctrl/Tiltfile` |
|
||||
|
||||
Anything else in the folder is the overlay's own business: Dockerfiles, DAGs,
|
||||
folders of repos or data it mounts, its `.gitignore`, the secrets its kustomize
|
||||
generators read. rig does not look.
|
||||
|
||||
## Layers
|
||||
|
||||
```
|
||||
built-in defaults < ctrl/versions.env < ctrl/env.d/<profile>.env < <overlay>/rig.env < ctrl/.env < the caller
|
||||
```
|
||||
|
||||
A profile says how this machine reaches the world (a registry mirror, an
|
||||
air-gapped install); an overlay says what runs. `ctrl/.env` is still this
|
||||
machine's, and the caller still wins over everything.
|
||||
|
||||
`rig.env` may not set `PROFILE` or `OVERLAY`: both are chosen before it loads.
|
||||
The paths it sets (`MANIFESTS_DIR`, `KIND_CONFIG`) are relative to the overlay.
|
||||
`MANIFESTS_DIR=none` means rig applies no manifests and the overlay's Tiltfile
|
||||
does, e.g. when kustomize needs flags.
|
||||
|
||||
## Identity
|
||||
|
||||
With an overlay named, the cluster, the kubectl context and the port block
|
||||
follow the overlay folder's name, sanitised the same way a rig folder's name is.
|
||||
One rig can therefore serve several overlays, each in its own cluster, and a
|
||||
project that carries rig at `./rig` does not name every cluster `rig`.
|
||||
|
||||
`ports.sh persist` refuses while an overlay is set: it writes to rig's
|
||||
`ctrl/.env`, and a pin there would follow every overlay.
|
||||
|
||||
## The Tiltfile handoff
|
||||
|
||||
rig's `ctrl/Tiltfile` does rig's part — the context guard, `default_registry`,
|
||||
the manifests, the namespaces they use — then publishes the facts and includes
|
||||
the overlay's `Tiltfile`:
|
||||
|
||||
```
|
||||
RIG_CLUSTER RIG_CONTEXT RIG_HTTP_PORT RIG_HTTPS_PORT RIG_TILT_PORT RIG_REGISTRY RIG_OVERLAY_DIR
|
||||
```
|
||||
|
||||
Read them with `os.getenv`. An included Tiltfile runs from its own folder, so
|
||||
every relative path in it (`docker_build` contexts, `sync`, `deps`, `local`) is
|
||||
relative to the overlay — it never needs a path back into rig.
|
||||
|
||||
## Addons
|
||||
|
||||
An addon is a bash script run by `ctrl/addons.sh` from rig's `ctrl/`, with
|
||||
`RIG_CTRL` exported. It starts like this, sources the config and does its work:
|
||||
|
||||
```bash
|
||||
cd "${RIG_CTRL:?run it through rig: bash ctrl/addons.sh install}"
|
||||
source ./lib/config.sh
|
||||
load_config
|
||||
```
|
||||
|
||||
`OVERLAY_DIR` is set, so an addon can find files beside it
|
||||
(`$(_from_ctrl "$OVERLAY_DIR")/...`). rig's own `ctrl/addons/` holds only what
|
||||
makes a cluster work (metallb, cert-manager, metrics-server);
|
||||
`examples/data/addons/` shows workload ones.
|
||||
|
||||
## The kind config
|
||||
|
||||
An overlay's `kind-config.yaml.tpl` replaces rig's whole file, so start from a
|
||||
copy of `ctrl/k8s/kind-config.yaml.tpl` and keep its containerd `config_path`
|
||||
patch: `registry.sh` needs it, and `make check` says so when it is missing.
|
||||
`${OVERLAY_DIR}` renders to the overlay's absolute path, for mounts:
|
||||
|
||||
```yaml
|
||||
extraMounts:
|
||||
- hostPath: ${OVERLAY_DIR}/datadir
|
||||
containerPath: /rig/datadir
|
||||
```
|
||||
|
||||
A kind config is fixed when the cluster is created: after changing it,
|
||||
`make cluster reset`.
|
||||
|
||||
## The vendored layout
|
||||
|
||||
A project folder can carry rig inside it and be the overlay itself:
|
||||
|
||||
```
|
||||
<project>/
|
||||
Makefile the forwarder below
|
||||
rig.env k8s/ Tiltfile kind-config.yaml.tpl addons/ ...
|
||||
rig/ rig, placed as it is; tracked or ignored by the project, its call
|
||||
```
|
||||
|
||||
The forwarder runs rig with `OVERLAY` set to this folder. It passes `OVERLAY` in
|
||||
the environment, not as a make argument, so rig's own `$(shell ...)` sees it
|
||||
under make 4.3 as well:
|
||||
|
||||
```make
|
||||
HERE := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
|
||||
ifeq ($(wildcard $(HERE)/rig/Makefile),)
|
||||
$(error rig/ is missing — this folder is an overlay; put rig in ./rig)
|
||||
endif
|
||||
GOALS := $(or $(MAKECMDGOALS),help)
|
||||
.PHONY: $(GOALS)
|
||||
$(firstword $(GOALS)):
|
||||
@OVERLAY='$(HERE)' $(MAKE) --no-print-directory -C '$(HERE)/rig' $(GOALS)
|
||||
$(wordlist 2,$(words $(GOALS)),$(GOALS)):
|
||||
@:
|
||||
```
|
||||
|
||||
The cluster is then named after `<project>`, exactly as a copied rig named
|
||||
`<project>` was, so moving a copied rig to this layout keeps its cluster and ports.
|
||||
|
||||
`rig.env` holds no secrets, by this contract. A repository whose `.gitignore` has a
|
||||
broad `*.env` (a common secrets rule) would still hide it, so an overlay living in
|
||||
such a repo re-includes it in its own `.gitignore`: `!rig.env`.
|
||||
|
||||
## Moving a copied rig to an overlay
|
||||
|
||||
A rig copied into a project and edited there splits cleanly:
|
||||
|
||||
| was, in the copy | goes to |
|
||||
| --- | --- |
|
||||
| `ctrl/k8s/base`, `ctrl/k8s/overlays` | `k8s/` |
|
||||
| the workload parts of `ctrl/Tiltfile` | `Tiltfile` (paths now relative to the overlay) |
|
||||
| `ctrl/env.d/<name>.env` | `rig.env` |
|
||||
| edits to `ctrl/k8s/kind-config.yaml.tpl` | `kind-config.yaml.tpl` (`${HOST_WORKDIR}` → `${OVERLAY_DIR}`) |
|
||||
| workload addons | `addons/` |
|
||||
| Dockerfiles for the workload | beside the Tiltfile |
|
||||
| `ctrl/.env` | `rig/ctrl/.env` (this machine's; drop a `MANIFESTS_DIR=ctrl/k8s/overlays/dev` line) |
|
||||
| everything else of rig's | replaced by rig as it is |
|
||||
Reference in New Issue
Block a user