Compare commits

...

6 Commits

Author SHA1 Message Date
a29e0708e8 new conventions 2026-09-14 06:13:22 -03:00
37c4d588ea Merge branch 'main' into docgen-graphgen 2026-09-13 21:41:29 -03:00
160ee31b8c docgen: drop the superseded first pass, port the style harvester
The IR supersedes both intermediate designs (requirements D6, D7):
station/tools/docgen and the graph model that briefly lived in graphgen.
Shipping them beside atlas2/docgen would mean two graph models, which is the
thing the architecture argues against.

Carried across rather than lost:
  - style/extract.py and tokens.py, the offline theme harvester (R31). Rewritten
    to emit a *theme* — a slot-to-hex binding — rather than a whole style file,
    since what harvesting recovers is which colour a slot should be, not what a
    kind should look like.
  - graphgen/README.md, rewritten for what graphgen actually is now: the
    schema explorer. It fixes the blank station-index entry at run.py:304.

Two bugs found while doing it:
  - Canvas and ink are the two lightness extremes, not the two most common
    values. In a Graphviz SVG every label carries a fill, so the ink outnumbers
    the canvas 87 to 43 and the old rule produced a theme whose text was
    invisible against its own background.
  - A name defined in both branches of an if/else produced a duplicate id, which
    failed validation on docgen's own source. Disambiguated by line.
2026-09-13 21:41:29 -03:00
358b98f826 site emitter 2026-09-12 07:08:00 -03:00
7cb892ccfe add cases for code, dbs, and outline notebook generation 2026-09-12 06:50:05 -03:00
542d704da4 docgen iter 2 2026-09-12 06:42:49 -03:00
73 changed files with 14491 additions and 1 deletions

4
soleprint/atlas2/docgen/.gitignore vendored Normal file
View File

@@ -0,0 +1,4 @@
# Everything this makes.
out/
__pycache__/
*.pyc

View File

@@ -0,0 +1,171 @@
# docgen — code to diagram, and to everything else the IR can feed.
#
# Derived from where this file sits, so the folder can be copied anywhere and
# renamed and still work. The logic lives in the Python, never here.
#
# make book SRC=../station the whole operation, measured at both ends
# make check prove docgen, on a tree it builds itself
# make check BOOK=out/book/x prove one book — its own level
# make ir SRC=../station extract -> out/ir.json (one step, on its own)
# make graph out/ir.json -> out/graph.svg
# make index out/ir.json -> out/index.md
# make self docgen's book of itself, then check it
# make doctor what this machine has
#
# Every target below is one step of a book and still works alone — that is the
# property the book spine exists to preserve, not to replace.
#
# The pipeline is three commands and they compose, which is the point:
#
# python3 -m docgen.extractors.python --root SRC -o ir.json
# python3 -m docgen.ops ir.json --overview -o view.json
# python3 -m docgen.emitters dot view.json -o graph.svg --theme dark
HERE := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST)))))
PKG := $(notdir $(HERE))
PARENT := $(patsubst %/,%,$(dir $(HERE)))
PY ?= python3
RUN := PYTHONPATH=$(PARENT) $(PY) -m
OUT ?= $(HERE)/out
SRC ?=
SCHEMA ?=
OPENAPI ?=
HAR ?=
STYLE ?= lucid
THEME ?=
DEPTH ?= 2
SCALE ?= 0.55
BOOK ?=
SLUG ?=
# NOT `LANG`: that is the shell's locale variable, so `?=` inherits
# en_US.UTF-8 from the environment and --lang rejects it.
READER ?= python
OVERLAY ?=
THEME_ARG := $(if $(THEME),--theme $(THEME))
SLUG_ARG := $(if $(SLUG),--slug $(SLUG))
OVER_ARG := $(if $(OVERLAY),--overlay $(OVERLAY))
.PHONY: help book check ir db code graph index site minimap explore docs view self doctor clean
help: ## List every target
@echo "docgen — static analysis of a tree, and the artifacts that fall out of it"
@echo
@grep -E '^[a-z-]+:.*?## .*$$' $(MAKEFILE_LIST) \
| awk 'BEGIN{FS=":.*?## "}{printf " \033[1m%-10s\033[0m %s\n", $$1, $$2}'
@echo
@echo " SRC=/path/to/tree what to read OUT=/path where output goes"
@echo " SCHEMA=schema.json a database instead STYLE=lucid THEME=dark|lucid"
@echo " OPENAPI=spec.yaml an API document HAR=session.har a recording"
@echo " BOOK=/path where a book goes, and which book to check"
@echo " READER=python|code ast, or tree-sitter SLUG=name what to call the book"
@echo " OVERLAY=overlay.json hand-written notebook additions, re-applied every build"
@echo " DEPTH=2 how deep to draw"
@echo
@echo " Three levels of test, by what they assert about:"
@echo " make doctor the machine. Never fails."
@echo " make check docgen. Exits 1."
@echo " make check BOOK=<dir> that book. Exits 1."
check: ## Prove docgen (or one book, with BOOK=<dir>)
@if [ -n "$(BOOK)" ]; then \
$(RUN) $(PKG).book.checks "$(BOOK)"; \
else \
$(PY) $(HERE)/selftest.py; \
fi
book: ## SRC (or SCHEMA/OPENAPI/HAR) -> one operation, measured at both ends
@test -n "$(SRC)$(SCHEMA)$(OPENAPI)$(HAR)" \
|| { echo "Error: set SRC=/path/to/tree (or SCHEMA=, OPENAPI=, HAR=)" >&2; exit 1; }
@$(RUN) $(PKG).book \
$(if $(SRC),--root "$(SRC)" --lang $(READER)) \
$(if $(SCHEMA),--schema "$(SCHEMA)") \
$(if $(OPENAPI),--openapi "$(OPENAPI)") \
$(if $(HAR),--har "$(HAR)") \
-o "$(if $(BOOK),$(BOOK),$(OUT)/book)" \
--style $(STYLE) $(THEME_ARG) $(SLUG_ARG) $(OVER_ARG)
ir: ## Extract SRC into OUT/ir.json
@test -n "$(SRC)" || { echo "Error: set SRC=/path/to/tree" >&2; exit 1; }
@mkdir -p $(OUT)
@$(RUN) $(PKG).extractors.python --root "$(SRC)" -o $(OUT)/ir.json
@$(RUN) $(PKG).ir $(OUT)/ir.json
db: ## Extract a graphgen-compatible SCHEMA into OUT/ir.json
@test -n "$(SCHEMA)" || { echo "Error: set SCHEMA=/path/to/schema.json" >&2; exit 1; }
@mkdir -p $(OUT)
@$(RUN) $(PKG).extractors --schema "$(SCHEMA)" -o $(OUT)/ir.json
@$(RUN) $(PKG).ir $(OUT)/ir.json
view: ## OUT/ir.json -> OUT/view.json, the default view for its source type
@$(RUN) $(PKG).ops $(OUT)/ir.json --overview -o $(OUT)/view.json
graph: view ## OUT/view.json -> whatever its structure asks for
@$(RUN) $(PKG).emitters auto $(OUT)/view.json -o $(OUT) \
--style $(STYLE) $(THEME_ARG)
site: view ## OUT/view.json -> a self-contained docs site in OUT/site
@$(RUN) $(PKG).emitters site $(OUT)/view.json -o $(OUT)/site \
--style $(STYLE) $(THEME_ARG)
@echo " open $(OUT)/site/index.html"
docs: ## Regenerate the figures in docs/ — docgen documented by docgen
@mkdir -p docs/img
@$(RUN) $(PKG).extractors.python --root $(HERE) -o /tmp/$(PKG)-docs.json >/dev/null
@$(RUN) $(PKG).ops /tmp/$(PKG)-docs.json --overview -o /tmp/$(PKG)-docs-view.json >/dev/null
@$(RUN) $(PKG).emitters dot /tmp/$(PKG)-docs-view.json -o docs/img/architecture.svg -q
@$(RUN) $(PKG).emitters minimap /tmp/$(PKG)-docs.json -o docs/img/minimap.svg --scale 0.5 --width 860
@$(RUN) $(PKG).emitters erd $(OUT)/ir.json -o docs/img/erd.svg 2>/dev/null \
|| echo " (erd figure kept — needs a schema IR at $(OUT)/ir.json to refresh)"
@PYTHONPATH=$(PARENT) $(PY) -c "from $(PKG).emitters.site import VIEWER, _slots, _fill; \
from $(PKG).style import Style; import pathlib; \
pathlib.Path('$(HERE)/docs/viewer.html').write_text( \
_fill(VIEWER.replace('__TITLE__', 'docgen docs'), _slots(Style.load('lucid'))))"
@echo " open $(HERE)/docs/index.html"
explore: ## OUT/ir.json -> OUT/explore/ — navigate on one side, explore on the other
@$(RUN) $(PKG).emitters explore $(OUT)/ir.json -o $(OUT)/explore \
--style $(STYLE) $(THEME_ARG) --scale $(SCALE)
@echo " open $(OUT)/explore/explore.html"
minimap: ## OUT/ir.json -> OUT/minimap.svg — what is where, read from the colours
@$(RUN) $(PKG).emitters minimap $(OUT)/ir.json -o $(OUT)/minimap.svg \
--style $(STYLE) $(THEME_ARG) --scale $(SCALE)
code: ## Extract C#/TypeScript from SRC (needs tree-sitter)
@test -n "$(SRC)" || { echo "Error: set SRC=/path/to/tree" >&2; exit 1; }
@mkdir -p $(OUT)
@$(RUN) $(PKG).extractors code --root "$(SRC)" -o $(OUT)/ir.json
@$(RUN) $(PKG).ir $(OUT)/ir.json
index: ## OUT/ir.json -> OUT/index.md and OUT/sidebar.json
@$(RUN) $(PKG).emitters index $(OUT)/ir.json -o $(OUT)/index.md
@$(RUN) $(PKG).emitters index $(OUT)/ir.json -o $(OUT)/sidebar.json
self: ## docgen's book of the widest tree it can see, then check it
@$(eval SELF_SRC := $(shell PYTHONPATH=$(PARENT) $(PY) -c "from $(PKG) import reference; \
r = reference.root(); print(r if r else '$(HERE)')"))
@echo " self-hosting on $(SELF_SRC)"
@$(MAKE) --no-print-directory book SRC=$(SELF_SRC) SLUG=self \
BOOK=$(OUT)/book/self OUT=$(OUT)
@echo
@$(MAKE) --no-print-directory check BOOK=$(OUT)/book/self
doctor: ## Report whether this machine can run it
@printf 'python : '; $(PY) --version 2>&1 || echo MISSING
@printf 'dot : '; (dot -V 2>&1) || echo 'MISSING — sudo apt install graphviz (only to render)'
@printf 'tree-sit : '; $(PY) -c 'import tree_sitter, tree_sitter_c_sharp, tree_sitter_typescript; print("ok — C# and TypeScript available")' 2>/dev/null || echo 'absent — Python only. pip install tree_sitter tree_sitter_c_sharp tree_sitter_typescript'
@printf 'networkx : '; $(PY) -c 'import networkx; print(networkx.__version__ + " — for lab/ experiments")' 2>/dev/null || echo 'absent — only used in lab/'
@printf 'package : %s (from %s)\n' '$(PKG)' '$(PARENT)'
@printf 'yaml : '; $(PY) -c 'import yaml; print("ok — needed only to read OpenAPI")' 2>/dev/null || echo 'absent — only used by the OpenAPI reader'
@printf 'reference: '; PYTHONPATH=$(PARENT) $(PY) -c "from $(PKG) import reference; print(reference.describe())"
@printf 'styles : '; $(RUN) $(PKG).style 2>/dev/null \
|| $(RUN) $(PKG) 2>/dev/null \
|| PYTHONPATH=$(PARENT) $(PY) -c "from $(PKG).style import Style; print(', '.join(Style.available()))"
@PYTHONPATH=$(PARENT) $(PY) -c "import $(PKG).ir, $(PKG).emitters.dot, $(PKG).ops" >/dev/null 2>&1 \
&& echo 'import : ok' || echo 'import : FAILED — is the folder intact?'
clean: ## Delete OUT. Nothing else is ever written to
@rm -rf "$(OUT)" && echo "Removed $(OUT)"

View File

@@ -0,0 +1,225 @@
# docgen
Static analysis of a tree, and the artifacts that fall out of it.
The point is not the diagram. The point is the format in the middle — diagrams
are one consumer of it, and not the one that reaches the most people.
```
extractors/ → graph IR (JSON) → emitters/
(per source type) (one schema) (per output target)
style/*.json
(consumed by emitters only)
```
```bash
make check # prove it, on a tree it builds itself
make self # run the whole thing over soleprint
make ir SRC=../../station/tools/histgen
make index && make graph
make help
```
Or as three composable commands, which is what the Makefile is wrapping:
```bash
python3 -m docgen.extractors.python --root SRC -o ir.json
python3 -m docgen.ops ir.json --drop-stdlib -o view.json
python3 -m docgen.emitters dot view.json -o graph.svg --theme dark
```
## DOT collapses three concerns; this separates them
| concern | question | owner |
|---|---|---|
| **structure** | what the graph *is* | `ir/schema.json` — versioned, golden-tested |
| **meaning** | what things *mean visually* | `style/*.json`, keyed on `kind` |
| **placement** | where things *go* | Graphviz defaults. Phase two |
An extractor has never heard of SVG, colours or layout. An emitter has never
heard of Python, `ast` or SQL. **The IR carries no visual information** — if a
field would change between light and dark theme, it does not belong in it.
`shape="cylinder"` is not a field; it is `kind="datastore"` plus a style rule,
which is what lets the same IR render in a theme that has no cylinders.
The selftest asserts all three of those, because they are the design rather than
a nicety and they are exactly what erodes first.
## The IR
```json
{
"meta": { "source": "python", "root": "app/", "schema_version": "1" },
"nodes": [ { "id": "app.models.User", "kind": "class", "label": "User",
"parent": "app.models",
"attrs": { "file": "app/models.py", "line": 12 } } ],
"edges": [ { "source": "app.models.User", "target": "app.db.Base",
"kind": "inherits", "attrs": {} } ]
}
```
- **`id`** is fully qualified and **stable across runs**. That is what makes two
graphs from two commits diffable.
- **`kind`** is the hinge, and the only field style and layout may key on.
- **`parent`** is containment. Relationships are edges.
- **`attrs`** is an open bag; `file`/`line` let a UI link a box to a line.
Stdlib dataclasses, not Pydantic. A format that needs a library installed to be
opened is not a format, it is an API. `ir/validate.py` is the check at the
boundary, and it reads the field lists out of `schema.json` so the two cannot
drift.
```bash
python3 -m docgen.ir ir.json
```
It catches what a schema cannot: an edge naming a node that does not exist, a
containment cycle, duplicate ids, and a visual field smuggled into `attrs`.
## Extraction is deterministic
**No LLM in the structural path.** A diagram from an AST cannot be out of date
with the code; one from a model's reading of the code is wrong the moment the
model has a bad day, which is the problem this exists to fix.
`ast` resolves nothing on its own — `class User(Base)` yields the literal string
`"Base"`. So there are two passes: one collects each module's definitions and
imports, the other resolves names against those tables.
```
from .db import Base ; class User(Base)
→ app.models.User --inherits--> app.db.Base not "Base"
```
**Unresolved names become `kind: "external"` nodes and keep their edges.**
Dropping them is the worse failure: the diagram looks complete and has quietly
lost a dependency. Gathered by the index emitter, they *are* the project's
dependency surface.
An unparseable file is recorded as a node with an `error` attr, not a crash —
one bad file must not cost you the other four hundred.
`calls` edges are deliberately **not** attempted. Resolving `self.foo()` needs
type inference, and a call graph that is quietly 60% right is worse than none
because it reads as authoritative.
### A second source
`extractors/db.py` reads the published `{models, relationships, source}`
contract that `modelgen` already emits and `graphgen` already consumes. Tables
become nodes, columns become contained nodes, foreign keys become edges — with
no new top-level field, which was the checkpoint on whether the schema was right.
Connecting to a live database is not here. `modelgen from-db --url ...` does
that and writes the schema this reads; the two-step also keeps credentials out
of this pipeline entirely.
## Views are not an emitter concern
The first real diagram out of this pipeline was a 3000px strip: four modules of
content and sixty `sys`/`json`/`typing` boxes, all peers. The emitter was
correct and the picture was useless. That is a **missing view**, and the fix
belongs to every consumer at once — the index, the diagram and the diff all want
"just this subsystem, two hops out, without the stdlib".
```bash
python3 -m docgen.ops ir.json --drop-stdlib --around docgen.ir --hops 2 -o view.json
```
`drop_stdlib`, `drop_external`, `only_kinds`, `drop_kinds`, `subtree`,
`neighbourhood`, `collapse_to_depth`. All IR→IR, all composable, each producing
a document that still validates.
Graph *algorithms* are not here. Transitive reduction, cycle detection and
dominators are `networkx`'s, and reimplementing them is the classic way to
acquire a quiet bug. `lab/` is where that dependency gets tried against real IRs
before anything depends on it — the aim being to learn which part of it is
actually attractive, rather than adopting all of it on faith.
## One colour language
A style rule names a **slot**, never a colour. `"border": "atlas"` is the rule;
the theme binds `atlas` to `#43A047` in print and `#15803d` on the docs site.
That indirection is the whole point. `common/theme/tokens.css`,
`docs/graphs/themes/*.gvpr` and `style/lucid.json` use the same slot names, so a
diagram and the page around it match by construction — which is the rule
`docs/graphs/README.md` already states. The `dark` theme's `artery`, `atlas` and
`station` slots are exactly the `--system-accent` values set in
`artery/index.html:30`, `atlas/index.html:25` and `station/index.html:29`, and
the selftest fails if they drift apart.
An unknown `kind` falls back to `default` rather than crashing, so a new
extractor renders plainly and legibly on day one instead of needing a style file
written first.
**How a container picks its colour without the IR naming one:** it does not. The
IR says which spr model a group belongs to (`attrs.domain` — semantic), and
`domain_slots` maps that to a slot. Same mechanism as `--system-accent`. With no
domain, the emitter assigns by sorted id, so two runs agree.
## Use DOT until it hits its limits
The emitter writes what DOT expresses natively and stops at the boundary rather
than growing machinery. The limits are recorded in `style/lucid.json` under
`limits` and reachable as `Style.limits()`:
| | |
|---|---|
| header bars | a cluster has a label and a fill, not a 100%-width header rectangle |
| `stroke-dasharray` | not parameterised — `4,4` and `5,5` collapse to one dash |
| corner radius | `rounded` is binary, so 4px and 6px are identical |
| icon above label | needs an HTML-like label table |
| sequence badges | `xlabel` carries the number; the circle does not exist |
Those mark where a richer emitter would begin. The style file carries the full
spec regardless, so that emitter needs no re-authoring.
One limit that *was* worth solving: DOT cannot use a cluster as an edge
endpoint, so every module-to-module import silently vanished. The native answer
is `compound=true` with `lhead`/`ltail` — draw between a representative leaf and
clip at the cluster border.
## The output is addressable
`id` and `kind` pass through to the SVG as the element's `id` and `class`, and
`attrs.file`/`attrs.line` become an `href`. A front end can bind behaviour to a
box and a box can link to the line it came from, without the emitter knowing
about either.
## Testing
```bash
make check # 59 checks, offline, nothing installed
```
**Golden tests go on the IR, never on the SVG.** Graphviz measures label text
with the host's fonts to size nodes, so identical input gives different geometry
on a machine with different fontconfig. The IR is deterministic; the SVG is not.
Self-hosting is the honest end-to-end check, and it is where the real bugs came
from — two name-resolution faults that no fixture had reached:
```bash
make self # extract soleprint, and read out/index.md
```
## Where this sits
`docgen` belongs to Atlas — documentation is whose concern it is. It is **not** a
station tool and is not under `station/tools/`; it *may depend on* station tools,
which is the permitted direction.
Atlas 2 is a successor, not a replacement. `soleprint/atlas/` is untouched: it
carries client information and an idea still worth extracting — deriving frontend
and backend tests from one source, which is the same shape as this pointed the
other way.
## Not here
No layout system, no positioning, no ELK. No HTML-like labels, no SVG post-pass.
No LLM in the structural path — annotation (summarising a module, naming a
cluster) is a later layer, cached to its own file keyed by node `id`, merged into
`attrs` at emit time, and extraction must work with it absent. No configuration
knobs until two real consumers disagree.

View File

@@ -0,0 +1 @@
"""Docgen — code to diagram. The IR is the product; diagrams are one consumer."""

View File

@@ -0,0 +1,315 @@
"""
A **book** — one docgen operation, measured at both ends.
larder ──► step ──► step ──► step ──► book
what each one usable what
came in by itself came out
The first step says what went into the operation. The last step says what came
out, and renders it for the web. Everything between is an ordinary artifact that
stands on its own — `ir.json` is still an `ir.json`, and `make ir` still works
without knowing this file exists.
## Why both ends, and why they are steps rather than a wrapper
Because the two numbers are only worth having **together**. "102 nodes" is not a
fact about anything; "49 files in, 102 nodes out, nothing lost" is. A book that
read 45 of 47 files and drew a clean diagram is lying by omission, and before
this there was no place for the other 2 to be mentioned.
They are steps, not a wrapper, because a wrapper is something you can forget to
apply. A step is in the sequence, and the sequence is the thing the notebook
emits — so the measure is in the document whether or not anyone remembered.
## The two things it is not
**Not a gate.** Running one step alone is still a book, just a short one. An
operation that cannot measure something says what it could not measure and
carries on. Gating would break the property that makes the intermediate
artifacts useful, and that property is the whole reason the sequence is worth
having.
**Not a new pipeline.** Everything here composes functions that already existed
— `ops.overview`, `emitters.dot`, `emitters.site`, `notebook.spec`. The spine
adds a ledger and two measurements. If it ever starts doing the work itself,
something has gone wrong.
## The web end is deliberately the loose one
It is last, so nothing depends on it, so it can be replaced wholesale without
touching a single thing upstream. That is what lets it rule what gets generated
without being a stable contract: the book measure is the promise, and the page
that displays it is free to change drastically and often.
"""
import json
from dataclasses import dataclass, field
from pathlib import Path
from .larder import Larder
# The two spine steps. Named here because the notebook, the site and the checks
# all have to agree on what they are called, and three string literals in three
# files is how they stop agreeing.
FIRST = "larder"
LAST = "book"
@dataclass
class Step:
"""One thing the operation did, and what it left behind."""
id: str
label: str
artifact: str | None = None # relative to the book directory
bytes: int = 0
note: str = ""
skipped: str = "" # why, when it did not run
def to_dict(self) -> dict:
out = {"id": self.id, "label": self.label}
if self.artifact:
out["artifact"] = self.artifact
out["bytes"] = self.bytes
if self.note:
out["note"] = self.note
if self.skipped:
out["skipped"] = self.skipped
return out
# How a larder's unit shows up on the output side. Per-kind because the relation
# genuinely differs, and because getting it wrong produces a check that passes
# for the wrong reason — which is what the first version of this did.
#
# relation "exact" one unit in, one node out. Fewer means input was dropped.
# "collapse" many units in, fewer nodes out, by design.
# noun what to call the output-side thing, in the claim
# represents whether a unit that FAILED still appears as a node. Where it does,
# that is checkable and is the whole-input form of the rule that an
# unresolved name becomes an `external` node rather than vanishing.
RELATION = {
"python": ("exact", "module", True),
"code": ("exact", "module", True),
"db": ("exact", "table", False),
"openapi": ("exact", "endpoint", False),
# A HAR entry with no URL has nothing to represent, so there is no node to
# look for. Its absence is the correct outcome and is recorded in `failed`.
"usage": ("collapse", "call", False),
}
def _unit_counts(ir: dict, larder: Larder) -> tuple[int, int]:
"""(nodes from units that were read, nodes from units that failed).
Split because a failed file still gets a module node — carrying
`attrs.error`, so the gap is visible in the graph rather than only in a log.
Counting them together made "2 files read produced 4 modules" pass a check
that was supposed to prove nothing had been dropped.
"""
nodes = ir.get("nodes") or []
if larder.kind in ("python", "code"):
mods = [n for n in nodes
if n["kind"] == "module" and (n.get("attrs") or {}).get("file")]
broken = {n["attrs"]["file"] for n in mods if (n.get("attrs") or {}).get("error")}
whole = {n["attrs"]["file"] for n in mods} - broken
return len(whole), len(broken)
if larder.kind == "db":
return sum(1 for n in nodes if n["kind"] == "table"), 0
if larder.kind == "openapi":
return len({
(n.get("attrs") or {}).get("path") for n in nodes
if n["kind"] == "endpoint" and (n.get("attrs") or {}).get("path")
}), 0
if larder.kind == "usage":
return sum(1 for n in nodes if n["kind"] in ("endpoint", "operation")), 0
return 0, 0
class Book:
"""A named operation, its ledger, and the two measures that bracket it."""
def __init__(self, slug: str, larder: Larder, out):
self.slug = slug
self.larder = larder
self.out = Path(out)
self.steps: list[Step] = []
self.ir: dict | None = None
self.notebook: str | None = None
# The first step, recorded before any work happens. Doing it here rather
# than at the end is the difference between a measure and a summary: it
# says what the operation *set out* to read, so a crash halfway leaves a
# book that still says what went in.
self.steps.append(Step(
id=FIRST,
label="what came in",
note=larder.line(),
))
# -- the ledger -------------------------------------------------------
def step(self, id: str, label: str, path=None, note: str = "",
skipped: str = "") -> Step:
"""Record a step. `path` is written already; this measures it."""
artifact, size = None, 0
if path is not None:
path = Path(path)
if path.exists():
artifact = str(path.relative_to(self.out)) if self.out in path.parents \
or path.parent == self.out else str(path)
size = path.stat().st_size if path.is_file() else _tree_bytes(path)
s = Step(id=id, label=label, artifact=artifact, bytes=size,
note=note, skipped=skipped)
self.steps.append(s)
return s
# -- the last step ----------------------------------------------------
def measure(self) -> dict:
"""What came out. Counted off the final IR and the ledger."""
ir = self.ir or {"nodes": [], "edges": []}
by_kind: dict[str, int] = {}
for n in ir.get("nodes") or []:
by_kind[n["kind"]] = by_kind.get(n["kind"], 0) + 1
edge_kinds: dict[str, int] = {}
for e in ir.get("edges") or []:
edge_kinds[e["kind"]] = edge_kinds.get(e["kind"], 0) + 1
artifacts = [s for s in self.steps if s.artifact]
return {
"nodes": sum(by_kind.values()),
"by_kind": {k: by_kind[k] for k in sorted(by_kind)},
"edges": sum(edge_kinds.values()),
"edges_by_kind": {k: edge_kinds[k] for k in sorted(edge_kinds)},
"external": by_kind.get("external", 0),
"steps": len(self.steps),
"artifacts": [{"path": s.artifact, "bytes": s.bytes} for s in artifacts],
"bytes": sum(s.bytes for s in artifacts),
}
def compare(self) -> list[dict]:
"""Reconcile the two ends. This is what having both is *for*.
Returns observations, each with a verdict, rather than raising: the book
is already built by the time anyone can compare, and refusing to write it
would destroy the evidence. `book/checks.py` turns these into pass/fail
at the book test level, and the CLI exits 1 when one of them is not ok.
"""
out = []
ir = self.ir or {}
produced, represented = _unit_counts(ir, self.larder)
relation, noun, represents = RELATION.get(
self.larder.kind, ("collapse", "node", False))
read, unit = self.larder.read, self.larder.unit
n_failed = len(self.larder.failed)
if relation == "exact":
ok = produced >= read
out.append({
"id": "units-accounted-for",
"ok": ok,
"claim": f"{read} {unit}(s) read produced {produced} {noun}(s)",
"why": (
"one unit in, one node out" if ok else
f"{read - produced} {unit}(s) were read and produced nothing — "
"the input was dropped between the extractor and the document, "
"which is the failure this measure exists to catch"
),
})
else:
out.append({
"id": "collapse-is-intended",
"ok": produced >= 1 or read == 0,
"claim": f"{read} {unit}(s) collapsed to {produced} {noun}(s)",
"why": "many recorded requests describe few endpoints — that is the point",
})
if n_failed:
out.append({
"id": "failures-surfaced",
"ok": True,
"claim": f"{n_failed} {unit}(s) could not be read",
"why": "named in the larder measure and on the landing page, not only in a log",
})
# The whole-input form of "an unresolved name becomes an `external` node".
# A file that failed to parse must still be in the graph, or the diagram
# shows a tree that is smaller than the tree on disk and says nothing.
if represents and n_failed:
out.append({
"id": "failures-still-in-the-graph",
"ok": represented >= n_failed,
"claim": f"{n_failed} unreadable {unit}(s) appear as {represented} "
f"marked {noun}(s)",
"why": (
"a gap that is drawn can be seen" if represented >= n_failed else
f"{n_failed - represented} unreadable {unit}(s) are missing from the "
"graph entirely — the picture is smaller than the source and does "
"not say so"
),
})
return out
# -- writing ----------------------------------------------------------
def close(self, *, site=None) -> Step:
"""Append the last step. Call once, after the web output is written.
Must be called BEFORE the notebook is built, because the notebook quotes
the book measure and the measure is not complete until this step exists.
The site is this step's *artifact*. The notebook is not: it is the whole
sequence's rendering rather than an item in it, and it is set on
`self.notebook` afterwards, by name only — a notebook cannot report its
own byte count without changing it.
"""
m = self.measure()
summary = " · ".join(f"{v} {k}" for k, v in
sorted(m["by_kind"].items(), key=lambda kv: -kv[1]))
artifact, size = None, 0
if site is not None:
site = Path(site)
if site.exists():
artifact, size = str(site.relative_to(self.out)), _tree_bytes(site)
self.steps.append(Step(
id=LAST,
label="what came out",
artifact=artifact,
bytes=size,
note=f"{summary} · {m['edges']} edges" if summary else "nothing was produced",
))
return self.steps[-1]
def to_dict(self) -> dict:
"""The book, as the ledger someone else can read.
`generated_at` is absent for the same reason the IR's is: an unchanged
larder must serialise to the same bytes, or nothing downstream can tell
a real change from a rebuild.
"""
out = {
"slug": self.slug,
"larder": self.larder.to_dict(),
"steps": [s.to_dict() for s in self.steps],
"book": self.measure(),
"reconciled": self.compare(),
}
if self.notebook:
out["notebook"] = self.notebook
return out
def write(self) -> Path:
self.out.mkdir(parents=True, exist_ok=True)
path = self.out / "book.json"
path.write_text(json.dumps(self.to_dict(), indent=2, sort_keys=False) + "\n")
return path
def _tree_bytes(root: Path) -> int:
return sum(p.stat().st_size for p in root.rglob("*") if p.is_file())

View File

@@ -0,0 +1,80 @@
""" python3 -m docgen.book --root ../station -o out/book/station
One book: larder measure, the steps, the web output, book measure. The step
artifacts land in `steps/` and are ordinary files — nothing here needs this
command to have been the thing that produced them.
"""
import argparse
import sys
from pathlib import Path
KINDS = ("python", "code", "db", "openapi", "usage")
def main(argv=None) -> int:
p = argparse.ArgumentParser(
prog="python3 -m docgen.book",
description="Run one docgen operation, measured at both ends.",
)
src = p.add_mutually_exclusive_group(required=True)
src.add_argument("--root", type=Path, help="A source tree (Python, or --lang code).")
src.add_argument("--schema", type=Path, help="A graphgen-compatible schema.json.")
src.add_argument("--openapi", type=Path, help="An OpenAPI document.")
src.add_argument("--har", type=Path, help="A HAR recording.")
p.add_argument("--lang", choices=("python", "code"), default="python",
help="With --root: the stdlib ast reader, or tree-sitter. Default python.")
p.add_argument("--output", "-o", type=Path, required=True,
help="The book directory. Created if absent.")
p.add_argument("--slug", help="What to call it. Defaults to the source's name.")
p.add_argument("--style", default="lucid")
p.add_argument("--theme", default=None, help="dark (default) or lucid.")
p.add_argument("--overlay", type=Path,
help="A hand-written overlay, re-applied on every build.")
p.add_argument("--exclude", action="append", default=[],
help="Directory name to skip. Repeatable.")
p.add_argument("--quiet", "-q", action="store_true")
args = p.parse_args(argv)
if args.root is not None:
kind, source = args.lang, args.root
elif args.schema is not None:
kind, source = "db", args.schema
elif args.openapi is not None:
kind, source = "openapi", args.openapi
else:
kind, source = "usage", args.har
if not Path(source).exists():
print(f"Error: {source} does not exist", file=sys.stderr)
return 1
overlay = None
if args.overlay:
if not args.overlay.exists():
# Absent is fine and is the documented default; named-but-missing is
# a typo, and quietly building without it would hide the typo.
print(f"Error: overlay {args.overlay} does not exist", file=sys.stderr)
return 1
from ..notebook import spec as spec_mod
overlay = spec_mod.load(args.overlay)
from .build import run
try:
book = run(kind, source, args.output, slug=args.slug, style=args.style,
theme=args.theme, exclude=tuple(args.exclude), overlay=overlay,
quiet=args.quiet)
except Exception as e: # noqa: BLE001 - the CLI reports, it does not traceback
print(f"Error: {type(e).__name__}: {e}", file=sys.stderr)
return 1
# Exit 1 when the two ends do not reconcile. The book is still written —
# the evidence is the point — but a build that lost input should fail a
# pipeline rather than pass quietly.
lost = [r for r in book.compare() if not r.get("ok")]
return 1 if lost else 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,350 @@
"""
Run a book: larder, the steps, the web output, the book measure.
python3 -m docgen.book --root ../station -o out/book/station
Composes functions that already exist. Nothing here parses, lays out or styles
anything — `ops.overview`, `emitters.dot`, `emitters.site` and `notebook.spec`
do all of it, and this decides the order and writes the ledger. If this file
ever starts doing the work, the seam has moved to the wrong place.
## The notebook is the sequence, not an item in it
So it is not a step. Its **first cell is the larder measure and its last cell is
the book measure**, which is what makes the two ends part of the document rather
than part of the tooling. Between them, one pair of cells per step: what the
step did, and a code cell that loads that step's artifact on its own.
That last part is the "usable by themselves" property made executable. Each cell
reads one artifact and prints one fact, so a reader can start anywhere in the
sequence, and `selftest.py` runs them.
"""
import json
from pathlib import Path
from . import Book
from .larder import Larder
# Loader for the notebook's step cells. Guarded on purpose: a notebook is
# opened from wherever somebody happens to open it, and a traceback on cell 2
# is a worse answer than a sentence saying which directory to run it from.
PRELUDE = '''from pathlib import Path
import json
BOOK = Path.cwd() # the book directory — change if you opened this elsewhere
def load(rel):
"""One step's artifact, on its own. Returns None when it is not here.
A step's artifact can be a directory — the explorer is one — so this returns
a file list for those rather than trying to read a directory as text.
"""
p = BOOK / rel
if not p.exists():
print(f"{rel} is not here — run: make book OUT={BOOK}")
return None
if p.is_dir():
return sorted(f.relative_to(p).as_posix() for f in p.rglob("*") if f.is_file())
return json.loads(p.read_text()) if p.suffix == ".json" else p.read_text()
'''
# Every extractor a book can run, as data rather than as an if/elif chain.
#
# This is the shape that gives the extension contract teeth. `selftest.py` loops
# over this dict and asserts each entry reports a larder measure — so adding an
# extractor makes the test start asking about it without anyone remembering to
# go and add a case. rig's `CONFIG_OVERRIDABLE` loop is the precedent, and it
# found two real bugs that way.
#
# kind -> (module, function, what the source is)
EXTRACTORS = {
"python": (".extractors.python", "extract", "tree"),
"code": (".extractors.code", "extract", "tree"),
"db": (".extractors.db", "extract", "file"),
"openapi": (".extractors.openapi", "extract", "file"),
"usage": (".extractors.usage", "extract", "file"),
}
# Extractors that walk a directory take an exclude list; the ones that read a
# single document have nothing to exclude.
TAKES_EXCLUDE = {"python", "code"}
def extract(kind: str, source, *, exclude=(), identity=None):
"""(IR dict, Larder) for one source type, dispatched through EXTRACTORS."""
from importlib import import_module
if kind not in EXTRACTORS:
raise ValueError(
f"no extractor named {kind!r} — have {', '.join(sorted(EXTRACTORS))}"
)
module_name, fn_name, _ = EXTRACTORS[kind]
# The package name is derived, not written: the Makefile takes PKG from the
# directory name so the folder can be copied anywhere and renamed, and a
# literal "docgen" here would quietly undo that.
root_pkg = __package__.rsplit(".", 1)[0]
fn = getattr(import_module(module_name, package=root_pkg), fn_name)
kwargs = {"identity": identity or str(source)}
if kind in TAKES_EXCLUDE:
kwargs["exclude"] = exclude
g = fn(source, **kwargs)
ir = g.to_dict()
measured = ir["meta"].get("larder")
if measured is None:
# Not fatal. An extractor that cannot count its input still produces a
# book; what it does not get to do is pretend it measured one.
return ir, None
return ir, Larder.from_dict(measured)
def spec_from(book: Book, ir: dict) -> dict:
"""The book's ledger -> a notebook spec, with the two measures at the ends.
Reuses `notebook.spec`'s step vocabulary rather than inventing one, so
`merge()` keeps working and a hand-written overlay can annotate a spine step
the same way it annotates any other.
"""
from ..notebook import spec as spec_mod
steps = [
spec_mod._step(
"larder", "md", title=f"{book.slug} — what came in",
text=(
f"`{book.larder.identity}`\n\n**{book.larder.line()}**\n\n"
"This is the first step of the book and the only measure of the "
"input. Everything below is derived from it, so a number here "
"that looks wrong makes everything below it suspect."
+ (
"\n\nCould not be read:\n\n"
+ "\n".join(f"- `{f['name']}` — {f['error']}"
for f in book.larder.failed)
if book.larder.failed else ""
)
),
),
spec_mod._step("prelude", "code", title="Reading a step on its own",
code=PRELUDE),
]
for s in book.steps:
if s.id in ("larder", "book") or not s.artifact:
continue
steps.append(spec_mod._step(
f"step-{s.id}", "md", title=s.label,
text=f"`{s.artifact}` — {s.bytes:,} bytes" + (f"\n\n{s.note}" if s.note else ""),
))
steps.append(spec_mod._step(
f"load-{s.id}", "code", code=_load_cell(s.id, s.artifact),
))
# Where the larder is API-shaped, the generated endpoint walkthrough slots in
# as further steps. For a tree of source there are no endpoints and this adds
# nothing, which is the correct amount for it to add.
if any(n["kind"] in ("endpoint", "operation") for n in ir.get("nodes") or []):
generated = spec_mod.from_ir(ir)
steps.extend(s for s in generated["steps"] if s["id"] not in ("intro",))
m = book.measure()
# `external` is reported on its own below, so it is dropped here rather than
# appearing twice in one line — which is how it read before.
summary = " · ".join(f"{v} {k}" for k, v in
sorted(m["by_kind"].items(), key=lambda kv: -kv[1])
if k != "external")
reconciled = "\n".join(
f"- {'' if r['ok'] else ''} {r['claim']}{r['why']}"
for r in book.compare()
)
steps.append(spec_mod._step(
"book", "md", title=f"{book.slug} — what came out",
text=(
f"**{summary} · {m['edges']} edges · {m['external']} external**\n\n"
f"{len(m['artifacts'])} artifact(s), {m['bytes']:,} bytes.\n\n"
f"Reconciled against what came in:\n\n{reconciled}\n\n"
"The web output is `site/index.html`."
),
))
return {"version": spec_mod.SPEC_VERSION, "steps": steps}
def _load_cell(step_id: str, artifact: str) -> str:
"""A cell that reads one artifact and prints one fact about it.
The fact has to suit the artifact. An earlier version printed "nodes, edges"
for every JSON file, so `sidebar.json` — which has neither — reported
"0 nodes, 0 edges", which is a true sentence about the wrong thing and worse
than saying nothing.
"""
if not artifact.endswith((".json", ".svg", ".md")):
# A directory, e.g. the explorer.
return (
f'files = load("{artifact}")\n'
'if files is not None:\n'
' print(f"{len(files)} file(s)")\n'
' print("\\n".join(files[:5]))'
)
if artifact.endswith(".json"):
return (
f'data = load("{artifact}")\n'
'if data:\n'
' if "nodes" in data:\n'
' print(f\'{len(data["nodes"])} nodes, {len(data.get("edges", []))} edges\')\n'
' else:\n'
' print(", ".join(f"{k}: {len(v) if isinstance(v, (list, dict)) else v}"\n'
' for k, v in data.items()))'
)
if artifact.endswith(".svg"):
# No IPython import, guarded or otherwise. A generated notebook is a
# build artifact and has to run wherever it is opened; requiring a
# kernel package in order to *load a file* would make the cell fail on
# the machine that produced it, which is where this was found.
return (
f'svg = load("{artifact}")\n'
'if svg:\n'
' print(f"{len(svg):,} bytes of SVG")\n'
' # In Jupyter: from IPython.display import SVG; SVG(svg)'
)
if artifact.endswith(".md"):
return (
f'text = load("{artifact}")\n'
'if text:\n'
' print(text[:400])'
)
return f'print(load("{artifact}") is not None)'
def run(kind: str, source, out, *, slug: str | None = None, style: str = "lucid",
theme: str | None = None, exclude=(), overlay=None, quiet: bool = False) -> Book:
"""The whole book, in one process. Returns it; `book.json` is written."""
from ..ir import check
from ..ops import classify, overview
from ..style import Style
out = Path(out)
slug = slug or Path(str(source)).name or "book"
steps_dir = out / "steps"
steps_dir.mkdir(parents=True, exist_ok=True)
def say(text):
if not quiet:
print(text)
ir, larder = extract(kind, source, exclude=exclude)
if larder is None:
larder = Larder(kind=kind, identity=str(source), unit="document", seen=0)
book = Book(slug=slug, larder=larder, out=out)
book.ir = ir
say(f" larder {larder.line()}")
problems = check(ir)
if problems:
# Recorded as a step rather than raised: a book that cannot be trusted
# should exist and say so, because the alternative is that nobody can
# see what went wrong.
book.step("validate", "the IR did not validate", note="; ".join(problems[:3]))
say(f" WARNING IR has {len(problems)} problem(s)")
p = steps_dir / "ir.json"
p.write_text(json.dumps(ir, indent=2) + "\n")
book.step("ir", "the graph, extracted", p, note=f"{len(ir['nodes'])} nodes")
view = overview(ir)
p = steps_dir / "view.json"
p.write_text(json.dumps(view, indent=2) + "\n")
verdict = classify(view)
book.step("view", "the default view for this source", p,
note=f"{verdict['kind']}{verdict['why']}")
style_obj = Style.load(style, theme=theme)
# The drawing, chosen by structure rather than by the caller. `classify`
# already decided; this runs what it named.
graph_rel = None
if verdict["emitter"] == "erd":
from ..emitters.erd import emit as erd_emit
p = steps_dir / "graph.svg"
p.write_text(erd_emit(view, style_obj))
graph_rel, label = "steps/graph.svg", "drawn as an ERD"
elif verdict["emitter"] == "index":
from ..emitters.index import to_markdown
p = steps_dir / "graph.md"
p.write_text(to_markdown(view))
graph_rel, label = None, "not a diagram — written as a list"
else:
from ..emitters.dot import emit as dot_emit, render
p = steps_dir / "graph.svg"
try:
p.write_bytes(render(dot_emit(view, style_obj,
rankdir=(verdict.get("options") or {}).get("rankdir"))))
graph_rel, label = "steps/graph.svg", f"drawn as a {verdict['kind']}"
except Exception as e: # noqa: BLE001 - graphviz may not be installed
p = None
label = f"not drawn — {type(e).__name__}"
book.step("graph", label, skipped=str(e)[:200])
if p is not None:
book.step("graph", label, p, note=verdict["why"])
from ..emitters.index import to_markdown, to_sidebar
p = steps_dir / "index.md"
p.write_text(to_markdown(ir))
book.step("index", "readable without a diagram", p)
p = steps_dir / "sidebar.json"
p.write_text(json.dumps(to_sidebar(ir), indent=2) + "\n")
book.step("sidebar", "navigation, for whatever renders it", p)
try:
from ..emitters.minimap import emit as mm_emit
p = steps_dir / "minimap.svg"
p.write_text(mm_emit(ir, style_obj))
book.step("minimap", "what is where, read from the colours", p)
except Exception as e: # noqa: BLE001
book.step("minimap", "minimap not drawn", skipped=f"{type(e).__name__}: {e}")
try:
from ..emitters.explore import write as exp_write
exp_write(ir, style_obj, out / "explore")
book.step("explore", "navigate on one side, explore on the other",
out / "explore")
except Exception as e: # noqa: BLE001
book.step("explore", "explorer not built", skipped=f"{type(e).__name__}: {e}")
# -- the last step ----------------------------------------------------
from ..emitters.site import write as site_write
site_dir = out / "site"
site_write(view, style_obj, site_dir,
graph=Path(graph_rel).name if graph_rel else None,
title=slug, book=book.to_dict())
if graph_rel and (out / graph_rel).exists():
(site_dir / Path(graph_rel).name).write_bytes((out / graph_rel).read_bytes())
# close() BEFORE the notebook spec is built, not after. The spec quotes the
# book measure, and the book measure only includes the site once the last
# step exists — build it the other way round and the notebook says 7
# artifacts while book.json says 8, which is exactly what it did.
book.close(site=site_dir)
from ..emitters.notebook import write as nb_write
from ..notebook import spec as spec_mod
spec, spec_problems = spec_mod.merge(spec_from(book, ir), overlay)
nb_write(spec, out / "notebook.ipynb")
# Recorded by name and not by size: a notebook cannot report its own byte
# count without changing it.
book.notebook = "notebook.ipynb"
for pr in spec_problems:
say(f" overlay {pr}")
path = book.write()
m = book.measure()
say(f" book {m['nodes']} nodes · {m['edges']} edges · "
f"{m['external']} external · {len(m['artifacts'])} artifacts")
for r in book.compare():
say(f" {'ok ' if r['ok'] else 'LOST'} {r['claim']}")
say(f" open {site_dir / 'index.html'}")
say(f" {path}")
return book

View File

@@ -0,0 +1,305 @@
"""
Does *this* book still hold — the third test level.
python3 -m docgen.book.checks out/book/station
docgen has three levels of test, and they differ by what they assert *about*.
The distinction is rig's, from `rig/ctrl/selftest.sh`, and it is worth keeping
because it decides what a failure means:
make doctor the MACHINE. Never fails; it reports.
make check DOCGEN. Exits 1. "docgen no longer does what it says."
make check BOOK=<dir>
THIS BOOK. Exits 1. "this book no longer holds."
The third is the one that reaches a project docgen has never seen. It is also
where **framework and custom checks live together**, at different levels:
generated the spine's own assertions, identical for every book. Both
measures present, the reconciliation holding, every artifact
where the ledger says it is, the notebook still executing.
custom hand-written, in the book's own `checks.py`, using the same
check/note/skip helpers — so a project's line and a framework
line read identically and fail identically.
Same split as the notebook's base and overlay, for the same reason: generation
alone cannot know what *this* project cares about, and hand-authoring alone rots.
## The discipline these are written in
Carried from rig verbatim, because it is the point of the whole level:
> Each check is ONE decision that has already been made, with the reason above
> it — not coverage, and deliberately not an exhaustive sweep of use cases. A
> rule without its reason gets overridden the first time it is inconvenient.
> Failing one should read as **"you are about to undo this"** rather than
> "something broke".
## Writing a book's own checks
Put a `checks.py` next to `book.json`:
def checks(book, check, note, skip):
note("what this project will not give up")
# Payments moved once already and the move broke three dashboards.
# If it is not here, something renamed it again.
check("payments is still a module", True,
any(n["id"] == "app.payments" for n in book.ir["nodes"]))
`book` carries `.data` (the parsed `book.json`), `.ir` (the extracted graph),
and `.dir`. `check` takes (name, expected, actual) — expected first, because a
failure report is only useful if it says what was wanted.
"""
import json
import sys
from pathlib import Path
from . import FIRST, LAST
class Loaded:
"""A book read back off disk, for checking rather than building."""
def __init__(self, directory):
self.dir = Path(directory)
path = self.dir / "book.json"
if not path.exists():
raise FileNotFoundError(
f"{path} does not exist — is {self.dir} a book directory?\n"
"Build one with: python3 -m docgen.book --root <src> -o <dir>"
)
self.data = json.loads(path.read_text())
ir_path = self.dir / "steps" / "ir.json"
self.ir = json.loads(ir_path.read_text()) if ir_path.exists() else None
nb_path = self.dir / (self.data.get("notebook") or "notebook.ipynb")
self.notebook = json.loads(nb_path.read_text()) if nb_path.exists() else None
@property
def larder(self) -> dict:
return self.data.get("larder") or {}
@property
def measure(self) -> dict:
return self.data.get("book") or {}
class Report:
"""rig's reporting shape: sections, one line per decision, nothing aborts."""
def __init__(self):
self.passed, self.failed, self.skipped = 0, [], 0
def note(self, text: str) -> None:
print(f"\n{text}")
def check(self, name: str, expected, actual) -> bool:
if expected == actual:
print(f" ok {name}")
self.passed += 1
return True
print(f" FAIL {name}\n expected: {expected!r}\n got: {actual!r}")
self.failed.append(name)
return False
def skip(self, name: str, why: str) -> None:
print(f" -- {name} ({why})")
self.skipped += 1
def total(self) -> int:
print()
# stdout is block-buffered when redirected and stderr is not, so the
# failure summary printed below would otherwise arrive BEFORE the checks
# it summarises — which is how this was found, piping to `tail`.
sys.stdout.flush()
if not self.failed:
print(f"{self.passed} checks passed — this book still holds"
+ (f", {self.skipped} skipped" if self.skipped else ""))
return 0
print(f"FAILED — {len(self.failed)} of {self.passed + len(self.failed)}: "
f"{', '.join(self.failed)}", file=sys.stderr)
print("Read the comment next to the check. A failure here means a decision "
"has drifted, not that a tool is broken.", file=sys.stderr)
return 1
def generated(book: Loaded, r: Report) -> None:
"""The spine's own assertions. Identical for every book, by design."""
r.note("the book is bracketed — both ends present")
# The one structural promise the whole design makes: an operation is
# measured at both ends. A book missing either end is not a short book, it
# is a book whose numbers cannot be reconciled against anything.
ids = [s["id"] for s in book.data.get("steps") or []]
r.check("the first step is the larder measure", FIRST, ids[0] if ids else None)
r.check("the last step is the book measure", LAST, ids[-1] if ids else None)
r.check("nothing is bracketed twice", [1, 1],
[ids.count(FIRST), ids.count(LAST)])
r.note("what came in was measured, not guessed")
larder = book.larder
for key in ("kind", "identity", "unit", "seen", "read", "failed"):
r.check(f"larder records {key}", True, key in larder)
# read is derived, so a stored value that disagrees means somebody wrote it
# by hand. This is the same arithmetic ir/validate.py enforces; asserted
# again here because a book can be assembled without going through the IR.
r.check("read == seen - failed", larder.get("read"),
max(0, larder.get("seen", 0) - len(larder.get("failed") or [])))
# The one field that could carry a secret out of a database URL.
r.check("the identity carries no password", True,
":***@" in larder.get("identity", "") or "@" not in larder.get("identity", ""))
r.note("the two ends reconcile")
reconciled = book.data.get("reconciled") or []
r.check("something was reconciled", True, len(reconciled) > 0)
for item in reconciled:
# Each of these is a claim the book makes about itself. A false one
# means the document below is smaller than the source and does not say
# so, which is the single failure this level exists to catch.
r.check(item["claim"], True, item["ok"])
r.note("the ledger describes files that exist")
for step in book.data.get("steps") or []:
artifact = step.get("artifact")
if not artifact:
continue
path = book.dir / artifact
# A ledger naming a file that is not there is worse than no ledger: it
# is a manifest somebody will build tooling against.
r.check(f"{artifact} is where the ledger says", True, path.exists())
if path.is_file():
r.check(f"{artifact} is {step['bytes']} bytes", step["bytes"],
path.stat().st_size)
r.note("the notebook carries the sequence, with the measures at its ends")
if book.notebook is None:
r.skip("notebook", "no notebook.ipynb in this book")
else:
cells = book.notebook.get("cells") or []
first = "".join(cells[0]["source"]) if cells else ""
last = "".join(cells[-1]["source"]) if cells else ""
r.check("its first cell is what came in", True, "what came in" in first)
r.check("its last cell is what came out", True, "what came out" in last)
# Compiling is not enough — see selftest.py. `json.dumps` writes
# `false`/`true`/`null`, which are valid Python *identifiers*, so a
# generated body full of them compiles and then raises NameError.
ran, failure = _run_cells(cells, book.dir)
r.check(f"its {ran} code cells run, not merely compile", None, failure)
r.note("the web output exists and shows both ends")
index = book.dir / "site" / "index.html"
if not index.exists():
r.skip("site", "no site/index.html in this book")
else:
html = index.read_text()
# The site is the last step precisely because it is the artifact somebody
# definitely opens. A page that shows the result without showing what
# went in is the thing this whole change corrects.
r.check("the page says what came in", True, "what came in" in html)
r.check("the page says what came out", True, "what came out" in html)
if larder.get("failed"):
r.check("the page admits the book is incomplete", True,
"This book is incomplete" in html)
r.note("the book is reproducible")
# A timestamp would make two builds of an unchanged larder differ, which
# destroys the only useful property a ledger has: that a diff means a real
# change. Same rule as the IR's `generated_at`.
r.check("no timestamp in the ledger", True, "generated_at" not in json.dumps(book.data))
def custom(book: Loaded, r: Report) -> None:
"""This book's own assertions, if it has any."""
path = book.dir / "checks.py"
if not path.exists():
r.note("this book's own checks")
r.skip("custom checks", f"no {path.name} — write one to assert what this project cares about")
return
namespace: dict = {"__file__": str(path), "__name__": "book_checks"}
try:
exec(compile(path.read_text(), str(path), "exec"), namespace)
except Exception as e: # noqa: BLE001 - report, do not traceback
r.note("this book's own checks")
r.check(f"{path.name} loads", None, f"{type(e).__name__}: {e}")
return
fn = namespace.get("checks")
if not callable(fn):
r.note("this book's own checks")
r.check(f"{path.name} defines checks(book, check, note, skip)", True, False)
return
try:
fn(book, r.check, r.note, r.skip)
except Exception as e: # noqa: BLE001
r.check(f"{path.name} ran to completion", None, f"{type(e).__name__}: {e}")
def _run_cells(cells: list, cwd: Path) -> tuple[int, str | None]:
"""Execute the notebook's code cells from the book directory."""
import contextlib
import io
import os
ns, ran, failure = {}, 0, None
previous = os.getcwd()
try:
os.chdir(cwd)
with contextlib.redirect_stdout(io.StringIO()):
for c in cells:
if c.get("cell_type") != "code":
continue
src = "".join(c["source"])
if "urlopen" in src or ("call(" in src and "def call" not in src):
# Anything that would reach the network is compiled, not run.
try:
compile(src, c["id"], "exec")
ran += 1
except SyntaxError as e:
failure = f"{c['id']}: {e}"
break
continue
try:
exec(compile(src, c["id"], "exec"), ns)
ran += 1
except Exception as e: # noqa: BLE001 - any failure is the finding
failure = f"{c['id']}: {type(e).__name__}: {e}"
break
finally:
os.chdir(previous)
return ran, failure
def main(argv=None) -> int:
import argparse
p = argparse.ArgumentParser(
prog="python3 -m docgen.book.checks",
description="Check one book: the spine's assertions, then its own.",
)
p.add_argument("book", type=Path, help="A book directory (holding book.json).")
p.add_argument("--only", choices=("generated", "custom"),
help="Run one level rather than both.")
args = p.parse_args(argv)
try:
book = Loaded(args.book)
except (FileNotFoundError, json.JSONDecodeError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
print(f"{book.dir}{book.data.get('slug', '?')}")
r = Report()
if args.only != "custom":
generated(book, r)
if args.only != "generated":
custom(book, r)
return r.total()
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,195 @@
"""
What went in — the first step of every book.
A **larder** is the stocked source together with what connects it: a repo, a
live database, an OpenAPI endpoint, a HAR capture. Deliberately not "bucket".
A bucket is somewhere bytes sit; a larder is stocked from outside, has an
inventory, and goes stale. All three of those are properties worth measuring,
and they are the reason this file exists.
## Why a measure at all
Every extractor knew what it read and threw the number away. `Meta` carried
`source` and `root` — which extractor ran, and what it was pointed at — and
nothing about what was actually consumed. So a run that read 45 of 47 files
produced exactly the same document as one that read all 47, and the diagram
looked complete either way.
That is the failure this prevents, and it is the same failure the IR already
guards against one level down: an unresolved name becomes an `external` node
rather than being dropped, because *silently losing a thing is worse than
recording an unresolved one*. A larder measure is that rule applied to the
input as a whole.
## seen, failed, read
seen how many units the larder offered
failed the ones that could not be consumed, by name, with the reason
read seen - len(failed), derived and never stored
Three fields where two numbers would do, on purpose: `read` as a stored value
invites the question "does it include the failures", and every reader answers it
differently. Derived, there is nothing to get wrong.
## Redaction
`identity` is the one field in this whole tool that can carry a credential —
a database DSN has the password in it. So it is scrubbed here at construction,
and `ir/validate.py` sweeps for the scrub having worked. That is the same
discipline as `VISUAL_KEYS`: the architectural rule is a check, not a convention.
Paths are left as the caller gave them rather than resolved. An absolute path is
not a secret but it is machine-specific, and `meta.root` is already a bare name
for exactly that reason.
"""
import re
from dataclasses import dataclass, field
# What `seen` counts, per source type. Closed, like the IR's `kind` vocabulary:
# a larder that counts something with no name here is a larder nobody can
# compare against another one.
UNITS = ("file", "table", "path", "entry", "document")
# Query-string and connection-string keys whose value is a secret. Matched
# case-insensitively, because ODBC writes `Password=` and URLs write `password=`.
SECRET_KEYS = (
"password", "passwd", "pwd", "secret", "token", "access_token", "refresh_token",
"api_key", "apikey", "key", "sig", "signature", "auth", "credentials",
)
MASK = "***"
def _count(label: str, n: int) -> str:
"""Agree a label with its count, both directions.
Two directions because both arise: the closed `UNITS` are singular and need
pluralising, while `extra` keys are written plural by the extractor that
knows them ("packages", "hosts") and need singularising at one. Naive `+ "s"`
gives "entrys"; naive nothing gives "1 hosts".
"""
if n == 1:
return label[:-3] + "y" if label.endswith("ies") else label.rstrip("s") or label
if label.endswith("y"):
return label[:-1] + "ies"
return label if label.endswith("s") else label + "s"
def redact(identity: str) -> str:
"""Strip credentials out of a path or connection string.
Three shapes, which is all of them in practice:
postgresql://user:hunter2@host:5432/db -> postgresql://user:***@host:5432/db
https://api/x?token=abc123 -> https://api/x?token=***
Driver=x;Server=y;Password=hunter2; -> Driver=x;Server=y;Password=***;
The user, host, port and database survive. Those are what someone reading
the measure needs in order to recognise which larder this was, and none of
them is a secret.
"""
if not identity:
return identity
# scheme://user:secret@host — the password is between the first colon after
# the scheme and the last @ of the authority.
identity = re.sub(
r"(?P<scheme>[a-zA-Z][a-zA-Z0-9+.\-]*://)(?P<user>[^:/@\s]+):(?P<secret>[^@/\s]*)@",
lambda m: f"{m.group('scheme')}{m.group('user')}:{MASK}@",
identity,
)
# key=value, in a query string or a semicolon-delimited connection string.
keys = "|".join(re.escape(k) for k in SECRET_KEYS)
identity = re.sub(
rf"(?i)\b(?P<key>{keys})(?P<sep>\s*=\s*)(?P<secret>[^&;\s]*)",
lambda m: f"{m.group('key')}{m.group('sep')}{MASK}",
identity,
)
return identity
@dataclass
class Larder:
"""What one source offered, and how much of it was consumed.
Lands in `meta.larder`. Never in `nodes` or `attrs` — the IR is structure,
and provenance is meta. That split is the same one that keeps colour out.
"""
kind: str # which extractor stocked it
identity: str # path or redacted DSN
unit: str # one of UNITS
seen: int = 0
failed: list[dict] = field(default_factory=list)
extra: dict = field(default_factory=dict) # per-source facts
def __post_init__(self):
self.identity = redact(self.identity)
if self.unit not in UNITS:
raise ValueError(
f"larder unit {self.unit!r} is not one of {UNITS}"
"a unit nobody else uses cannot be compared against another larder"
)
@property
def read(self) -> int:
"""`seen` minus what failed. Derived, so it cannot disagree with itself."""
return max(0, self.seen - len(self.failed))
def fail(self, name: str, error: str) -> None:
"""Record a unit that could not be consumed, by name and reason."""
self.failed.append({"name": name, "error": error})
def to_dict(self) -> dict:
"""Key order fixed so an unchanged larder serialises byte-identically."""
out = {
"kind": self.kind,
"identity": self.identity,
"unit": self.unit,
"seen": self.seen,
"read": self.read,
"failed": [{"name": f["name"], "error": f["error"]} for f in self.failed],
}
if self.extra:
out["extra"] = {k: self.extra[k] for k in sorted(self.extra)}
return out
@classmethod
def from_dict(cls, data: dict) -> "Larder":
"""Round-trip. `read` is dropped: it is derived, not carried."""
return cls(
kind=data["kind"],
identity=data["identity"],
unit=data["unit"],
seen=data.get("seen", 0),
failed=list(data.get("failed") or []),
extra=dict(data.get("extra") or {}),
)
def line(self) -> str:
"""The one-line human form — the notebook's first cell, and the CLI.
Reads as a sentence because it is the first thing anyone sees about a
book, and "47 files read, 2 failed" is the fact that decides whether the
rest of the document is worth trusting. Which is also why the grammar
gets attention it would not otherwise deserve: "2 entrys read, 1 hosts"
reads as a machine talking, and a measure nobody reads is not a measure.
"""
parts = [f"{self.read} {_count(self.unit, self.read)} read"]
if self.failed:
parts.append(f"{len(self.failed)} failed")
for key in sorted(self.extra):
value = self.extra[key]
label = key.replace("_", " ")
if isinstance(value, int):
parts.append(f"{value} {_count(label, value)}")
else:
parts.append(f"{value} {label}")
return f"{self.identity}" + ", ".join(parts)
def of(kind: str, identity: str, unit: str, **extra) -> Larder:
"""Shorthand for the common case: `of("db", dsn, "table", dialect=...)`."""
return Larder(kind=kind, identity=identity, unit=unit, extra=extra)

View File

@@ -0,0 +1,135 @@
/* docgen docs. The layout five demos under semester/ converged on: a sticky
sidebar beside a bounded content column. Colours are tokens.css values, the
same ones the diagrams are drawn in. */
:root {
--bg: #0a0a0a;
--surface: #141414;
--surface-2: #1a1a1a;
--border: #333333;
--border-strong: #4a4a4a;
--text: #e5e5e5;
--muted: #a3a3a3;
--dim: #666666;
--accent: #d4a574;
--station: #1d4ed8;
--atlas: #15803d;
--artery: #b91c1c;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
background: var(--bg);
color: var(--text);
font-family: "Segoe UI", Inter, system-ui, -apple-system, Arial, sans-serif;
font-size: 14px;
line-height: 1.72;
}
.layout { display: flex; min-height: 100vh; }
/* ── sidebar ─────────────────────────────────────────────────────────── */
.sidebar {
width: 232px; flex-shrink: 0;
background: var(--surface);
border-right: 1px solid var(--border);
position: sticky; top: 0; height: 100vh; overflow-y: auto;
padding: 1.4rem 0 3rem;
scrollbar-width: none;
}
.sidebar::-webkit-scrollbar { display: none; }
.sidebar-header { padding: 0 1.15rem 1rem; border-bottom: 1px solid var(--border); }
.sidebar-header b { display: block; font-size: 14px; color: var(--text); }
.sidebar-header small { display: block; color: var(--dim); font-size: 11px; margin-top: 3px; }
.sidebar nav { padding-top: .75rem; }
.sidebar .group {
color: var(--dim); font-size: 9.5px; text-transform: uppercase;
letter-spacing: .07em; padding: 1rem 1.15rem .3rem;
}
.sidebar a {
display: block; padding: 3px 1.15rem;
color: var(--muted); text-decoration: none; font-size: 12.5px;
border-left: 2px solid transparent;
}
.sidebar a:hover { color: var(--text); background: var(--surface-2); }
.sidebar a.active { color: var(--accent); border-left-color: var(--accent); }
/* ── content ─────────────────────────────────────────────────────────── */
.content { flex: 1; min-width: 0; max-width: 820px; padding: 2.5rem 3.25rem 6rem; }
h1 { font-size: 26px; letter-spacing: -.01em; margin-bottom: .35rem; }
.lede { color: var(--muted); font-size: 15px; margin-bottom: 2.5rem; }
h2 {
font-size: 18px; margin: 3rem 0 .9rem; padding-top: 1.6rem;
border-top: 1px solid var(--border);
}
h2:first-of-type { border-top: none; padding-top: 0; }
h3 { font-size: 14px; margin: 1.9rem 0 .5rem; color: var(--text); }
h4 { font-size: 12.5px; margin: 1.3rem 0 .35rem; color: var(--muted); font-weight: 600; }
p { margin-bottom: .95rem; color: var(--muted); }
p strong, li strong { color: var(--text); font-weight: 600; }
em { color: var(--text); font-style: italic; }
ul, ol { margin: 0 0 1rem 1.15rem; color: var(--muted); }
li { margin-bottom: .3rem; }
a { color: var(--accent); text-decoration: none; }
a:hover { text-decoration: underline; }
code {
font-family: ui-monospace, "Cascadia Mono", Consolas, "SF Mono", monospace;
font-size: 12px; background: var(--surface); color: var(--text);
padding: 1px 5px; border-radius: 3px;
}
pre {
background: var(--surface); border: 1px solid var(--border);
border-radius: 7px; padding: 13px 16px; overflow-x: auto; margin: .9rem 0 1.2rem;
}
pre code { background: none; padding: 0; font-size: 12px; line-height: 1.62; color: var(--muted); }
pre .c { color: var(--dim); }
pre .k { color: var(--accent); }
table { border-collapse: collapse; margin: 1rem 0 1.4rem; width: 100%; font-size: 13px; }
th, td { text-align: left; padding: 7px 16px 7px 0; border-bottom: 1px solid var(--border);
color: var(--muted); vertical-align: top; }
th { color: var(--dim); font-size: 10px; text-transform: uppercase; letter-spacing: .06em;
font-weight: 600; }
td code { white-space: nowrap; }
blockquote {
border-left: 2px solid var(--border-strong); padding: .15rem 0 .15rem 1.1rem;
margin: 1.1rem 0; color: var(--dim); font-style: italic;
}
/* a claim worth not losing in the prose */
.note {
border: 1px solid var(--border); border-left: 3px solid var(--accent);
background: var(--surface); border-radius: 6px;
padding: .85rem 1.1rem; margin: 1.2rem 0; font-size: 13px;
}
.note b { color: var(--accent); }
.note.warn { border-left-color: var(--artery); }
.note.warn b { color: var(--artery); }
/* figures — inline and scaled, click for the viewer */
figure { margin: 1.3rem 0 1.7rem; }
figure a { display: block; border: 1px solid var(--border); border-radius: 8px;
overflow: hidden; background: var(--surface-2); }
figure a:hover { border-color: var(--accent); }
figure img { display: block; width: 100%; height: auto; }
figcaption { color: var(--dim); font-size: 11px; margin-top: .45rem; }
.pill {
display: inline-block; font-size: 10px; letter-spacing: .04em;
border: 1px solid var(--border-strong); border-radius: 20px;
padding: 1px 9px; color: var(--dim); margin-left: .5em; vertical-align: middle;
}
.pill.on { color: var(--atlas); border-color: var(--atlas); }
.pill.opt { color: var(--accent); border-color: var(--accent); }
.cols { display: flex; gap: 2rem; flex-wrap: wrap; }
.cols > div { flex: 1 1 250px; }

File diff suppressed because it is too large Load Diff

After

Width:  |  Height:  |  Size: 83 KiB

View File

@@ -0,0 +1,103 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" width="870pt" height="474pt" viewBox="0 0 870 474">
<rect width="870" height="474" fill="#0a0a0a"/>
<defs>
<marker id="fk" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="6" markerHeight="6" orient="auto-start-reverse"><path d="M 0 0 L 10 5 L 0 10 z" fill="#1d4ed8"/></marker>
</defs>
<g class="relationships">
<path d="M 330,103.0 C 280,103.0 300,155.0 250,155.0" fill="none" stroke="#1d4ed8" stroke-width="1.5" marker-end="url(#fk)" class="edge foreign_key"/>
<path d="M 620,155.0 C 570,155.0 590,155.0 540,155.0" fill="none" stroke="#1d4ed8" stroke-width="1.5" marker-end="url(#fk)" class="edge foreign_key"/>
<path d="M 250,369.0 C 300,369.0 280,155.0 330,155.0" fill="none" stroke="#1d4ed8" stroke-width="1.5" marker-end="url(#fk)" class="edge foreign_key"/>
</g>
<g class="table">
<rect x="40" y="40" width="210" height="154" rx="8" fill="#0a0a0a" stroke="#333333" stroke-width="1" data-id="Customer" data-kind="table" class="blk"/>
<path d="M 40,48 a 8,8 0 0 1 8,-8 h 194 a 8,8 0 0 1 8,8 v 42 h -210 z" fill="#1a1a1a"/>
<line x1="40" y1="90" x2="250" y2="90" stroke="#333333" stroke-width="1"/>
<text x="52" y="62" font-family="Helvetica,sans-Serif" font-size="12" font-weight="bold" fill="#e5e5e5">Customer</text>
<text x="52" y="78" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">A fixture customer (obviously…</text>
<text x="80" y="107" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">created_at</text>
<text x="238" y="107" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">datetime</text>
<line x1="41" y1="116" x2="249" y2="116" stroke="#1a1a1a" stroke-width="1"/>
<text x="80" y="133" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">email</text>
<text x="238" y="133" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">str</text>
<line x1="41" y1="142" x2="249" y2="142" stroke="#1a1a1a" stroke-width="1"/>
<text x="52" y="159" font-family="Helvetica,sans-Serif" font-size="8" font-weight="bold" fill="#d4a574">PK</text>
<text x="80" y="159" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">id</text>
<text x="238" y="159" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">int</text>
<line x1="41" y1="168" x2="249" y2="168" stroke="#1a1a1a" stroke-width="1"/>
<text x="80" y="185" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">name</text>
<text x="238" y="185" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">str</text>
</g>
<g class="table">
<rect x="330" y="40" width="210" height="206" rx="8" fill="#0a0a0a" stroke="#333333" stroke-width="1" data-id="Invoice" data-kind="table" class="blk"/>
<path d="M 330,48 a 8,8 0 0 1 8,-8 h 194 a 8,8 0 0 1 8,8 v 42 h -210 z" fill="#1a1a1a"/>
<line x1="330" y1="90" x2="540" y2="90" stroke="#333333" stroke-width="1"/>
<text x="342" y="62" font-family="Helvetica,sans-Serif" font-size="12" font-weight="bold" fill="#e5e5e5">Invoice</text>
<text x="342" y="78" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">An invoice issued to a custom…</text>
<text x="342" y="107" font-family="Helvetica,sans-Serif" font-size="8" font-weight="bold" fill="#1d4ed8">FK</text>
<text x="370" y="107" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">customer_id</text>
<text x="528" y="107" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">Customer</text>
<line x1="331" y1="116" x2="539" y2="116" stroke="#1a1a1a" stroke-width="1"/>
<text x="370" y="133" font-family="Helvetica,sans-Serif" font-size="10" fill="#a3a3a3">due_at</text>
<text x="528" y="133" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">datetime</text>
<line x1="331" y1="142" x2="539" y2="142" stroke="#1a1a1a" stroke-width="1"/>
<text x="342" y="159" font-family="Helvetica,sans-Serif" font-size="8" font-weight="bold" fill="#d4a574">PK</text>
<text x="370" y="159" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">id</text>
<text x="528" y="159" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">int</text>
<line x1="331" y1="168" x2="539" y2="168" stroke="#1a1a1a" stroke-width="1"/>
<text x="370" y="185" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">issued_at</text>
<text x="528" y="185" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">datetime</text>
<line x1="331" y1="194" x2="539" y2="194" stroke="#1a1a1a" stroke-width="1"/>
<text x="370" y="211" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">number</text>
<text x="528" y="211" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">str</text>
<line x1="331" y1="220" x2="539" y2="220" stroke="#1a1a1a" stroke-width="1"/>
<text x="370" y="237" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">status</text>
<text x="528" y="237" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">str</text>
</g>
<g class="table">
<rect x="620" y="40" width="210" height="180" rx="8" fill="#0a0a0a" stroke="#333333" stroke-width="1" data-id="LineItem" data-kind="table" class="blk"/>
<path d="M 620,48 a 8,8 0 0 1 8,-8 h 194 a 8,8 0 0 1 8,8 v 42 h -210 z" fill="#1a1a1a"/>
<line x1="620" y1="90" x2="830" y2="90" stroke="#333333" stroke-width="1"/>
<text x="632" y="62" font-family="Helvetica,sans-Serif" font-size="12" font-weight="bold" fill="#e5e5e5">LineItem</text>
<text x="632" y="78" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">A single billable line on an …</text>
<text x="660" y="107" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">description</text>
<text x="818" y="107" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">str</text>
<line x1="621" y1="116" x2="829" y2="116" stroke="#1a1a1a" stroke-width="1"/>
<text x="632" y="133" font-family="Helvetica,sans-Serif" font-size="8" font-weight="bold" fill="#d4a574">PK</text>
<text x="660" y="133" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">id</text>
<text x="818" y="133" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">int</text>
<line x1="621" y1="142" x2="829" y2="142" stroke="#1a1a1a" stroke-width="1"/>
<text x="632" y="159" font-family="Helvetica,sans-Serif" font-size="8" font-weight="bold" fill="#1d4ed8">FK</text>
<text x="660" y="159" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">invoice_id</text>
<text x="818" y="159" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">Invoice</text>
<line x1="621" y1="168" x2="829" y2="168" stroke="#1a1a1a" stroke-width="1"/>
<text x="660" y="185" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">quantity</text>
<text x="818" y="185" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">int</text>
<line x1="621" y1="194" x2="829" y2="194" stroke="#1a1a1a" stroke-width="1"/>
<text x="660" y="211" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">unit_price</text>
<text x="818" y="211" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">Decimal</text>
</g>
<g class="table">
<rect x="40" y="254" width="210" height="180" rx="8" fill="#0a0a0a" stroke="#333333" stroke-width="1" data-id="Payment" data-kind="table" class="blk"/>
<path d="M 40,262 a 8,8 0 0 1 8,-8 h 194 a 8,8 0 0 1 8,8 v 42 h -210 z" fill="#1a1a1a"/>
<line x1="40" y1="304" x2="250" y2="304" stroke="#333333" stroke-width="1"/>
<text x="52" y="276" font-family="Helvetica,sans-Serif" font-size="12" font-weight="bold" fill="#e5e5e5">Payment</text>
<text x="52" y="292" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">A payment recorded against an…</text>
<text x="80" y="321" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">amount</text>
<text x="238" y="321" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">Decimal</text>
<line x1="41" y1="330" x2="249" y2="330" stroke="#1a1a1a" stroke-width="1"/>
<text x="52" y="347" font-family="Helvetica,sans-Serif" font-size="8" font-weight="bold" fill="#d4a574">PK</text>
<text x="80" y="347" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">id</text>
<text x="238" y="347" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">int</text>
<line x1="41" y1="356" x2="249" y2="356" stroke="#1a1a1a" stroke-width="1"/>
<text x="52" y="373" font-family="Helvetica,sans-Serif" font-size="8" font-weight="bold" fill="#1d4ed8">FK</text>
<text x="80" y="373" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">invoice_id</text>
<text x="238" y="373" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">Invoice</text>
<line x1="41" y1="382" x2="249" y2="382" stroke="#1a1a1a" stroke-width="1"/>
<text x="80" y="399" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">method</text>
<text x="238" y="399" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">str</text>
<line x1="41" y1="408" x2="249" y2="408" stroke="#1a1a1a" stroke-width="1"/>
<text x="80" y="425" font-family="Helvetica,sans-Serif" font-size="10" fill="#e5e5e5">paid_at</text>
<text x="238" y="425" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">datetime</text>
</g>
</svg>

After

Width:  |  Height:  |  Size: 9.4 KiB

View File

@@ -0,0 +1,386 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<svg xmlns="http://www.w3.org/2000/svg" width="886pt" height="2251pt" viewBox="0 0 886 2251">
<rect width="886" height="2251" fill="#0a0a0a"/>
<rect x="28.0" y="46.0" width="74.0" height="247.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.site — 494 lines</title></rect>
<rect x="28.0" y="182.0" width="74.0" height="27.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.site._ledger" data-kind="function" class="blk"><title>_ledger — function, 54 lines</title></rect>
<rect x="28.0" y="210.0" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.site._slots" data-kind="function" class="blk"><title>_slots — function, 13 lines</title></rect>
<rect x="28.0" y="217.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.site._fill" data-kind="function" class="blk"><title>_fill — function, 4 lines</title></rect>
<rect x="28.0" y="220.5" width="74.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.site._sidebar" data-kind="function" class="blk"><title>_sidebar — function, 17 lines</title></rect>
<rect x="28.0" y="230.0" width="74.0" height="7.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.site._sections" data-kind="function" class="blk"><title>_sections — function, 15 lines</title></rect>
<rect x="28.0" y="238.5" width="74.0" height="48.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.site.emit" data-kind="function" class="blk"><title>emit — function, 96 lines</title></rect>
<rect x="28.0" y="287.5" width="74.0" height="5.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.site.write" data-kind="function" class="blk"><title>write — function, 10 lines</title></rect>
<rect x="110.0" y="46.0" width="74.0" height="158.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.explore — 316 lines</title></rect>
<rect x="110.0" y="65.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.explore._is_schema" data-kind="function" class="blk"><title>_is_schema — function, 4 lines</title></rect>
<rect x="110.0" y="68.5" width="74.0" height="24.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.explore._neighbourhood_svgs" data-kind="function" class="blk"><title>_neighbourhood_svgs — function, 48 lines</title></rect>
<rect x="117.0" y="75.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.explore._neighbourhood_svgs.render_one" data-kind="function" class="blk"><title>render_one — function, 2 lines</title></rect>
<rect x="117.0" y="78.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.explore._neighbourhood_svgs.render_one#L66" data-kind="function" class="blk"><title>render_one — function, 2 lines</title></rect>
<rect x="110.0" y="93.5" width="74.0" height="13.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.explore._facts" data-kind="function" class="blk"><title>_facts — function, 27 lines</title></rect>
<rect x="110.0" y="108.0" width="74.0" height="86.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.explore.emit" data-kind="function" class="blk"><title>emit — function, 172 lines</title></rect>
<rect x="110.0" y="195.0" width="74.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.explore.write" data-kind="function" class="blk"><title>write — function, 17 lines</title></rect>
<rect x="192.0" y="46.0" width="74.0" height="146.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.dot — 292 lines</title></rect>
<rect x="192.0" y="64.5" width="74.0" height="2.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.emitters.dot.RenderError" data-kind="class" class="blk"><title>RenderError — class, 2 lines</title></rect>
<rect x="192.0" y="66.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._esc" data-kind="function" class="blk"><title>_esc — function, 2 lines</title></rect>
<rect x="192.0" y="68.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._attrs" data-kind="function" class="blk"><title>_attrs — function, 3 lines</title></rect>
<rect x="192.0" y="71.0" width="74.0" height="4.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._style_words" data-kind="function" class="blk"><title>_style_words — function, 9 lines</title></rect>
<rect x="192.0" y="76.5" width="74.0" height="12.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._node_attrs" data-kind="function" class="blk"><title>_node_attrs — function, 24 lines</title></rect>
<rect x="192.0" y="89.5" width="74.0" height="52.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot.emit" data-kind="function" class="blk"><title>emit — function, 105 lines</title></rect>
<rect x="199.0" y="103.0" width="60.0" height="14.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.dot.emit.write" data-kind="function" class="blk"><title>write — function, 29 lines</title></rect>
<rect x="192.0" y="143.0" width="74.0" height="4.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._within" data-kind="function" class="blk"><title>_within — function, 8 lines</title></rect>
<rect x="192.0" y="148.0" width="74.0" height="20.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._endpoints" data-kind="function" class="blk"><title>_endpoints — function, 41 lines</title></rect>
<rect x="199.0" y="153.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.dot._endpoints.walk" data-kind="function" class="blk"><title>walk — function, 4 lines</title></rect>
<rect x="199.0" y="157.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.dot._endpoints.collapsed" data-kind="function" class="blk"><title>collapsed — function, 2 lines</title></rect>
<rect x="199.0" y="158.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.dot._endpoints.first_leaf" data-kind="function" class="blk"><title>first_leaf — function, 4 lines</title></rect>
<rect x="192.0" y="169.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._safe" data-kind="function" class="blk"><title>_safe — function, 2 lines</title></rect>
<rect x="192.0" y="171.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot._q" data-kind="function" class="blk"><title>_q — function, 2 lines</title></rect>
<rect x="192.0" y="175.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot.have_graphviz" data-kind="function" class="blk"><title>have_graphviz — function, 2 lines</title></rect>
<rect x="192.0" y="177.0" width="74.0" height="14.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.dot.render" data-kind="function" class="blk"><title>render — function, 29 lines</title></rect>
<rect x="274.0" y="46.0" width="74.0" height="139.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.notebook — 279 lines</title></rect>
<rect x="274.0" y="67.0" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook._cell" data-kind="function" class="blk"><title>_cell — function, 13 lines</title></rect>
<rect x="274.0" y="74.5" width="74.0" height="13.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook._example" data-kind="function" class="blk"><title>_example — function, 26 lines</title></rect>
<rect x="274.0" y="108.0" width="74.0" height="9.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook._params_cell" data-kind="function" class="blk"><title>_params_cell — function, 18 lines</title></rect>
<rect x="274.0" y="118.0" width="74.0" height="20.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook._call_cell" data-kind="function" class="blk"><title>_call_cell — function, 40 lines</title></rect>
<rect x="274.0" y="139.0" width="74.0" height="13.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook._call_md" data-kind="function" class="blk"><title>_call_md — function, 26 lines</title></rect>
<rect x="274.0" y="153.0" width="74.0" height="26.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook.build" data-kind="function" class="blk"><title>build — function, 52 lines</title></rect>
<rect x="274.0" y="180.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook.emit" data-kind="function" class="blk"><title>emit — function, 3 lines</title></rect>
<rect x="274.0" y="182.5" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.notebook.write" data-kind="function" class="blk"><title>write — function, 5 lines</title></rect>
<rect x="356.0" y="46.0" width="74.0" height="139.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.minimap — 278 lines</title></rect>
<rect x="356.0" y="77.5" width="74.0" height="28.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.minimap._files" data-kind="function" class="blk"><title>_files — function, 57 lines</title></rect>
<rect x="363.0" y="81.5" width="60.0" height="9.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.minimap._files.declared" data-kind="function" class="blk"><title>declared — function, 18 lines</title></rect>
<rect x="363.0" y="91.0" width="60.0" height="5.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.minimap._files.build" data-kind="function" class="blk"><title>build — function, 11 lines</title></rect>
<rect x="356.0" y="107.0" width="74.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.minimap._bands" data-kind="function" class="blk"><title>_bands — function, 7 lines</title></rect>
<rect x="356.0" y="111.5" width="74.0" height="9.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.minimap._blocks" data-kind="function" class="blk"><title>_blocks — function, 19 lines</title></rect>
<rect x="356.0" y="122.0" width="74.0" height="60.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.minimap.emit" data-kind="function" class="blk"><title>emit — function, 120 lines</title></rect>
<rect x="356.0" y="183.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.minimap.marks_to_labels" data-kind="function" class="blk"><title>marks_to_labels — function, 3 lines</title></rect>
<rect x="438.0" y="46.0" width="74.0" height="130.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.erd — 260 lines</title></rect>
<rect x="438.0" y="73.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.erd._truncate" data-kind="function" class="blk"><title>_truncate — function, 3 lines</title></rect>
<rect x="438.0" y="75.5" width="74.0" height="10.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.erd._tables" data-kind="function" class="blk"><title>_tables — function, 20 lines</title></rect>
<rect x="438.0" y="86.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.erd._card_height" data-kind="function" class="blk"><title>_card_height — function, 3 lines</title></rect>
<rect x="438.0" y="89.0" width="74.0" height="10.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.erd.layout" data-kind="function" class="blk"><title>layout — function, 20 lines</title></rect>
<rect x="438.0" y="100.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.erd._field_y" data-kind="function" class="blk"><title>_field_y — function, 3 lines</title></rect>
<rect x="438.0" y="102.5" width="74.0" height="73.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.erd.emit" data-kind="function" class="blk"><title>emit — function, 146 lines</title></rect>
<rect x="520.0" y="46.0" width="74.0" height="82.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.index — 164 lines</title></rect>
<rect x="520.0" y="63.0" width="74.0" height="4.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.index._tree" data-kind="function" class="blk"><title>_tree — function, 8 lines</title></rect>
<rect x="520.0" y="68.0" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.index._anchor" data-kind="function" class="blk"><title>_anchor — function, 5 lines</title></rect>
<rect x="520.0" y="71.5" width="74.0" height="42.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.index.to_markdown" data-kind="function" class="blk"><title>to_markdown — function, 84 lines</title></rect>
<rect x="527.0" y="83.0" width="60.0" height="13.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.index.to_markdown.walk" data-kind="function" class="blk"><title>walk — function, 27 lines</title></rect>
<rect x="520.0" y="114.5" width="74.0" height="13.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.index.to_sidebar" data-kind="function" class="blk"><title>to_sidebar — function, 26 lines</title></rect>
<rect x="527.0" y="118.0" width="60.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.emitters.index.to_sidebar.build" data-kind="function" class="blk"><title>build — function, 13 lines</title></rect>
<rect x="602.0" y="46.0" width="74.0" height="43.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.cli_dot — 87 lines</title></rect>
<rect x="602.0" y="52.5" width="74.0" height="36.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.cli_dot.main" data-kind="function" class="blk"><title>main — function, 73 lines</title></rect>
<rect x="684.0" y="46.0" width="74.0" height="40.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.auto — 80 lines</title></rect>
<rect x="684.0" y="57.5" width="74.0" height="28.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.auto.main" data-kind="function" class="blk"><title>main — function, 56 lines</title></rect>
<rect x="766.0" y="46.0" width="74.0" height="37.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.cli_notebook — 74 lines</title></rect>
<rect x="766.0" y="54.0" width="74.0" height="28.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.cli_notebook.main" data-kind="function" class="blk"><title>main — function, 57 lines</title></rect>
<rect x="28.0" y="331.0" width="74.0" height="37.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.cli_site — 74 lines</title></rect>
<rect x="28.0" y="339.0" width="74.0" height="28.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.cli_site.main" data-kind="function" class="blk"><title>main — function, 57 lines</title></rect>
<rect x="110.0" y="331.0" width="74.0" height="25.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.cli_erd — 51 lines</title></rect>
<rect x="110.0" y="337.0" width="74.0" height="19.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.cli_erd.main" data-kind="function" class="blk"><title>main — function, 38 lines</title></rect>
<rect x="192.0" y="331.0" width="74.0" height="25.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.cli_minimap — 51 lines</title></rect>
<rect x="192.0" y="337.5" width="74.0" height="18.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.cli_minimap.main" data-kind="function" class="blk"><title>main — function, 37 lines</title></rect>
<rect x="274.0" y="331.0" width="74.0" height="24.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.cli_explore — 48 lines</title></rect>
<rect x="274.0" y="337.0" width="74.0" height="17.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.cli_explore.main" data-kind="function" class="blk"><title>main — function, 35 lines</title></rect>
<rect x="356.0" y="331.0" width="74.0" height="22.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.cli_index — 44 lines</title></rect>
<rect x="356.0" y="336.5" width="74.0" height="16.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.cli_index.main" data-kind="function" class="blk"><title>main — function, 32 lines</title></rect>
<rect x="438.0" y="331.0" width="74.0" height="18.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters.__main__ — 37 lines</title></rect>
<rect x="438.0" y="333.5" width="74.0" height="13.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.emitters.__main__.main" data-kind="function" class="blk"><title>main — function, 27 lines</title></rect>
<rect x="538.0" y="331.0" width="74.0" height="1007.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.selftest — 2014 lines</title></rect>
<rect x="538.0" y="375.5" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest.check" data-kind="function" class="blk"><title>check — function, 5 lines</title></rect>
<rect x="538.0" y="379.0" width="74.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest._err" data-kind="function" class="blk"><title>_err — function, 7 lines</title></rect>
<rect x="538.0" y="383.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest.skip" data-kind="function" class="blk"><title>skip — function, 3 lines</title></rect>
<rect x="538.0" y="386.0" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest.build_tree" data-kind="function" class="blk"><title>build_tree — function, 5 lines</title></rect>
<rect x="538.0" y="515.5" width="74.0" height="97.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest._harvesting" data-kind="function" class="blk"><title>_harvesting — function, 194 lines</title></rect>
<rect x="538.0" y="858.0" width="74.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest._entry" data-kind="function" class="blk"><title>_entry — function, 7 lines</title></rect>
<rect x="538.0" y="1111.0" width="74.0" height="139.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest._books" data-kind="function" class="blk"><title>_books — function, 279 lines</title></rect>
<rect x="538.0" y="1255.5" width="74.0" height="76.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.selftest._standalone" data-kind="function" class="blk"><title>_standalone — function, 153 lines</title></rect>
<rect x="620.0" y="331.0" width="74.0" height="158.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.book — 316 lines</title></rect>
<rect x="620.0" y="360.0" width="74.0" height="10.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.book.Step" data-kind="class" class="blk"><title>Step — class, 20 lines</title></rect>
<rect x="627.0" y="365.0" width="60.0" height="5.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Step.to_dict" data-kind="function" class="blk"><title>to_dict — function, 10 lines</title></rect>
<rect x="620.0" y="381.5" width="74.0" height="15.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book._unit_counts" data-kind="function" class="blk"><title>_unit_counts — function, 30 lines</title></rect>
<rect x="620.0" y="397.5" width="74.0" height="89.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.book.Book" data-kind="class" class="blk"><title>Book — class, 178 lines</title></rect>
<rect x="627.0" y="399.0" width="60.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Book.__init__" data-kind="function" class="blk"><title>__init__ — function, 17 lines</title></rect>
<rect x="627.0" y="409.0" width="60.0" height="7.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Book.step" data-kind="function" class="blk"><title>step — function, 14 lines</title></rect>
<rect x="627.0" y="417.5" width="60.0" height="10.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Book.measure" data-kind="function" class="blk"><title>measure — function, 21 lines</title></rect>
<rect x="627.0" y="428.5" width="60.0" height="31.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Book.compare" data-kind="function" class="blk"><title>compare — function, 62 lines</title></rect>
<rect x="627.0" y="461.0" width="60.0" height="13.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Book.close" data-kind="function" class="blk"><title>close — function, 27 lines</title></rect>
<rect x="627.0" y="475.0" width="60.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Book.to_dict" data-kind="function" class="blk"><title>to_dict — function, 17 lines</title></rect>
<rect x="627.0" y="484.0" width="60.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.Book.write" data-kind="function" class="blk"><title>write — function, 5 lines</title></rect>
<rect x="620.0" y="487.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book._tree_bytes" data-kind="function" class="blk"><title>_tree_bytes — function, 2 lines</title></rect>
<rect x="702.0" y="331.0" width="74.0" height="97.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.style — 194 lines</title></rect>
<rect x="702.0" y="353.5" width="74.0" height="2.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.style.StyleError" data-kind="class" class="blk"><title>StyleError — class, 2 lines</title></rect>
<rect x="702.0" y="355.5" width="74.0" height="66.5" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.style.Style" data-kind="class" class="blk"><title>Style — class, 133 lines</title></rect>
<rect x="709.0" y="357.0" width="60.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.__init__" data-kind="function" class="blk"><title>__init__ — function, 17 lines</title></rect>
<rect x="709.0" y="367.5" width="60.0" height="6.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.load" data-kind="function" class="blk"><title>load — function, 12 lines</title></rect>
<rect x="709.0" y="374.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.available" data-kind="function" class="blk"><title>available — function, 2 lines</title></rect>
<rect x="709.0" y="376.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.themes" data-kind="function" class="blk"><title>themes — function, 2 lines</title></rect>
<rect x="709.0" y="378.5" width="60.0" height="16.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.validate" data-kind="function" class="blk"><title>validate — function, 32 lines</title></rect>
<rect x="709.0" y="396.0" width="60.0" height="6.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style._resolve" data-kind="function" class="blk"><title>_resolve — function, 12 lines</title></rect>
<rect x="709.0" y="402.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style._lookup" data-kind="function" class="blk"><title>_lookup — function, 3 lines</title></rect>
<rect x="709.0" y="404.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.node" data-kind="function" class="blk"><title>node — function, 2 lines</title></rect>
<rect x="709.0" y="406.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.group" data-kind="function" class="blk"><title>group — function, 2 lines</title></rect>
<rect x="709.0" y="407.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.edge" data-kind="function" class="blk"><title>edge — function, 2 lines</title></rect>
<rect x="709.0" y="409.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.graph" data-kind="function" class="blk"><title>graph — function, 2 lines</title></rect>
<rect x="709.0" y="410.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.geom" data-kind="function" class="blk"><title>geom — function, 2 lines</title></rect>
<rect x="709.0" y="412.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.slot" data-kind="function" class="blk"><title>slot — function, 2 lines</title></rect>
<rect x="709.0" y="413.5" width="60.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.domain_slot" data-kind="function" class="blk"><title>domain_slot — function, 13 lines</title></rect>
<rect x="709.0" y="420.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.Style.limits" data-kind="function" class="blk"><title>limits — function, 3 lines</title></rect>
<rect x="702.0" y="423.0" width="74.0" height="4.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.harvest" data-kind="function" class="blk"><title>harvest — function, 9 lines</title></rect>
<rect x="784.0" y="331.0" width="74.0" height="54.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.reference — 108 lines</title></rect>
<rect x="784.0" y="354.0" width="74.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.reference._candidates" data-kind="function" class="blk"><title>_candidates — function, 7 lines</title></rect>
<rect x="784.0" y="358.5" width="74.0" height="4.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.reference.root" data-kind="function" class="blk"><title>root — function, 9 lines</title></rect>
<rect x="784.0" y="364.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.reference.station_tools" data-kind="function" class="blk"><title>station_tools — function, 4 lines</title></rect>
<rect x="784.0" y="367.0" width="74.0" height="5.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.reference.describe" data-kind="function" class="blk"><title>describe — function, 10 lines</title></rect>
<rect x="784.0" y="373.0" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.reference.on_path" data-kind="function" class="blk"><title>on_path — function, 13 lines</title></rect>
<rect x="784.0" y="380.5" width="74.0" height="4.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.reference.missing" data-kind="function" class="blk"><title>missing — function, 8 lines</title></rect>
<rect x="28.0" y="1376.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.ops — 28 lines</title></rect>
<rect x="110.0" y="1376.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.notebook — 15 lines</title></rect>
<rect x="192.0" y="1376.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.emitters — 12 lines</title></rect>
<rect x="274.0" y="1376.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.lab — 11 lines</title></rect>
<rect x="356.0" y="1376.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.ir — 7 lines</title></rect>
<rect x="438.0" y="1376.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors — 2 lines</title></rect>
<rect x="538.0" y="1376.0" width="74.0" height="141.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.code — 282 lines</title></rect>
<rect x="538.0" y="1423.5" width="74.0" height="2.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.extractors.code.MissingParser" data-kind="class" class="blk"><title>MissingParser — class, 2 lines</title></rect>
<rect x="538.0" y="1425.5" width="74.0" height="10.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.code._parser" data-kind="function" class="blk"><title>_parser — function, 21 lines</title></rect>
<rect x="538.0" y="1437.0" width="74.0" height="5.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.code._name" data-kind="function" class="blk"><title>_name — function, 10 lines</title></rect>
<rect x="538.0" y="1443.0" width="74.0" height="17.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.code._walk" data-kind="function" class="blk"><title>_walk — function, 34 lines</title></rect>
<rect x="538.0" y="1461.0" width="74.0" height="13.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.code.extract_file" data-kind="function" class="blk"><title>extract_file — function, 27 lines</title></rect>
<rect x="538.0" y="1475.5" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.code._count_errors" data-kind="function" class="blk"><title>_count_errors — function, 5 lines</title></rect>
<rect x="538.0" y="1479.0" width="74.0" height="37.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.code.extract" data-kind="function" class="blk"><title>extract — function, 75 lines</title></rect>
<rect x="620.0" y="1376.0" width="74.0" height="134.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.usage — 269 lines</title></rect>
<rect x="620.0" y="1408.0" width="74.0" height="13.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.usage._template" data-kind="function" class="blk"><title>_template — function, 27 lines</title></rect>
<rect x="620.0" y="1422.5" width="74.0" height="5.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.usage._body" data-kind="function" class="blk"><title>_body — function, 10 lines</title></rect>
<rect x="620.0" y="1428.5" width="74.0" height="7.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.usage._shape" data-kind="function" class="blk"><title>_shape — function, 15 lines</title></rect>
<rect x="620.0" y="1437.0" width="74.0" height="5.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.usage._graphql" data-kind="function" class="blk"><title>_graphql — function, 11 lines</title></rect>
<rect x="620.0" y="1443.5" width="74.0" height="66.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.usage.extract" data-kind="function" class="blk"><title>extract — function, 133 lines</title></rect>
<rect x="702.0" y="1376.0" width="74.0" height="95.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.openapi — 191 lines</title></rect>
<rect x="702.0" y="1391.5" width="74.0" height="14.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.openapi._modelgen" data-kind="function" class="blk"><title>_modelgen — function, 28 lines</title></rect>
<rect x="702.0" y="1406.5" width="74.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.openapi._type_name" data-kind="function" class="blk"><title>_type_name — function, 6 lines</title></rect>
<rect x="702.0" y="1410.5" width="74.0" height="21.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.openapi._refs" data-kind="function" class="blk"><title>_refs — function, 43 lines</title></rect>
<rect x="702.0" y="1433.0" width="74.0" height="38.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.openapi.extract" data-kind="function" class="blk"><title>extract — function, 76 lines</title></rect>
<rect x="784.0" y="1376.0" width="74.0" height="80.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.db — 161 lines</title></rect>
<rect x="784.0" y="1395.5" width="74.0" height="41.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.db.from_schema_dict" data-kind="function" class="blk"><title>from_schema_dict — function, 82 lines</title></rect>
<rect x="784.0" y="1437.5" width="74.0" height="5.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.db._relation" data-kind="function" class="blk"><title>_relation — function, 10 lines</title></rect>
<rect x="784.0" y="1443.5" width="74.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.db._plain_type" data-kind="function" class="blk"><title>_plain_type — function, 6 lines</title></rect>
<rect x="784.0" y="1447.5" width="74.0" height="4.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.db._dedupe" data-kind="function" class="blk"><title>_dedupe — function, 9 lines</title></rect>
<rect x="784.0" y="1453.0" width="74.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.db.extract" data-kind="function" class="blk"><title>extract — function, 6 lines</title></rect>
<rect x="28.0" y="1555.0" width="74.0" height="20.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.code_main — 40 lines</title></rect>
<rect x="28.0" y="1559.0" width="74.0" height="15.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.code_main.main" data-kind="function" class="blk"><title>main — function, 31 lines</title></rect>
<rect x="110.0" y="1555.0" width="74.0" height="18.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.python — 37 lines</title></rect>
<rect x="110.0" y="1564.5" width="74.0" height="7.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.extract" data-kind="function" class="blk"><title>extract — function, 14 lines</title></rect>
<rect x="192.0" y="1555.0" width="74.0" height="16.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.usage_main — 33 lines</title></rect>
<rect x="192.0" y="1559.0" width="74.0" height="12.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.usage_main.main" data-kind="function" class="blk"><title>main — function, 24 lines</title></rect>
<rect x="274.0" y="1555.0" width="74.0" height="16.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.db_main — 32 lines</title></rect>
<rect x="274.0" y="1560.0" width="74.0" height="10.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.db_main.main" data-kind="function" class="blk"><title>main — function, 21 lines</title></rect>
<rect x="356.0" y="1555.0" width="74.0" height="15.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.openapi_main — 30 lines</title></rect>
<rect x="356.0" y="1559.0" width="74.0" height="10.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.openapi_main.main" data-kind="function" class="blk"><title>main — function, 21 lines</title></rect>
<rect x="438.0" y="1555.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.__main__ — 26 lines</title></rect>
<rect x="438.0" y="1557.7" width="74.0" height="8.6" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.__main__.main" data-kind="function" class="blk"><title>main — function, 16 lines</title></rect>
<rect x="538.0" y="1555.0" width="74.0" height="175.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.book.build — 351 lines</title></rect>
<rect x="538.0" y="1592.5" width="74.0" height="13.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.build.extract" data-kind="function" class="blk"><title>extract — function, 27 lines</title></rect>
<rect x="538.0" y="1607.0" width="74.0" height="34.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.build.spec_from" data-kind="function" class="blk"><title>spec_from — function, 68 lines</title></rect>
<rect x="538.0" y="1642.0" width="74.0" height="22.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.build._load_cell" data-kind="function" class="blk"><title>_load_cell — function, 44 lines</title></rect>
<rect x="538.0" y="1665.0" width="74.0" height="65.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.build.run" data-kind="function" class="blk"><title>run — function, 130 lines</title></rect>
<rect x="545.0" y="1671.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.build.run.say" data-kind="function" class="blk"><title>say — function, 3 lines</title></rect>
<rect x="620.0" y="1555.0" width="74.0" height="153.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.book.checks — 306 lines</title></rect>
<rect x="620.0" y="1585.5" width="74.0" height="13.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.book.checks.Loaded" data-kind="class" class="blk"><title>Loaded — class, 26 lines</title></rect>
<rect x="627.0" y="1587.0" width="60.0" height="7.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Loaded.__init__" data-kind="function" class="blk"><title>__init__ — function, 15 lines</title></rect>
<rect x="627.0" y="1595.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Loaded.larder" data-kind="function" class="blk"><title>larder — function, 2 lines</title></rect>
<rect x="627.0" y="1597.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Loaded.measure" data-kind="function" class="blk"><title>measure — function, 2 lines</title></rect>
<rect x="620.0" y="1599.5" width="74.0" height="18.5" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.book.checks.Report" data-kind="class" class="blk"><title>Report — class, 37 lines</title></rect>
<rect x="627.0" y="1601.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Report.__init__" data-kind="function" class="blk"><title>__init__ — function, 2 lines</title></rect>
<rect x="627.0" y="1602.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Report.note" data-kind="function" class="blk"><title>note — function, 2 lines</title></rect>
<rect x="627.0" y="1604.0" width="60.0" height="4.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Report.check" data-kind="function" class="blk"><title>check — function, 8 lines</title></rect>
<rect x="627.0" y="1608.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Report.skip" data-kind="function" class="blk"><title>skip — function, 3 lines</title></rect>
<rect x="627.0" y="1610.5" width="60.0" height="7.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.checks.Report.total" data-kind="function" class="blk"><title>total — function, 15 lines</title></rect>
<rect x="620.0" y="1619.0" width="74.0" height="41.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.checks.generated" data-kind="function" class="blk"><title>generated — function, 83 lines</title></rect>
<rect x="620.0" y="1661.5" width="74.0" height="13.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.checks.custom" data-kind="function" class="blk"><title>custom — function, 26 lines</title></rect>
<rect x="620.0" y="1675.5" width="74.0" height="16.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.checks._run_cells" data-kind="function" class="blk"><title>_run_cells — function, 33 lines</title></rect>
<rect x="620.0" y="1693.0" width="74.0" height="12.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.checks.main" data-kind="function" class="blk"><title>main — function, 25 lines</title></rect>
<rect x="702.0" y="1555.0" width="74.0" height="98.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.book.larder — 196 lines</title></rect>
<rect x="702.0" y="1586.5" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.larder._count" data-kind="function" class="blk"><title>_count — function, 13 lines</title></rect>
<rect x="702.0" y="1594.0" width="74.0" height="16.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.larder.redact" data-kind="function" class="blk"><title>redact — function, 32 lines</title></rect>
<rect x="702.0" y="1611.5" width="74.0" height="38.5" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.book.larder.Larder" data-kind="class" class="blk"><title>Larder — class, 77 lines</title></rect>
<rect x="709.0" y="1618.5" width="60.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.larder.Larder.__post_init__" data-kind="function" class="blk"><title>__post_init__ — function, 7 lines</title></rect>
<rect x="709.0" y="1623.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.larder.Larder.read" data-kind="function" class="blk"><title>read — function, 3 lines</title></rect>
<rect x="709.0" y="1625.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.larder.Larder.fail" data-kind="function" class="blk"><title>fail — function, 3 lines</title></rect>
<rect x="709.0" y="1627.0" width="60.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.larder.Larder.to_dict" data-kind="function" class="blk"><title>to_dict — function, 13 lines</title></rect>
<rect x="709.0" y="1634.5" width="60.0" height="5.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.larder.Larder.from_dict" data-kind="function" class="blk"><title>from_dict — function, 10 lines</title></rect>
<rect x="709.0" y="1640.0" width="60.0" height="10.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.book.larder.Larder.line" data-kind="function" class="blk"><title>line — function, 20 lines</title></rect>
<rect x="702.0" y="1651.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.larder.of" data-kind="function" class="blk"><title>of — function, 3 lines</title></rect>
<rect x="784.0" y="1555.0" width="74.0" height="40.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.book.__main__ — 81 lines</title></rect>
<rect x="784.0" y="1562.0" width="74.0" height="31.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.book.__main__.main" data-kind="function" class="blk"><title>main — function, 62 lines</title></rect>
<rect x="28.0" y="1768.5" width="74.0" height="118.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.python.collect — 236 lines</title></rect>
<rect x="28.0" y="1785.0" width="74.0" height="5.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect.Definition" data-kind="class" class="blk"><title>Definition — class, 10 lines</title></rect>
<rect x="28.0" y="1791.5" width="74.0" height="6.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect.Module" data-kind="class" class="blk"><title>Module — class, 12 lines</title></rect>
<rect x="28.0" y="1798.5" width="74.0" height="33.5" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect._Collector" data-kind="class" class="blk"><title>_Collector — class, 67 lines</title></rect>
<rect x="35.0" y="1800.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.extractors.python.collect._Collector.__init__" data-kind="function" class="blk"><title>__init__ — function, 3 lines</title></rect>
<rect x="35.0" y="1803.0" width="60.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.extractors.python.collect._Collector._define" data-kind="function" class="blk"><title>_define — function, 17 lines</title></rect>
<rect x="35.0" y="1812.0" width="60.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.extractors.python.collect._Collector.visit_ClassDef" data-kind="function" class="blk"><title>visit_ClassDef — function, 5 lines</title></rect>
<rect x="35.0" y="1815.0" width="60.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.extractors.python.collect._Collector.visit_FunctionDef" data-kind="function" class="blk"><title>visit_FunctionDef — function, 5 lines</title></rect>
<rect x="35.0" y="1820.5" width="60.0" height="4.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.extractors.python.collect._Collector.visit_Import" data-kind="function" class="blk"><title>visit_Import — function, 8 lines</title></rect>
<rect x="35.0" y="1825.0" width="60.0" height="7.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.extractors.python.collect._Collector.visit_ImportFrom" data-kind="function" class="blk"><title>visit_ImportFrom — function, 14 lines</title></rect>
<rect x="28.0" y="1833.0" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect._first_line" data-kind="function" class="blk"><title>_first_line — function, 5 lines</title></rect>
<rect x="28.0" y="1836.5" width="74.0" height="7.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect._name_of" data-kind="function" class="blk"><title>_name_of — function, 15 lines</title></rect>
<rect x="28.0" y="1845.0" width="74.0" height="8.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect._resolve_relative" data-kind="function" class="blk"><title>_resolve_relative — function, 16 lines</title></rect>
<rect x="28.0" y="1854.0" width="74.0" height="13.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect.module_name" data-kind="function" class="blk"><title>module_name — function, 26 lines</title></rect>
<rect x="28.0" y="1868.0" width="74.0" height="10.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect.collect_file" data-kind="function" class="blk"><title>collect_file — function, 21 lines</title></rect>
<rect x="28.0" y="1879.5" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.collect.collect" data-kind="function" class="blk"><title>collect — function, 13 lines</title></rect>
<rect x="110.0" y="1768.5" width="74.0" height="92.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.python.resolve — 184 lines</title></rect>
<rect x="110.0" y="1782.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.resolve._id_for" data-kind="function" class="blk"><title>_id_for — function, 2 lines</title></rect>
<rect x="110.0" y="1784.5" width="74.0" height="16.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.resolve._resolve" data-kind="function" class="blk"><title>_resolve — function, 32 lines</title></rect>
<rect x="110.0" y="1801.5" width="74.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.resolve.larder_of" data-kind="function" class="blk"><title>larder_of — function, 17 lines</title></rect>
<rect x="110.0" y="1811.0" width="74.0" height="49.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.resolve.to_ir" data-kind="function" class="blk"><title>to_ir — function, 98 lines</title></rect>
<rect x="117.0" y="1840.5" width="60.0" height="4.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.extractors.python.resolve.to_ir._point_at" data-kind="function" class="blk"><title>_point_at — function, 9 lines</title></rect>
<rect x="192.0" y="1768.5" width="74.0" height="19.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.extractors.python.__main__ — 38 lines</title></rect>
<rect x="192.0" y="1773.5" width="74.0" height="11.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.extractors.python.__main__.main" data-kind="function" class="blk"><title>main — function, 23 lines</title></rect>
<rect x="292.0" y="1768.5" width="74.0" height="156.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.ir.validate — 313 lines</title></rect>
<rect x="292.0" y="1790.0" width="74.0" height="2.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.ir.validate.IRError" data-kind="class" class="blk"><title>IRError — class, 2 lines</title></rect>
<rect x="292.0" y="1792.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate._schema" data-kind="function" class="blk"><title>_schema — function, 2 lines</title></rect>
<rect x="292.0" y="1794.0" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate._props" data-kind="function" class="blk"><title>_props — function, 5 lines</title></rect>
<rect x="292.0" y="1797.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate._fields" data-kind="function" class="blk"><title>_fields — function, 4 lines</title></rect>
<rect x="292.0" y="1800.5" width="74.0" height="54.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate.check" data-kind="function" class="blk"><title>check — function, 108 lines</title></rect>
<rect x="292.0" y="1862.0" width="74.0" height="29.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate._check_larder" data-kind="function" class="blk"><title>_check_larder — function, 58 lines</title></rect>
<rect x="292.0" y="1892.0" width="74.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate.validate" data-kind="function" class="blk"><title>validate — function, 6 lines</title></rect>
<rect x="292.0" y="1896.0" width="74.0" height="11.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate.check_model_matches_schema" data-kind="function" class="blk"><title>check_model_matches_schema — function, 23 lines</title></rect>
<rect x="292.0" y="1908.5" width="74.0" height="16.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ir.validate.main" data-kind="function" class="blk"><title>main — function, 32 lines</title></rect>
<rect x="374.0" y="1768.5" width="74.0" height="82.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.ir.model — 165 lines</title></rect>
<rect x="374.0" y="1785.0" width="74.0" height="19.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.ir.model.Meta" data-kind="class" class="blk"><title>Meta — class, 38 lines</title></rect>
<rect x="381.0" y="1795.0" width="60.0" height="9.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Meta.to_dict" data-kind="function" class="blk"><title>to_dict — function, 18 lines</title></rect>
<rect x="374.0" y="1805.5" width="74.0" height="10.5" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.ir.model.Node" data-kind="class" class="blk"><title>Node — class, 21 lines</title></rect>
<rect x="381.0" y="1810.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Node.__post_init__" data-kind="function" class="blk"><title>__post_init__ — function, 3 lines</title></rect>
<rect x="381.0" y="1812.0" width="60.0" height="4.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Node.to_dict" data-kind="function" class="blk"><title>to_dict — function, 8 lines</title></rect>
<rect x="374.0" y="1817.5" width="74.0" height="7.5" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.ir.model.Edge" data-kind="class" class="blk"><title>Edge — class, 15 lines</title></rect>
<rect x="381.0" y="1821.5" width="60.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Edge.to_dict" data-kind="function" class="blk"><title>to_dict — function, 7 lines</title></rect>
<rect x="374.0" y="1826.5" width="74.0" height="24.0" rx="2" fill="#1d4ed8" stroke="none" opacity="0.95" data-id="docgen.ir.model.Graph" data-kind="class" class="blk"><title>Graph — class, 48 lines</title></rect>
<rect x="381.0" y="1831.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Graph.node" data-kind="function" class="blk"><title>node — function, 4 lines</title></rect>
<rect x="381.0" y="1833.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Graph.edge" data-kind="function" class="blk"><title>edge — function, 4 lines</title></rect>
<rect x="381.0" y="1836.0" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Graph.has" data-kind="function" class="blk"><title>has — function, 2 lines</title></rect>
<rect x="381.0" y="1838.5" width="60.0" height="8.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Graph.to_dict" data-kind="function" class="blk"><title>to_dict — function, 16 lines</title></rect>
<rect x="381.0" y="1847.5" width="60.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ir.model.Graph.from_dict" data-kind="function" class="blk"><title>from_dict — function, 6 lines</title></rect>
<rect x="456.0" y="1768.5" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.ir.__main__ — 9 lines</title></rect>
<rect x="556.0" y="1768.5" width="74.0" height="242.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.ops.filter — 485 lines</title></rect>
<rect x="556.0" y="1786.5" width="74.0" height="28.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter._rebuild" data-kind="function" class="blk"><title>_rebuild — function, 57 lines</title></rect>
<rect x="563.0" y="1791.0" width="60.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ops.filter._rebuild.surviving_parent" data-kind="function" class="blk"><title>surviving_parent — function, 5 lines</title></rect>
<rect x="563.0" y="1798.0" width="60.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ops.filter._rebuild.lift" data-kind="function" class="blk"><title>lift — function, 6 lines</title></rect>
<rect x="556.0" y="1816.0" width="74.0" height="7.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.drop_kinds" data-kind="function" class="blk"><title>drop_kinds — function, 14 lines</title></rect>
<rect x="556.0" y="1824.0" width="74.0" height="7.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.only_kinds" data-kind="function" class="blk"><title>only_kinds — function, 14 lines</title></rect>
<rect x="556.0" y="1832.0" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.drop_stdlib" data-kind="function" class="blk"><title>drop_stdlib — function, 13 lines</title></rect>
<rect x="556.0" y="1839.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.drop_external" data-kind="function" class="blk"><title>drop_external — function, 3 lines</title></rect>
<rect x="556.0" y="1842.0" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.subtree" data-kind="function" class="blk"><title>subtree — function, 13 lines</title></rect>
<rect x="556.0" y="1849.5" width="74.0" height="22.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.neighbourhood" data-kind="function" class="blk"><title>neighbourhood — function, 45 lines</title></rect>
<rect x="556.0" y="1873.0" width="74.0" height="9.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.collapse_to_depth" data-kind="function" class="blk"><title>collapse_to_depth — function, 18 lines</title></rect>
<rect x="563.0" y="1878.0" width="60.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ops.filter.collapse_to_depth.level" data-kind="function" class="blk"><title>level — function, 6 lines</title></rect>
<rect x="556.0" y="1883.0" width="74.0" height="7.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.drop_builtins" data-kind="function" class="blk"><title>drop_builtins — function, 14 lines</title></rect>
<rect x="556.0" y="1891.0" width="74.0" height="17.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.overview" data-kind="function" class="blk"><title>overview — function, 35 lines</title></rect>
<rect x="556.0" y="1909.5" width="74.0" height="36.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.shape" data-kind="function" class="blk"><title>shape — function, 72 lines</title></rect>
<rect x="563.0" y="1933.5" width="60.0" height="4.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.ops.filter.shape.rank_of" data-kind="function" class="blk"><title>rank_of — function, 9 lines</title></rect>
<rect x="556.0" y="1946.5" width="74.0" height="12.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.split" data-kind="function" class="blk"><title>split — function, 24 lines</title></rect>
<rect x="556.0" y="1959.5" width="74.0" height="51.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.filter.classify" data-kind="function" class="blk"><title>classify — function, 102 lines</title></rect>
<rect x="638.0" y="1768.5" width="74.0" height="51.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.ops.__main__ — 102 lines</title></rect>
<rect x="638.0" y="1775.0" width="74.0" height="42.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.ops.__main__.main" data-kind="function" class="blk"><title>main — function, 84 lines</title></rect>
<rect x="738.0" y="1768.5" width="74.0" height="111.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.style.extract — 222 lines</title></rect>
<rect x="738.0" y="1799.0" width="74.0" height="8.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.extract._values_from" data-kind="function" class="blk"><title>_values_from — function, 17 lines</title></rect>
<rect x="738.0" y="1808.5" width="74.0" height="22.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.extract._normalise" data-kind="function" class="blk"><title>_normalise — function, 45 lines</title></rect>
<rect x="738.0" y="1832.0" width="74.0" height="12.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.extract._svg_files" data-kind="function" class="blk"><title>_svg_files — function, 25 lines</title></rect>
<rect x="738.0" y="1845.5" width="74.0" height="21.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.extract.harvest" data-kind="function" class="blk"><title>harvest — function, 43 lines</title></rect>
<rect x="738.0" y="1868.0" width="74.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.extract.write" data-kind="function" class="blk"><title>write — function, 7 lines</title></rect>
<rect x="738.0" y="1872.5" width="74.0" height="6.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.extract.summarise" data-kind="function" class="blk"><title>summarise — function, 13 lines</title></rect>
<rect x="28.0" y="2049.0" width="74.0" height="105.5" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.style.tokens — 211 lines</title></rect>
<rect x="28.0" y="2077.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.tokens._top" data-kind="function" class="blk"><title>_top — function, 2 lines</title></rect>
<rect x="28.0" y="2079.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.tokens._mode" data-kind="function" class="blk"><title>_mode — function, 3 lines</title></rect>
<rect x="28.0" y="2081.5" width="74.0" height="3.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.tokens._luminance" data-kind="function" class="blk"><title>_luminance — function, 6 lines</title></rect>
<rect x="28.0" y="2085.5" width="74.0" height="50.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.tokens.derive" data-kind="function" class="blk"><title>derive — function, 101 lines</title></rect>
<rect x="35.0" y="2107.5" width="60.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.style.tokens.derive.accent" data-kind="function" class="blk"><title>accent — function, 2 lines</title></rect>
<rect x="28.0" y="2137.0" width="74.0" height="7.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.tokens.geometry" data-kind="function" class="blk"><title>geometry — function, 14 lines</title></rect>
<rect x="28.0" y="2145.0" width="74.0" height="3.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.tokens.write" data-kind="function" class="blk"><title>write — function, 7 lines</title></rect>
<rect x="28.0" y="2149.5" width="74.0" height="4.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.style.tokens.from_folder" data-kind="function" class="blk"><title>from_folder — function, 9 lines</title></rect>
<rect x="128.0" y="2049.0" width="74.0" height="14.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen — 2 lines</title></rect>
<rect x="228.0" y="2049.0" width="74.0" height="75.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.lab.pg_probe — 150 lines</title></rect>
<rect x="228.0" y="2084.5" width="74.0" height="15.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.lab.pg_probe.probe" data-kind="function" class="blk"><title>probe — function, 30 lines</title></rect>
<rect x="228.0" y="2106.5" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.lab.pg_probe._simplify" data-kind="function" class="blk"><title>_simplify — function, 3 lines</title></rect>
<rect x="228.0" y="2109.0" width="74.0" height="12.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.lab.pg_probe.main" data-kind="function" class="blk"><title>main — function, 25 lines</title></rect>
<rect x="328.0" y="2049.0" width="74.0" height="132.0" rx="2" fill="#141414" stroke="#333333" stroke-width="1"><title>docgen.notebook.spec — 264 lines</title></rect>
<rect x="328.0" y="2077.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.notebook.spec._step" data-kind="function" class="blk"><title>_step — function, 4 lines</title></rect>
<rect x="328.0" y="2080.0" width="74.0" height="54.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.notebook.spec.from_ir" data-kind="function" class="blk"><title>from_ir — function, 108 lines</title></rect>
<rect x="335.0" y="2087.0" width="60.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.8" data-id="docgen.notebook.spec.from_ir._order" data-kind="function" class="blk"><title>_order — function, 5 lines</title></rect>
<rect x="328.0" y="2135.0" width="74.0" height="10.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.notebook.spec.scaffold" data-kind="function" class="blk"><title>scaffold — function, 20 lines</title></rect>
<rect x="328.0" y="2146.0" width="74.0" height="29.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.notebook.spec.merge" data-kind="function" class="blk"><title>merge — function, 58 lines</title></rect>
<rect x="328.0" y="2176.0" width="74.0" height="2.0" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.notebook.spec.load" data-kind="function" class="blk"><title>load — function, 2 lines</title></rect>
<rect x="328.0" y="2178.0" width="74.0" height="2.5" rx="2" fill="#15803d" stroke="none" opacity="0.95" data-id="docgen.notebook.spec.dump" data-kind="function" class="blk"><title>dump — function, 5 lines</title></rect>
<text x="28" y="40" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.emitters</text>
<text x="28" y="325" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.emitters</text>
<text x="538" y="325" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen</text>
<text x="28" y="1370" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen</text>
<text x="538" y="1370" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.extractors</text>
<text x="28" y="1549" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.extractors</text>
<text x="538" y="1549" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.book</text>
<text x="28" y="1762" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.extractors.python</text>
<text x="292" y="1762" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.ir</text>
<text x="556" y="1762" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.ops</text>
<text x="738" y="1762" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.style</text>
<text x="28" y="2043" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.style</text>
<text x="128" y="2043" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">(root)</text>
<text x="228" y="2043" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.lab</text>
<text x="328" y="2043" font-family="Helvetica,sans-Serif" font-size="10" font-weight="bold" fill="#a3a3a3">docgen.notebook</text>
<text x="28" y="302" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">site</text>
<text x="110" y="213" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">explore</text>
<text x="192" y="201" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">dot</text>
<text x="274" y="194" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">notebook</text>
<text x="356" y="194" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">minimap</text>
<text x="438" y="185" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">erd</text>
<text x="520" y="137" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">index</text>
<text x="602" y="98" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">cli_dot</text>
<text x="684" y="95" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">auto</text>
<text x="766" y="92" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">cli_noteboo</text>
<text x="28" y="377" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">cli_site</text>
<text x="110" y="366" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">cli_erd</text>
<text x="192" y="366" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">cli_minimap</text>
<text x="274" y="364" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">cli_explore</text>
<text x="356" y="362" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">cli_index</text>
<text x="438" y="358" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">__main__</text>
<text x="538" y="1347" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">selftest</text>
<text x="620" y="498" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">book</text>
<text x="702" y="437" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">style</text>
<text x="784" y="394" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">reference</text>
<text x="28" y="1399" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">ops</text>
<text x="110" y="1399" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">notebook</text>
<text x="192" y="1399" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">emitters</text>
<text x="274" y="1399" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">lab</text>
<text x="356" y="1399" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">ir</text>
<text x="438" y="1399" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">extractors</text>
<text x="538" y="1526" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">code</text>
<text x="620" y="1520" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">usage</text>
<text x="702" y="1480" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">openapi</text>
<text x="784" y="1466" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">db</text>
<text x="28" y="1584" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">code_main</text>
<text x="110" y="1582" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">python</text>
<text x="192" y="1580" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">usage_main</text>
<text x="274" y="1580" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">db_main</text>
<text x="356" y="1579" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">openapi_mai</text>
<text x="438" y="1578" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">__main__</text>
<text x="538" y="1740" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">build</text>
<text x="620" y="1717" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">checks</text>
<text x="702" y="1662" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">larder</text>
<text x="784" y="1604" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">__main__</text>
<text x="28" y="1896" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">collect</text>
<text x="110" y="1870" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">resolve</text>
<text x="192" y="1796" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">__main__</text>
<text x="292" y="1934" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">validate</text>
<text x="374" y="1860" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">model</text>
<text x="456" y="1792" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">__main__</text>
<text x="556" y="2020" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">filter</text>
<text x="638" y="1828" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">__main__</text>
<text x="738" y="1888" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">extract</text>
<text x="28" y="2164" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">tokens</text>
<text x="128" y="2072" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">docgen</text>
<text x="228" y="2133" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">pg_probe</text>
<text x="328" y="2190" font-family="Helvetica,sans-Serif" font-size="7" fill="#666666">spec</text>
<rect x="28" y="2230.0" width="9" height="9" rx="2" fill="#1a1a1a"/>
<text x="41" y="2238.0" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">module</text>
<rect x="86" y="2230.0" width="9" height="9" rx="2" fill="#1d4ed8"/>
<text x="99" y="2238.0" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">class</text>
<rect x="138" y="2230.0" width="9" height="9" rx="2" fill="#d4a574"/>
<text x="151" y="2238.0" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">interface</text>
<rect x="214" y="2230.0" width="9" height="9" rx="2" fill="#15803d"/>
<text x="227" y="2238.0" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">function</text>
<text x="858" y="2238.0" text-anchor="end" font-family="Helvetica,sans-Serif" font-size="9" fill="#666666">53 files · 9,752 lines · 1px ≈ 2.0 lines</text>
</svg>

After

Width:  |  Height:  |  Size: 73 KiB

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,119 @@
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>docgen docs</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { background: #0a0a0a; overflow: hidden; width: 100vw; height: 100vh;
font-family: "Segoe UI", Inter, system-ui, -apple-system, Arial, sans-serif; }
#container { width: 100vw; height: 100vh; overflow: hidden; cursor: grab; }
#container.dragging { cursor: grabbing; }
img { transform-origin: 0 0; user-select: none; -webkit-user-drag: none; }
#hud {
position: fixed; bottom: 14px; left: 14px; display: flex; gap: 8px;
align-items: center; font-size: 11px; color: #a3a3a3;
background: #141414; border: 1px solid #333333;
border-radius: 6px; padding: 5px 9px; user-select: none;
}
#hud b { color: #e5e5e5; font-weight: 600; font-variant-numeric: tabular-nums; }
#hud span { opacity: .7; }
a.back { position: fixed; top: 14px; left: 14px; font-size: 11px;
color: #a3a3a3; text-decoration: none; background: #141414;
border: 1px solid #333333; border-radius: 6px; padding: 5px 9px; }
a.back:hover { color: #e5e5e5; }
</style>
</head>
<body>
<div id="container"><img id="img" alt=""></div>
<a class="back" href="index.html">&larr; docs</a>
<div id="hud"><b id="pct">100%</b><span id="mode">fit</span><span>&middot; click 1:1 &middot; drag &middot; wheel</span></div>
<script>
var src = new URLSearchParams(location.search).get('src');
var img = document.getElementById('img');
var container = document.getElementById('container');
var pct = document.getElementById('pct');
var modeEl = document.getElementById('mode');
if (src) { img.src = src; document.title = src + ' — docgen docs'; }
var scale = 1, x = 0, y = 0, fitScale = 1, mode = 'fit';
var dragging = false, moved = false, startX, startY, startPanX, startPanY;
function apply() {
img.style.transform = 'translate(' + x + 'px,' + y + 'px) scale(' + scale + ')';
pct.textContent = Math.round(scale * 100) + '%';
modeEl.textContent = mode;
}
function fit() {
var sw = window.innerWidth / img.naturalWidth;
var sh = window.innerHeight / img.naturalHeight;
fitScale = Math.min(sw, sh) * 0.95;
scale = fitScale;
x = (window.innerWidth - img.naturalWidth * scale) / 2;
y = (window.innerHeight - img.naturalHeight * scale) / 2;
mode = 'fit';
apply();
}
// Zoom about a point in the viewport, so what is under the cursor stays there.
function zoomAt(px, py, factor) {
x = px - (px - x) * factor;
y = py - (py - y) * factor;
scale *= factor;
mode = Math.abs(scale - fitScale) < 0.001 ? 'fit'
: (Math.abs(scale - 1) < 0.001 ? '1:1' : 'free');
apply();
}
img.onload = fit;
window.addEventListener('resize', function () { if (mode === 'fit') fit(); });
container.addEventListener('wheel', function (e) {
e.preventDefault();
var rect = container.getBoundingClientRect();
zoomAt(e.clientX - rect.left, e.clientY - rect.top, e.deltaY < 0 ? 1.12 : 0.89);
}, { passive: false });
container.addEventListener('mousedown', function (e) {
if (e.button !== 0) return;
dragging = true; moved = false;
startX = e.clientX; startY = e.clientY; startPanX = x; startPanY = y;
container.classList.add('dragging');
e.preventDefault();
});
window.addEventListener('mousemove', function (e) {
if (!dragging) return;
if (Math.abs(e.clientX - startX) > 3 || Math.abs(e.clientY - startY) > 3) moved = true;
x = startPanX + (e.clientX - startX);
y = startPanY + (e.clientY - startY);
apply();
});
window.addEventListener('mouseup', function (e) {
if (!dragging) return;
dragging = false;
container.classList.remove('dragging');
// A click that moved the mouse was a drag, and must not also toggle.
if (moved) return;
if (mode === '1:1') { fit(); return; }
// Toggle to actual size about the point clicked, so the thing you aimed at
// is the thing you end up looking at.
var rect = container.getBoundingClientRect();
zoomAt(e.clientX - rect.left, e.clientY - rect.top, 1 / scale);
mode = '1:1';
apply();
});
container.addEventListener('dblclick', fit);
window.addEventListener('keydown', function (e) {
if (e.key === '0' || e.key === 'f') fit();
if (e.key === '1') { var r = container.getBoundingClientRect();
zoomAt(r.width / 2, r.height / 2, 1 / scale); mode = '1:1'; apply(); }
if (e.key === 'Escape') location.href = 'index.html';
});
</script>
</body>
</html>

View File

@@ -0,0 +1,11 @@
"""
Emitters: IR -> an artifact. None of them has heard of Python, `ast` or SQL.
dot .dot -> Graphviz -> SVG static docs, embedding
index markdown / sidebar JSON no graph literacy required
diff two IRs -> what changed review
notebook .ipynb a runnable document
The non-visual ones matter most for reach. A sorted, described index of what
exists is readable by people who will never open a diagram.
"""

View File

@@ -0,0 +1,36 @@
""" python3 -m docgen.emitters <emitter> <ir.json> [options]"""
import sys
def main(argv=None):
argv = sys.argv[1:] if argv is None else argv
if not argv:
print("usage: python3 -m docgen.emitters <auto|dot|index|erd|notebook|site|minimap|explore> <ir.json> [-o OUT] [--style NAME] [--theme NAME]",
file=sys.stderr)
return 2
name, rest = argv[0], argv[1:]
if name == "dot":
from .cli_dot import main as run
elif name == "index":
from .cli_index import main as run
elif name == "erd":
from .cli_erd import main as run
elif name == "auto":
from .auto import main as run
elif name == "notebook":
from .cli_notebook import main as run
elif name == "site":
from .cli_site import main as run
elif name == "minimap":
from .cli_minimap import main as run
elif name == "explore":
from .cli_explore import main as run
else:
print(f"Error: no emitter {name!r} — have: auto, dot, index, erd, notebook, site, minimap, explore", file=sys.stderr)
return 1
return run(rest)
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,79 @@
"""
Draw it the way its structure asks to be drawn.
python3 -m docgen.emitters auto ir.json -o out/
`ops.classify` reads the structure and names an emitter; this runs it. The whole
point is that nobody should have to know that a schema wants cards and a module
graph wants ranks — or discover it from a 235:1 image.
When the answer is "this is not a diagram", it says so and writes the index,
because that *is* the right artifact for a flat list of peers.
"""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from ..ops import classify
from ..style import Style, StyleError
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters auto")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path, help="Directory to write into.")
p.add_argument("--style", default="lucid")
p.add_argument("--theme", default=None)
p.add_argument("--force", help="Use this emitter regardless of what fits.")
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)}):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
verdict = classify(data)
chosen = args.force or verdict["emitter"]
print(f" {verdict['kind']:<8} -> {chosen}")
print(f" {verdict['why']}")
out_dir = args.output or Path(".")
out_dir.mkdir(parents=True, exist_ok=True)
stem = args.ir.stem
try:
style = Style.load(args.style, theme=args.theme)
except StyleError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
if chosen == "erd":
from .erd import emit as erd_emit
path = out_dir / f"{stem}.svg"
path.write_text(erd_emit(data, style))
elif chosen == "index":
from .index import to_markdown
path = out_dir / f"{stem}.md"
path.write_text(to_markdown(data))
else:
from .dot import emit as dot_emit, render
path = out_dir / f"{stem}.svg"
opts = verdict.get("options") or {}
path.write_bytes(render(dot_emit(data, style, rankdir=opts.get("rankdir"))))
print(f" {path}")
return 0

View File

@@ -0,0 +1,86 @@
""" python3 -m docgen.emitters dot <ir.json> [-o out.svg] [--style lucid] [--theme dark]"""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from ..ops import shape
from ..style import Style, StyleError
from .dot import RenderError, emit, render
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters dot")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path, help="Write here. .dot or .svg by suffix.")
p.add_argument("--style", default="lucid")
p.add_argument("--theme", default=None)
p.add_argument("--max-depth", type=int, default=None)
p.add_argument("--quiet", "-q", action="store_true", help="Do not warn about shape.")
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)} problem(s)):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
try:
style = Style.load(args.style, theme=args.theme)
except StyleError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
# Aspect ratio is a property of the graph, not of the renderer: a layered
# engine puts one dependency level in one row, so the widest level is the
# width. Say so before writing the file, because the alternative is finding
# out from a 13671pt image — and the fix is never a layout flag, it is a
# smaller question.
if not args.quiet:
sh = shape(data)
if sh["widest_level"] > 20 or sh["nodes"] > 60:
est = sh["widest_level"] / max(sh["levels"], 1)
print(
f" note: {sh['nodes']} nodes, {sh['levels']} levels, widest level "
f"{sh['widest_level']} — this will render roughly {est:.0f}:1.",
file=sys.stderr,
)
print(
" Around 20 nodes is where it stops being a diagram. Try "
"`ops --split`,\n `--around <id> --hops 2`, or `--subtree <id>`. "
"Layout flags will not fix it.",
file=sys.stderr,
)
if sh["isolated"] > sh["nodes"] // 3:
print(
f" {sh['isolated']} of {sh['nodes']} nodes have no edges; they are "
"laid out side by side.",
file=sys.stderr,
)
dot_text = emit(data, style, max_depth=args.max_depth)
if not args.output:
sys.stdout.write(dot_text)
return 0
args.output.parent.mkdir(parents=True, exist_ok=True)
if args.output.suffix == ".dot":
args.output.write_text(dot_text)
else:
try:
args.output.write_bytes(render(dot_text, fmt=args.output.suffix.lstrip(".") or "svg"))
except RenderError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
print(f" {args.style}/{style.theme:6} {args.output}")
return 0

View File

@@ -0,0 +1,50 @@
""" python3 -m docgen.emitters erd <ir.json> [-o out.svg] [--theme dark]"""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from ..style import Style, StyleError
from .erd import emit
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters erd")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path)
p.add_argument("--style", default="lucid")
p.add_argument("--theme", default=None)
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)} problem(s)):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
try:
style = Style.load(args.style, theme=args.theme)
svg = emit(data, style)
except (StyleError, ValueError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(svg)
import re
m = re.search(r'width="(\d+)pt" height="(\d+)pt"', svg)
size = f"{m.group(1)}x{m.group(2)} {int(m.group(1))/int(m.group(2)):.1f}:1" if m else ""
print(f" erd/{style.theme:6} {args.output} {size}")
else:
sys.stdout.write(svg)
return 0

View File

@@ -0,0 +1,47 @@
""" python3 -m docgen.emitters explore <ir.json> -o DIR [--scale 0.55] [--hops 1]"""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from ..style import Style, StyleError
from .explore import write
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters explore")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path, required=True, help="Directory.")
p.add_argument("--style", default="lucid")
p.add_argument("--theme", default=None)
p.add_argument("--scale", type=float, default=0.55)
p.add_argument("--width", type=int, default=1100)
p.add_argument("--hops", type=int, default=1)
p.add_argument("--title", default="")
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)}):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
try:
style = Style.load(args.style, theme=args.theme)
path = write(data, style, args.output, scale=args.scale, width=args.width,
hops=args.hops, title=args.title)
except (StyleError, ValueError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
graphs = len(list((args.output / "graphs").glob("*.svg"))) if (args.output / "graphs").exists() else 0
print(f" explore {path} {graphs} neighbourhood diagram(s)")
return 0

View File

@@ -0,0 +1,43 @@
""" python3 -m docgen.emitters index <ir.json> [-o out.md|out.json]"""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from .index import to_markdown, to_sidebar
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters index")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path, help=".md for the document, .json for a sidebar.")
p.add_argument("--title", default="")
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)} problem(s)):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
if args.output and args.output.suffix == ".json":
text = json.dumps(to_sidebar(data), indent=2) + "\n"
else:
text = to_markdown(data, title=args.title)
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
print(f" index {args.output}")
else:
sys.stdout.write(text)
return 0

View File

@@ -0,0 +1,50 @@
""" python3 -m docgen.emitters minimap <ir.json> [-o out.svg] [--scale 0.55]"""
import argparse
import json
import re
import sys
from pathlib import Path
from ..ir import check
from ..style import Style, StyleError
from .minimap import emit
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters minimap")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path)
p.add_argument("--style", default="lucid")
p.add_argument("--theme", default=None)
p.add_argument("--scale", type=float, default=0.55, help="Pixels per source line.")
p.add_argument("--width", type=int, default=1180, help="Wrap a shelf past this.")
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)}):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
try:
style = Style.load(args.style, theme=args.theme)
svg = emit(data, style, scale=args.scale, target_width=args.width)
except (StyleError, ValueError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(svg)
m = re.search(r'width="(\d+)pt" height="(\d+)pt"', svg)
print(f" minimap {args.output} {m.group(1)}x{m.group(2)}" if m else "")
else:
sys.stdout.write(svg)
return 0

View File

@@ -0,0 +1,73 @@
""" python3 -m docgen.emitters notebook <ir.json> [-o out.ipynb] [--overlay f.json]
--spec-out FILE write the generated spec (the base), for reading/diffing
--scaffold FILE write a blank overlay listing every step id
"""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from ..notebook import dump, from_ir, merge, scaffold
from .notebook import emit
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters notebook")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path)
p.add_argument("--overlay", type=Path, help="Hand-written additions, re-applied.")
p.add_argument("--spec-out", type=Path, help="Write the generated spec too.")
p.add_argument("--scaffold", type=Path, help="Write a blank overlay and stop.")
p.add_argument("--base-url", default="https://api.example.invalid")
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)}):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
base = from_ir(data, base_url=args.base_url)
if args.scaffold:
dump(scaffold(base), args.scaffold)
print(f" overlay {args.scaffold} {len(base['steps'])} step(s), none filled in")
return 0
overlay = None
if args.overlay:
if args.overlay.exists():
overlay = json.loads(args.overlay.read_text())
else:
print(f" note: no overlay at {args.overlay} — generating the base only",
file=sys.stderr)
spec, drift = merge(base, overlay)
for d in drift:
# The base moved under the overlay. Worth saying out loud; not a reason
# to refuse to build the document.
print(f" drift: {d}", file=sys.stderr)
if args.spec_out:
dump(spec, args.spec_out)
print(f" spec {args.spec_out}")
text = emit(spec)
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
n = len(json.loads(text)["cells"])
extra = f", {len(drift)} drift" if drift else ""
print(f" notebook {args.output} {len(spec['steps'])} steps, {n} cells{extra}")
else:
sys.stdout.write(text)
return 0

View File

@@ -0,0 +1,73 @@
""" python3 -m docgen.emitters site <ir.json> -o DIR [--theme lucid]
Writes index.html, viewer.html, site.css and the graph — self-contained, offline.
"""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from ..ops import classify
from ..style import Style, StyleError
from .site import write
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.emitters site")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path, required=True, help="Directory.")
p.add_argument("--style", default="lucid")
p.add_argument("--theme", default=None)
p.add_argument("--title", default="")
p.add_argument("--no-graph", action="store_true")
args = p.parse_args(argv)
try:
data = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"Error: {args.ir} is not a valid IR ({len(problems)}):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
try:
style = Style.load(args.style, theme=args.theme)
except StyleError as e:
print(f"Error: {e}", file=sys.stderr)
return 1
args.output.mkdir(parents=True, exist_ok=True)
graph_name = None
if not args.no_graph:
# Whatever the structure asks for, so the page carries the right picture.
verdict = classify(data)
if verdict["emitter"] == "erd":
from .erd import emit as draw
(args.output / "graph.svg").write_text(draw(data, style))
graph_name = "graph.svg"
elif verdict["emitter"] == "dot":
from .dot import emit as dot_emit, have_graphviz, render
if have_graphviz():
opts = verdict.get("options") or {}
(args.output / "graph.svg").write_bytes(
render(dot_emit(data, style, rankdir=opts.get("rankdir")))
)
graph_name = "graph.svg"
else:
print(" note: graphviz absent — the site is text only", file=sys.stderr)
else:
print(f" note: {verdict['kind']}{verdict['why']}", file=sys.stderr)
print(" no diagram on the page; the index is the artifact", file=sys.stderr)
files = write(data, style, args.output, graph=graph_name, title=args.title)
for f in files:
print(f" site {f}")
if graph_name:
print(f" site {args.output / graph_name}")
return 0

View File

@@ -0,0 +1,291 @@
"""
IR + style -> DOT -> SVG.
This module has never heard of Python, `ast` or SQL. It walks nodes, looks up a
rule by `kind`, and writes attributes. That is deliberately the whole algorithm:
if it starts making decisions about what something *is*, the decision belongs in
an extractor, and if it starts making decisions about what something *looks
like*, it belongs in a style file.
from docgen.emitters.dot import emit, render
svg = render(emit(ir, Style.load("lucid")))
## Containment becomes clusters
A node with children is a `subgraph cluster_*`; a leaf is a node. That is the
only structural interpretation made here, and it follows from `parent` meaning
containment and nothing else.
## The SVG is addressable
`id` and `kind` are written through to the SVG as the element's `id` and
`class`, and `attrs.file`/`attrs.line` become an `href`. So a front end can
attach behaviour to a box, and a box can link to the line it came from, without
this emitter knowing anything about either.
## Known limits
Recorded in `style/lucid.json` under `limits` and reachable as `Style.limits()`.
DOT is used until it genuinely cannot express a rule, and then it stops rather
than growing machinery — an HTML-like label table for header bars, a post-pass
for badge circles. Those mark where a richer emitter begins.
"""
import shutil
import subprocess
class RenderError(RuntimeError):
"""Graphviz is absent, or refused the graph."""
def _esc(text: str) -> str:
return str(text).replace("\\", "\\\\").replace('"', '\\"').replace("\n", "\\n")
def _attrs(pairs: dict) -> str:
inner = " ".join(f'{k}="{_esc(v)}"' for k, v in pairs.items() if v not in (None, "", []))
return f" [{inner}]" if inner else ""
def _style_words(rule: dict, *, filled: bool = True) -> str:
words = ["filled"] if filled else []
# `record` and `plaintext` ignore rounding; asking for it warns and changes
# nothing, which is noise in the build output.
if rule.get("rounded") and rule.get("shape") not in ("record", "Mrecord", "plaintext"):
words.append("rounded")
if rule.get("dashed"):
words.append("dashed")
return ",".join(words)
def _node_attrs(node, rule: dict, style) -> dict:
a = {
"label": node.get("label") or node["id"],
"shape": rule.get("shape", "box"),
"style": _style_words(rule),
"fillcolor": rule.get("fill"),
"color": rule.get("border"),
"fontcolor": rule.get("text"),
"penwidth": style.geom("hairline"),
"fontname": style.geom("font-bold" if rule.get("bold") else "font"),
"fontsize": rule.get("font-size") or style.geom("font-size-base"),
"margin": style.geom("padding"),
# Addressability: through to the SVG, for whatever reads it later.
"id": node["id"],
"class": node["kind"],
}
attrs = node.get("attrs") or {}
if attrs.get("file"):
line = attrs.get("line")
a["href"] = f"{attrs['file']}#L{line}" if line else attrs["file"]
a["tooltip"] = attrs.get("doc") or node["id"]
elif attrs.get("doc"):
a["tooltip"] = attrs["doc"]
return a
def emit(ir: dict, style, *, max_depth: int | None = None,
rankdir: str | None = None) -> str:
"""IR document (a dict) + Style -> DOT text."""
nodes = {n["id"]: n for n in ir["nodes"]}
children: dict[str | None, list[str]] = {}
for n in ir["nodes"]:
children.setdefault(n.get("parent"), []).append(n["id"])
g = style.graph()
out = [
"digraph ir {",
f' bgcolor="{g.get("bgcolor", "transparent")}"',
f' rankdir={rankdir or g.get("rankdir", "TB")}',
f' nodesep="{g.get("nodesep", 0.5)}"',
f' ranksep="{g.get("ranksep", 0.6)}"',
f' pad="{g.get("pad", 0.3)}"',
f' fontname="{style.geom("font")}"',
" compound=true",
"",
]
# Deterministic: groups are numbered by sorted id, so the rotation of
# domain colours is the same on every run.
group_index = {
nid: i for i, nid in enumerate(sorted(k for k in children if k is not None))
}
def write(node_id: str, depth: int) -> None:
node = nodes[node_id]
kids = sorted(children.get(node_id, []))
too_deep = max_depth is not None and depth >= max_depth
pad = " " * (depth + 1)
if not kids or too_deep:
out.append(f"{pad}{_q(node_id)}{_attrs(_node_attrs(node, style.node(node['kind']), style))}")
return
rule = style.group(node["kind"])
domain = (node.get("attrs") or {}).get("domain")
border = rule.get("border") or style.slot(
style.domain_slot(domain, group_index.get(node_id, 0))
)
out.append(f"{pad}subgraph cluster_{_safe(node_id)} {{")
out.append(f'{pad} label="{_esc(node.get("label") or node_id)}"')
out.append(f'{pad} style="{_style_words(rule)}"')
out.append(f'{pad} color="{border}"')
out.append(f'{pad} fillcolor="{rule.get("fill", "transparent")}"')
out.append(f'{pad} fontcolor="{rule.get("text", "")}"')
out.append(f'{pad} fontname="{style.geom("font-bold" if rule.get("bold") else "font")}"')
out.append(f'{pad} fontsize="{style.geom("font-size-header")}"')
out.append(f'{pad} labeljust=l')
out.append(f'{pad} id="{_esc(node_id)}"')
out.append(f'{pad} class="{node["kind"]}"')
for kid in kids:
write(kid, depth + 1)
out.append(f"{pad}}}")
for root in sorted(children.get(None, [])):
write(root, 0)
out.append("")
# DOT cannot use a cluster as an edge endpoint. The native answer is
# `compound=true` plus lhead/ltail: draw between a representative leaf
# inside each cluster and clip the line at the cluster boundary. Without
# this, every module-to-module import silently disappears — which is most of
# the graph a Python extractor produces.
endpoint = _endpoints(nodes, children, max_depth)
for e in ir["edges"]:
src, dst = endpoint.get(e["source"]), endpoint.get(e["target"])
if not src or not dst or src[0] == dst[0]:
continue
if src[1] and src[1] == dst[1]:
continue # both collapsed into the same cluster
# A package importing its own submodule gives an edge whose head sits
# inside its tail's cluster. Graphviz warns and draws it oddly; clipping
# to the enclosing boundary is meaningless there, so drop that side's
# clip and let the line run to the box.
ltail, lhead = src[1], dst[1]
if ltail and _within(ltail, dst[0], nodes):
ltail = None
if lhead and _within(lhead, src[0], nodes):
lhead = None
rule = style.edge(e["kind"])
out.append(
f" {_q(src[0])} -> {_q(dst[0])}"
+ _attrs(
{
"color": rule.get("color"),
"fontcolor": rule.get("text"),
"penwidth": style.geom("hairline"),
"arrowhead": rule.get("arrowhead", "normal"),
"arrowsize": rule.get("arrowsize", 0.7),
"style": "dashed" if rule.get("dashed") else None,
"fontname": style.geom("font"),
"fontsize": style.geom("font-size-sm"),
"label": (e.get("attrs") or {}).get("label"),
"ltail": ltail,
"lhead": lhead,
"class": e["kind"],
}
)
)
out.append("}")
return "\n".join(out) + "\n"
def _within(cluster_name: str, node_id: str, nodes) -> bool:
"""Is `node_id` inside the cluster named `cluster_name`?"""
cur = node_id
while cur:
if f"cluster_{_safe(cur)}" == cluster_name:
return True
cur = nodes.get(cur, {}).get("parent")
return False
def _endpoints(nodes, children, max_depth):
"""id -> (leaf to draw from, cluster to clip to or None).
A leaf is its own endpoint. A node that became a cluster is represented by
its first leaf descendant in sorted order — deterministic, so the same graph
twice produces the same DOT — with `ltail`/`lhead` naming the cluster so the
line stops at its border instead of burrowing to the inner box.
"""
depth_of: dict[str, int] = {}
def walk(nid, depth):
depth_of[nid] = depth
for kid in children.get(nid, []):
walk(kid, depth + 1)
for root in children.get(None, []):
walk(root, 0)
def collapsed(nid: str) -> bool:
return max_depth is not None and depth_of.get(nid, 0) >= max_depth
def first_leaf(nid: str) -> str:
while children.get(nid) and not collapsed(nid):
nid = sorted(children[nid])[0]
return nid
out: dict[str, tuple[str, str | None]] = {}
for nid in nodes:
cur = nid
# Anything past the depth limit is represented by the ancestor that
# survived it.
while max_depth is not None and depth_of.get(cur, 0) > max_depth:
parent = nodes[cur].get("parent")
if not parent:
break
cur = parent
if children.get(cur) and not collapsed(cur):
out[nid] = (first_leaf(cur), f"cluster_{_safe(cur)}")
else:
out[nid] = (cur, None)
return out
def _safe(text: str) -> str:
return "".join(c if c.isalnum() else "_" for c in text)
def _q(text: str) -> str:
return f'"{_esc(text)}"'
# -- render ----------------------------------------------------------------
def have_graphviz(engine: str = "dot") -> bool:
return shutil.which(engine) is not None
def render(dot_text: str, fmt: str = "svg", engine: str = "dot") -> bytes:
"""DOT -> bytes, via the graphviz binary.
The binary, not a wrapper library: it is what the render hosts have and what
`docs/graphs/render.sh` already shells out to.
Note for anything reading geometry back out: Graphviz is y-up in points and
the SVG backend flips it with a wrapper `<g transform="...">`, and layout
measures label text with the host's fonts — so the same graph on a machine
with different fontconfig produces different coordinates. Pin golden tests
to the IR, never to the SVG.
"""
if not have_graphviz(engine):
raise RenderError(
f"{engine!r} not found — install with: sudo apt install graphviz\n"
"(already-rendered files keep working; this is only needed to re-render)"
)
proc = subprocess.run([engine, f"-T{fmt}"], input=dot_text.encode(), capture_output=True)
if proc.returncode != 0:
raise RenderError(
f"{engine} -T{fmt} failed ({proc.returncode}):\n"
+ proc.stderr.decode("utf-8", "replace").strip()
)
if proc.stderr.strip():
# Graphviz warns and still renders — a missing font, an ignored
# attribute. Worth seeing, not worth failing on.
for line in proc.stderr.decode("utf-8", "replace").strip().splitlines():
print(f" graphviz: {line}")
return proc.stdout

View File

@@ -0,0 +1,259 @@
"""
A schema, as an entity-relationship diagram. SVG written directly — no Graphviz.
**This is the answer to the aspect-ratio problem, and it is not a layout engine.**
A layered engine puts every node at one dependency level into one row, so the
widest level is the width; soleprint's 7-by-109 overview rendered 14:1 and no
Graphviz flag helped. A schema is not layered anyway — tables are peers that
reference each other — so laying it out in ranks was the wrong shape from the
start.
The design is lifted from `station/tools/graphgen/templates/index.html`, the
Supabase-style schema explorer already in this repo. It had solved this:
const cols = Math.max(2, Math.ceil(Math.sqrt(sorted.length * 1.2)));
**Columns from the square root of the table count.** The result is near-square
whatever the size — 4 tables or 400 — because the aspect ratio is chosen rather
than emergent. That is the one thing a rank-based engine cannot do.
Three more things it gets right that a generic node-edge drawing does not:
- **A table is a card**, not a box: a header and a list of its columns. That is
what a schema *is*, and it is the form every ER tool has converged on.
- **An edge starts at the column that holds the key** and ends on the target's
primary key, rather than joining two box centres. That is what makes a
foreign key readable rather than merely present.
- **The geometry is computed, never measured.** Card width and row height are
constants, so this produces identical bytes on any machine — unlike Graphviz,
which measures label text with the host's fonts and so renders differently
wherever fontconfig differs.
Colours come from the same style slots as every other emitter, so an ER diagram
and a code diagram are still one visual language.
"""
from xml.sax.saxutils import escape
# Geometry, from the explorer. Fixed on purpose: nothing here measures text, so
# the output is deterministic and can be golden-tested.
CARD_W = 210
HDR_H = 36
HDR_H_DOC = 50
FIELD_H = 26
COL_GAP = 80
ROW_GAP = 60
PAD = 40
BADGE_W = 28
RADIUS = 8
# ~6.2px per character at 11px in a humanist sans. An estimate, and it only
# decides where a long name is cut — never where anything is placed.
CHAR_W = 6.2
def _truncate(text: str, px: float) -> str:
limit = max(3, int(px / CHAR_W))
return text if len(text) <= limit else text[: limit - 1] + ""
def _tables(ir: dict) -> list[dict]:
"""Tables with their columns, in a stable order."""
columns: dict[str, list] = {}
for n in ir["nodes"]:
if n["kind"] == "column" and n.get("parent"):
columns.setdefault(n["parent"], []).append(n)
out = []
for n in ir["nodes"]:
if n["kind"] != "table":
continue
fields = columns.get(n["id"], [])
out.append(
{
"id": n["id"],
"name": n.get("label") or n["id"],
"doc": (n.get("attrs") or {}).get("doc"),
"fields": fields,
}
)
return sorted(out, key=lambda t: t["id"])
def _card_height(table: dict, columns: bool = True) -> int:
header = HDR_H_DOC if table["doc"] else HDR_H
return header + (len(table["fields"]) * FIELD_H if columns else 0)
def layout(tables: list[dict], edges: list[dict], columns: bool = True) -> dict[str, tuple[int, int]]:
"""Place cards in √n columns, referenced tables first.
Sorting by "is the target of a foreign key" puts the tables everything
points at into the left columns, so the majority of edges run left to right
and stop crossing each other. Same trick the explorer uses.
"""
referenced = {e["target"] for e in edges}
ordered = sorted(tables, key=lambda t: (t["id"] not in referenced, t["id"]))
cols = max(2, int((len(ordered) * 1.2) ** 0.5 + 0.999))
col_w = CARD_W + COL_GAP
cursor = [PAD] * cols
pos: dict[str, tuple[int, int]] = {}
for i, table in enumerate(ordered):
col = i % cols
pos[table["id"]] = (col * col_w + PAD, cursor[col])
cursor[col] += _card_height(table, columns) + ROW_GAP
return pos
def _field_y(table: dict, index: int, top: int) -> float:
header = HDR_H_DOC if table["doc"] else HDR_H
return top + header + (max(index, 0) + 0.5) * FIELD_H
def emit(ir: dict, style, *, columns: bool = True) -> str:
"""IR (a db document) + Style -> SVG text.
`columns=False` draws the header of every card and none of its contents —
the whole schema at a glance, which is what you want before you know which
table you care about. Two hundred tables with their columns is a reference;
two hundred names is a map.
"""
tables = _tables(ir)
if not tables:
raise ValueError(
"no tables in this IR — erd draws a schema, and this one has none. "
"Was it extracted with the python reader?"
)
edges = [e for e in ir["edges"] if e["kind"] in ("foreign_key", "references")]
by_id = {t["id"]: t for t in tables}
pos = layout(tables, edges, columns)
s = style.slot
width = max(x for x, _ in pos.values()) + CARD_W + PAD
height = max(y + _card_height(by_id[t], columns) for t, (_, y) in pos.items()) + PAD
out = [
'<?xml version="1.0" encoding="UTF-8" standalone="no"?>',
f'<svg xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" '
f'width="{width}pt" height="{height}pt" viewBox="0 0 {width} {height}">',
f'<rect width="{width}" height="{height}" fill="{s("surface-0")}"/>',
"<defs>",
f'<marker id="fk" viewBox="0 0 10 10" refX="9" refY="5" markerWidth="6" '
f'markerHeight="6" orient="auto-start-reverse">'
f'<path d="M 0 0 L 10 5 L 0 10 z" fill="{s("station")}"/></marker>',
"</defs>",
]
# Edges first, so cards sit on top of them where they meet.
out.append('<g class="relationships">')
for e in edges:
src, dst = by_id.get(e["source"]), by_id.get(e["target"])
if not src or not dst:
continue
sx, sy_top = pos[src["id"]]
dx_, dy_top = pos[dst["id"]]
label = (e.get("attrs") or {}).get("label")
from_idx = next(
(i for i, f in enumerate(src["fields"]) if f.get("label") == label), 0
)
to_idx = next(
(i for i, f in enumerate(dst["fields"]) if (f.get("attrs") or {}).get("pk")), 0
)
# Leave from whichever side faces the target, so a line never crosses
# its own card to get out.
leaving_right = dx_ >= sx
x1 = sx + CARD_W if leaving_right else sx
x2 = dx_ if leaving_right else dx_ + CARD_W
if columns:
y1 = _field_y(src, from_idx, sy_top)
y2 = _field_y(dst, to_idx, dy_top)
else:
y1 = sy_top + _card_height(src, False) / 2
y2 = dy_top + _card_height(dst, False) / 2
ctrl = min(max(abs(x2 - x1) * 0.5, 50), 180)
c1 = x1 + ctrl if leaving_right else x1 - ctrl
c2 = x2 - ctrl if leaving_right else x2 + ctrl
out.append(
f'<path d="M {x1},{y1:.1f} C {c1},{y1:.1f} {c2},{y2:.1f} {x2},{y2:.1f}" '
f'fill="none" stroke="{s("station")}" stroke-width="1.5" '
f'marker-end="url(#fk)" class="edge {e["kind"]}"/>'
)
out.append("</g>")
# Cards.
for table in tables:
x, y = pos[table["id"]]
header = HDR_H_DOC if table["doc"] else HDR_H
h = _card_height(table, columns)
out.append(f'<g class="table">')
out.append(
f'<rect x="{x}" y="{y}" width="{CARD_W}" height="{h}" rx="{RADIUS}" '
f'fill="{s("surface-0")}" stroke="{s("border")}" stroke-width="1" '
f'data-id="{escape(table["id"])}" data-kind="table" class="blk"/>'
)
# Header band, clipped to the card's rounded top by drawing a rounded
# rect and squaring its bottom with a second one.
out.append(
f'<path d="M {x},{y + RADIUS} a {RADIUS},{RADIUS} 0 0 1 {RADIUS},{-RADIUS} '
f'h {CARD_W - 2 * RADIUS} a {RADIUS},{RADIUS} 0 0 1 {RADIUS},{RADIUS} '
f'v {header - RADIUS} h {-CARD_W} z" fill="{s("surface-2")}"/>'
)
if columns:
out.append(
f'<line x1="{x}" y1="{y + header}" x2="{x + CARD_W}" y2="{y + header}" '
f'stroke="{s("border")}" stroke-width="1"/>'
)
out.append(
f'<text x="{x + 12}" y="{y + 22}" font-family="Helvetica,sans-Serif" '
f'font-size="12" font-weight="bold" fill="{s("text")}">'
f'{escape(_truncate(table["name"], CARD_W - 24))}</text>'
)
if table["doc"]:
out.append(
f'<text x="{x + 12}" y="{y + 38}" font-family="Helvetica,sans-Serif" '
f'font-size="9" fill="{s("text-dim")}">'
f'{escape(_truncate(table["doc"], CARD_W - 24))}</text>'
)
for i, field in enumerate(table["fields"] if columns else []):
fy = y + header + i * FIELD_H
attrs = field.get("attrs") or {}
name = field.get("label") or field["id"].rsplit(".", 1)[-1]
if attrs.get("pk"):
badge, badge_fill = "PK", s("accent")
elif attrs.get("references"):
badge, badge_fill = "FK", s("station")
else:
badge, badge_fill = "", s("text-dim")
if i:
out.append(
f'<line x1="{x + 1}" y1="{fy}" x2="{x + CARD_W - 1}" y2="{fy}" '
f'stroke="{s("surface-2")}" stroke-width="1"/>'
)
if badge:
out.append(
f'<text x="{x + 12}" y="{fy + 17}" font-family="Helvetica,sans-Serif" '
f'font-size="8" font-weight="bold" fill="{badge_fill}">{badge}</text>'
)
out.append(
f'<text x="{x + 12 + BADGE_W}" y="{fy + 17}" '
f'font-family="Helvetica,sans-Serif" font-size="10" '
f'fill="{s("text") if not attrs.get("nullable") else s("text-muted")}">'
f'{escape(_truncate(name, 96))}</text>'
)
type_text = attrs.get("references") or attrs.get("type", "")
if type_text:
out.append(
f'<text x="{x + CARD_W - 12}" y="{fy + 17}" text-anchor="end" '
f'font-family="Helvetica,sans-Serif" font-size="9" '
f'fill="{s("text-dim")}">{escape(_truncate(str(type_text), 60))}</text>'
)
out.append("</g>")
out.append("</svg>")
return "\n".join(out) + "\n"

View File

@@ -0,0 +1,315 @@
"""
Two panes: a minimap to navigate by, and a detail pane to explore with.
The minimap on its own showed shape and no meaning — a block said "a 30-line
class" and not *which* class, or what it touched. It is not the artifact. It is
the **selector**.
left the whole tree as coloured blocks. Scan, then click.
right what that block is, what it reaches, what reaches it — and the
neighbourhood drawn, small enough to read.
**This is also what solves the 14:1 problem.** The whole-graph diagram was
unusable because 109 nodes sat at one dependency level, and no engine draws that
well. Here the whole graph is never drawn: the minimap carries the overview, and
only the neighbourhood of a selection is rendered — a handful of nodes, which
lays out fine every time. Overview and detail stop competing for one picture.
## Selecting for an LLM
The other reason to navigate a tree quickly is to decide what to feed a model.
Blocks can be added to a basket, and the basket is a copyable list of file paths
plus a line count — enough to hand to `distill` or paste, and enough to see that
the selection got too big before spending the context on it.
## Offline and static
The neighbourhood diagrams are rendered at build time, one small SVG per module,
so the page needs no layout engine, no server and no network. Everything is
computed here and read there.
"""
import json
import re
from html import escape
from pathlib import Path
MAX_NEIGHBOURS = 24 # past this, a list reads better than a picture
def _is_schema(ir: dict) -> bool:
return (ir.get("meta") or {}).get("source") == "db" or any(
n["kind"] == "table" for n in ir["nodes"]
)
def _neighbourhood_svgs(ir: dict, style, out_dir: Path, hops: int) -> dict[str, str]:
"""One small diagram per navigable thing, pre-rendered. id -> filename.
A schema walks table by table: the selected table **with its columns**, plus
the tables its keys reach, each clickable to step further. A codebase walks
module by module through its imports. Same operation, different drawing —
which is the `classify` rule applied one level down.
"""
from ..ops import neighbourhood
schema = _is_schema(ir)
if schema:
from .erd import emit as draw
def render_one(view):
return draw(view, style).encode()
wanted = "table"
else:
from .dot import emit as dot_emit, have_graphviz, render
if not have_graphviz():
return {}
def render_one(view):
return render(dot_emit(view, style))
wanted = "module"
graphs_dir = out_dir / "graphs"
graphs_dir.mkdir(parents=True, exist_ok=True)
made: dict[str, str] = {}
for node in ir["nodes"]:
if node["kind"] != wanted:
continue
# A table without its columns is not a table, so a schema's
# neighbourhood carries contents; a module's does not, because its
# contents are the hundred functions that made the sheet unreadable.
view = neighbourhood(ir, node["id"], hops=hops, with_contents=schema)
if len(view["nodes"]) < 2:
continue
if not schema and len(view["nodes"]) > MAX_NEIGHBOURS:
continue
if schema and sum(1 for n in view["nodes"] if n["kind"] == "table") > 12:
continue
name = re.sub(r"[^A-Za-z0-9_.-]", "_", node["id"]) + ".svg"
try:
(graphs_dir / name).write_bytes(render_one(view))
except Exception: # noqa: BLE001 - one bad graph must not cost the page
continue
made[node["id"]] = f"graphs/{name}"
return made
def _facts(ir: dict) -> dict:
"""Everything the detail pane needs, keyed by id."""
out: dict[str, dict] = {}
for n in ir["nodes"]:
a = n.get("attrs") or {}
out[n["id"]] = {
"label": n.get("label") or n["id"],
"kind": n["kind"],
"parent": n.get("parent"),
"file": a.get("file"),
"line": a.get("line"),
"lines": a.get("lines"),
"doc": a.get("doc"),
"error": a.get("error"),
"out": [],
"in": [],
"members": [],
}
for n in ir["nodes"]:
if n.get("parent") in out:
out[n["parent"]]["members"].append(n["id"])
for e in ir["edges"]:
if e["source"] in out:
out[e["source"]]["out"].append([e["target"], e["kind"]])
if e["target"] in out:
out[e["target"]]["in"].append([e["source"], e["kind"]])
return out
def emit(ir: dict, style, minimap_svg: str, graphs: dict[str, str],
title: str = "") -> str:
s = style.slot
meta = ir.get("meta", {})
name = title or meta.get("root", "explore")
facts = _facts(ir)
# The minimap goes inline rather than in an <img>: a block has to be
# clickable, and an image is one opaque rectangle.
inner = minimap_svg[minimap_svg.index("<svg"):]
return f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{escape(name)} — explore</title>
<style>
:root {{
--bg: {s("surface-0")}; --surface: {s("surface-1", s("surface-2"))};
--surface-2: {s("surface-2")}; --border: {s("border")};
--text: {s("text")}; --muted: {s("text-muted")}; --dim: {s("text-dim")};
--accent: {s("accent")};
}}
* {{ margin: 0; padding: 0; box-sizing: border-box; }}
body {{ background: var(--bg); color: var(--text); overflow: hidden;
font-family: "Segoe UI", Inter, system-ui, -apple-system, Arial, sans-serif;
font-size: 12px; line-height: 1.6; }}
.split {{ display: flex; height: 100vh; }}
/* navigate */
.nav {{ flex: 1 1 58%; overflow: auto; padding: 14px; position: relative; }}
.nav svg {{ display: block; }}
.blk {{ cursor: pointer; }}
.blk:hover {{ stroke: var(--accent); stroke-width: 1.5; }}
.blk.sel {{ stroke: var(--accent); stroke-width: 2; }}
.blk.basket {{ stroke: var(--accent); stroke-width: 1; stroke-dasharray: 2 2; }}
/* explore */
.side {{ flex: 0 0 42%; max-width: 560px; border-left: 1px solid var(--border);
background: var(--surface); overflow: auto; padding: 18px 20px; }}
.side h2 {{ font-size: 15px; margin-bottom: 2px; }}
.side .kind {{ color: var(--accent); font-size: 10px; text-transform: uppercase;
letter-spacing: .05em; }}
.side .path {{ color: var(--dim); font-size: 11px; margin: 6px 0 12px;
font-family: ui-monospace, Consolas, monospace; }}
.side .doc {{ color: var(--muted); margin-bottom: 14px; }}
.side h3 {{ font-size: 10px; text-transform: uppercase; letter-spacing: .05em;
color: var(--dim); margin: 16px 0 6px; }}
.side ul {{ list-style: none; }}
.side li {{ padding: 2px 0; color: var(--muted); }}
.side a {{ color: var(--muted); text-decoration: none; cursor: pointer; }}
.side a:hover {{ color: var(--accent); }}
.side .rel {{ color: var(--dim); font-size: 10px; margin-left: .4em; }}
.side img {{ width: 100%; border: 1px solid var(--border); border-radius: 6px;
background: var(--bg); margin-top: 6px; }}
.empty {{ color: var(--dim); }}
.bar {{ position: sticky; top: 0; background: var(--surface);
border-bottom: 1px solid var(--border); margin: -18px -20px 14px;
padding: 10px 20px; display: flex; gap: 10px; align-items: center; }}
button {{ background: var(--surface-2); color: var(--muted); cursor: pointer;
border: 1px solid var(--border); border-radius: 5px; padding: 4px 9px;
font-size: 11px; font-family: inherit; }}
button:hover {{ color: var(--text); border-color: var(--accent); }}
#basket {{ color: var(--accent); }}
textarea {{ width: 100%; height: 120px; background: var(--bg); color: var(--muted);
border: 1px solid var(--border); border-radius: 6px; padding: 8px;
font-family: ui-monospace, Consolas, monospace; font-size: 10px; }}
</style>
</head>
<body>
<div class="split">
<div class="nav" id="nav">{inner}</div>
<aside class="side">
<div class="bar">
<b>{escape(name)}</b>
<span class="rel" id="basket">0 selected</span>
<button onclick="showBasket()">selection</button>
<button onclick="clearBasket()">clear</button>
</div>
<div id="detail"><p class="empty">Click a block. Shift-click adds it to the
selection, for feeding somewhere else.</p></div>
</aside>
</div>
<script>
var FACTS = {json.dumps(facts)};
var GRAPHS = {json.dumps(graphs)};
var basket = [];
function esc(t) {{ return String(t).replace(/[&<>"]/g, function (c) {{
return {{'&': '&amp;', '<': '&lt;', '>': '&gt;', '"': '&quot;'}}[c]; }}); }}
function link(id, rel) {{
var f = FACTS[id];
var label = f ? f.label : id;
return '<li><a onclick="select(\\'' + id.replace(/'/g, "\\\\'") + '\\')">' +
esc(label) + '</a>' + (rel ? '<span class="rel">' + esc(rel) + '</span>' : '') +
'<span class="rel">' + esc(id) + '</span></li>';
}}
function group(pairs) {{
if (!pairs.length) return '<p class="empty">none</p>';
return '<ul>' + pairs.slice(0, 60).map(function (p) {{ return link(p[0], p[1]); }}).join('') + '</ul>';
}}
function select(id) {{
var f = FACTS[id];
if (!f) return;
document.querySelectorAll('.blk.sel').forEach(function (b) {{ b.classList.remove('sel'); }});
document.querySelectorAll('[data-id="' + CSS.escape(id) + '"]').forEach(function (b) {{
b.classList.add('sel');
}});
var h = '<h2>' + esc(f.label) + '</h2><div class="kind">' + esc(f.kind) + '</div>';
var where = f.file ? f.file + (f.line ? ':' + f.line : '') : id;
h += '<div class="path">' + esc(where) + (f.lines ? ' · ' + f.lines + ' lines' : '') + '</div>';
if (f.doc) h += '<div class="doc">' + esc(f.doc) + '</div>';
if (f.error) h += '<div class="doc">⚠ ' + esc(f.error) + '</div>';
if (GRAPHS[id]) {{
h += '<h3>neighbourhood</h3><img src="' + GRAPHS[id] + '" alt="">';
}}
h += '<h3>reaches (' + f.out.length + ')</h3>' + group(f.out);
h += '<h3>reached by (' + f['in'].length + ')</h3>' + group(f['in']);
if (f.members.length) {{
h += '<h3>contains (' + f.members.length + ')</h3>' +
group(f.members.map(function (m) {{ return [m, FACTS[m] ? FACTS[m].kind : '']; }}));
}}
document.getElementById('detail').innerHTML = h;
}}
function toggleBasket(id) {{
var i = basket.indexOf(id);
if (i >= 0) basket.splice(i, 1); else basket.push(id);
document.querySelectorAll('[data-id="' + CSS.escape(id) + '"]').forEach(function (b) {{
b.classList.toggle('basket', basket.indexOf(id) >= 0);
}});
document.getElementById('basket').textContent = basket.length + ' selected';
}}
function showBasket() {{
// A copyable list of paths and a line count — enough to hand to distill, and
// enough to see the selection got too big before spending context on it.
var files = [], lines = 0, seen = {{}};
basket.forEach(function (id) {{
var f = FACTS[id];
if (!f) return;
var p = f.file || id;
if (!seen[p]) {{ seen[p] = 1; files.push(p); lines += (f.lines || 0); }}
}});
document.getElementById('detail').innerHTML =
'<h2>Selection</h2><div class="kind">' + files.length + ' files · ~' + lines +
' lines</div><div class="path">Paste, or feed to distill.</div>' +
'<textarea readonly>' + esc(files.join('\\n')) + '</textarea>';
}}
function clearBasket() {{
basket = [];
document.querySelectorAll('.blk.basket').forEach(function (b) {{ b.classList.remove('basket'); }});
document.getElementById('basket').textContent = '0 selected';
document.getElementById('detail').innerHTML = '<p class="empty">Cleared.</p>';
}}
document.getElementById('nav').addEventListener('click', function (e) {{
var b = e.target.closest('.blk');
if (!b) return;
if (e.shiftKey) toggleBasket(b.dataset.id); else select(b.dataset.id);
}});
</script>
</body>
</html>
"""
def write(ir: dict, style, out_dir, *, scale: float = 0.55, width: int = 1100,
hops: int = 1, title: str = "") -> Path:
from .minimap import emit as minimap_emit
out_dir = Path(out_dir)
out_dir.mkdir(parents=True, exist_ok=True)
if _is_schema(ir):
# The whole schema, no columns: a map rather than a reference. The
# minimap's line-span geometry means nothing for a table.
from .erd import emit as erd_emit
svg = erd_emit(ir, style, columns=False)
else:
svg = minimap_emit(ir, style, scale=scale, target_width=width)
graphs = _neighbourhood_svgs(ir, style, out_dir, hops)
path = out_dir / "explore.html"
path.write_text(emit(ir, style, svg, graphs, title=title))
return path

View File

@@ -0,0 +1,163 @@
"""
IR -> an index. Markdown for reading, JSON for a sidebar.
**This is the emitter that matters most for reach.** A sorted, described list of
what exists is readable by people who will never open a diagram — a PM checking
that a feature has a home, QA looking for the surface to test, someone new
trying to find where anything is. A diagram asks for graph literacy and a
screen; this asks for neither.
It is also the checkpoint on the whole design. If the IR were secretly
diagram-shaped, this emitter would be awkward to write — it would be reaching
for positions, or re-deriving containment from edges. It is not, because
`parent` is containment and `kind` is meaning, and that is all a table of
contents needs.
python3 -m docgen.emitters index ir.json # markdown to stdout
python3 -m docgen.emitters index ir.json -o x.json # sidebar JSON
## What it reports that a diagram cannot
- **what depends on what is outside**, gathered in one place. `external` nodes
are the project's real dependency surface, and in a diagram they are scattered
boxes.
- **what could not be parsed.** A file the extractor choked on is a hole in the
analysis; it is listed rather than quietly absent.
"""
import json
from collections import Counter, defaultdict
# Order matters for reading, not for correctness: containers before contents.
KIND_ORDER = ["module", "class", "function", "table", "column", "external"]
def _tree(ir: dict):
children = defaultdict(list)
for n in ir["nodes"]:
children[n.get("parent")].append(n)
for kids in children.values():
kids.sort(key=lambda n: (KIND_ORDER.index(n["kind"]) if n["kind"] in KIND_ORDER else 99,
n["id"]))
return children
def _anchor(node: dict) -> str:
attrs = node.get("attrs") or {}
if not attrs.get("file"):
return ""
return f"{attrs['file']}:{attrs['line']}" if attrs.get("line") else attrs["file"]
def to_markdown(ir: dict, title: str = "") -> str:
"""A document. Headings for containers, a list for their contents."""
children = _tree(ir)
nodes = {n["id"]: n for n in ir["nodes"]}
meta = ir.get("meta", {})
counts = Counter(n["kind"] for n in ir["nodes"])
edge_counts = Counter(e["kind"] for e in ir["edges"])
out = [f"# {title or meta.get('root', 'index')}", ""]
out.append(
f"Extracted from `{meta.get('root', '?')}` by the `{meta.get('source', '?')}` "
f"reader. {len(ir['nodes'])} nodes, {len(ir['edges'])} edges."
)
out.append("")
out.append("| | |")
out.append("|---|---|")
for kind, n in sorted(counts.items(), key=lambda kv: -kv[1]):
out.append(f"| {kind} | {n} |")
for kind, n in sorted(edge_counts.items(), key=lambda kv: -kv[1]):
out.append(f"| *{kind}* (edges) | {n} |")
out.append("")
# -- the contents -----------------------------------------------------
def walk(node: dict, depth: int):
kids = [k for k in children.get(node["id"], [])]
doc = (node.get("attrs") or {}).get("doc")
anchor = _anchor(node)
if depth == 0:
out.append(f"## {node['label']} <small>{node['kind']}</small>")
out.append("")
if doc:
out.append(doc)
out.append("")
if anchor:
out.append(f"`{anchor}`")
out.append("")
else:
bullet = " " * (depth - 1) + "-"
parts = [f"**{node['label']}**", f"*{node['kind']}*"]
if doc:
parts.append(f"{doc}")
if anchor:
parts.append(f"`{anchor}`")
out.append(f"{bullet} {' '.join(parts)}")
for kid in kids:
walk(kid, depth + 1)
if depth == 0 and kids:
out.append("")
roots = [n for n in children.get(None, []) if n["kind"] != "external"]
for root in roots:
walk(root, 0)
# -- what is outside --------------------------------------------------
externals = sorted(n["id"] for n in ir["nodes"] if n["kind"] == "external")
if externals:
out.append("## Depends on, outside this tree")
out.append("")
out.append(
"Names that could not be resolved to anything in the source. This is the "
"dependency surface — third-party imports, and anything reached dynamically."
)
out.append("")
incoming = Counter(e["target"] for e in ir["edges"] if e["target"] in set(externals))
for eid in sorted(externals, key=lambda e: (-incoming[e], e)):
n = incoming[eid]
out.append(f"- `{eid}`" + (f" — referenced {n}×" if n > 1 else ""))
out.append("")
# -- holes in the analysis --------------------------------------------
broken = [n for n in ir["nodes"] if (n.get("attrs") or {}).get("error")]
if broken:
out.append("## Not parsed")
out.append("")
out.append("These files were skipped, so anything they define is missing below.")
out.append("")
for n in sorted(broken, key=lambda n: n["id"]):
out.append(f"- `{(n.get('attrs') or {}).get('file', n['id'])}` — "
f"{(n.get('attrs') or {}).get('error')}")
out.append("")
return "\n".join(out).rstrip() + "\n"
def to_sidebar(ir: dict) -> dict:
"""Nested JSON for a navigation pane.
Shaped for a UI to render directly: `label`, `kind`, `href`, `children`.
"""
children = _tree(ir)
def build(node: dict) -> dict:
attrs = node.get("attrs") or {}
item = {"id": node["id"], "label": node["label"], "kind": node["kind"]}
if attrs.get("doc"):
item["doc"] = attrs["doc"]
if attrs.get("file"):
item["href"] = (
f"{attrs['file']}#L{attrs['line']}" if attrs.get("line") else attrs["file"]
)
kids = [build(k) for k in children.get(node["id"], [])]
if kids:
item["children"] = kids
return item
return {
"meta": ir.get("meta", {}),
"items": [build(n) for n in children.get(None, []) if n["kind"] != "external"],
"external": sorted(n["id"] for n in ir["nodes"] if n["kind"] == "external"),
}

View File

@@ -0,0 +1,277 @@
"""
IR -> a structural minimap. What is where, readable without reading.
Sublime's minimap shrinks the *characters*. This draws the *structure* at full
scale: one file is one column, one line is a fixed number of pixels, and every
construct is a block sized by the lines it actually occupies and coloured by
what it is. Nothing is summarised and no text is rendered — the point is to see
the shape of a codebase without reading a line of it.
a tall solid block one long class
a column of thin stripes many small functions
a wide pale gap module-level code, imports, comments
one block filling a file the 800-line thing everybody avoids
**The pattern has to come from the colours alone**, so `kind` is the only thing
that varies and nesting is drawn by inset rather than by hue: a method inside a
class is the class's colour, indented. Scanning a hundred files then shows which
are class-shaped, which are a pile of loose functions, and which are one block.
Not a node-edge graph, and deliberately not forced into one — it answers "what
is where", which a dependency diagram never does.
## Layout
Files are grouped by their package, packed left to right into shelves, and each
shelf is as tall as its tallest file. Grouping by package is what makes *where*
legible: the shape of a subsystem is the shape of its band.
## Geometry
Computed, never measured — `lines × SCALE`. Deterministic, portable, and
independent of any font, which is the same property `erd` has and `dot` does
not.
"""
from html import escape
COL_W = 74 # one file
COL_GAP = 8
SCALE = 0.55 # pixels per line of source
MIN_H = 14
PAD = 28
LABEL_H = 18
ROW_GAP = 26
PKG_GAP = 18
INSET = 7 # per level of nesting
TARGET_W = 1180 # wrap a shelf past this
# Which slot each kind is drawn in. Everything else lands on `default`, so an
# unfamiliar vocabulary still renders rather than vanishing.
KIND_SLOT = {
"class": "station",
"interface": "accent",
"function": "atlas",
"module": "surface-2",
"table": "station",
"endpoint": "accent",
"operation": "accent",
"task": "station",
"external": "muted",
}
def _files(ir: dict) -> list[dict]:
"""Modules with their constructs, nested, each carrying a line span."""
by_id = {n["id"]: n for n in ir["nodes"]}
kids: dict[str, list] = {}
for n in ir["nodes"]:
if n.get("parent"):
kids.setdefault(n["parent"], []).append(n)
def declared(node: dict) -> list:
"""A node's drawable children, with wrappers dissolved.
A C# `namespace` and a TypeScript `module` come through as `kind:
"module"` nested inside a file. Drawing them adds a level of inset to
everything without adding information — and worse, the file's own
contents then appear twice. They are descended through and not drawn.
"""
out = []
for c in sorted(kids.get(node["id"], []),
key=lambda c: ((c.get("attrs") or {}).get("line", 0))):
if not (c.get("attrs") or {}).get("line"):
continue
if c["kind"] == "module":
out.extend(declared(c)) # a wrapper: keep what is inside it
else:
out.append(c)
return out
def build(node: dict, depth: int) -> dict:
a = node.get("attrs") or {}
return {
"id": node["id"],
"label": node.get("label") or node["id"],
"kind": node["kind"],
"line": a.get("line", 1),
"lines": max(1, a.get("lines", 1)),
"depth": depth,
"children": [build(c, depth + 1) for c in declared(node)],
}
out = []
for n in ir["nodes"]:
if n["kind"] != "module":
continue
a = n.get("attrs") or {}
total = a.get("lines")
if not total:
continue
# A file, not a namespace. Both arrive as `module`; only a file has a
# length without a starting line, because a file starts at the start.
if a.get("line"):
continue
node = build(n, 0)
node["total"] = total
node["package"] = n["id"].rsplit(".", 1)[0] if "." in n["id"] else ""
node["error"] = a.get("error")
out.append(node)
return out
def _bands(files: list[dict]) -> list[tuple[str, list]]:
groups: dict[str, list] = {}
for f in files:
groups.setdefault(f["package"] or "(root)", []).append(f)
for members in groups.values():
members.sort(key=lambda f: -f["total"])
return sorted(groups.items(), key=lambda kv: (-len(kv[1]), kv[0]))
def _blocks(node: dict, top: float, height: float, x: float, out: list, style, file_lines: int):
"""Place a construct and everything inside it."""
slot = KIND_SLOT.get(node["kind"], "border")
inset = INSET * max(0, node["depth"] - 1)
out.append({
"x": x + inset,
"y": top,
"w": COL_W - 2 * inset,
"h": max(2.0, height),
"fill": style.slot(slot, style.slot("border")),
"depth": node["depth"],
"id": node["id"],
"kind": node["kind"],
"title": f'{node["label"]}{node["kind"]}, {node["lines"]} lines',
})
for child in node["children"]:
offset = (child["line"] - node["line"]) / max(node["lines"], 1)
child_h = child["lines"] / max(node["lines"], 1) * height
_blocks(child, top + offset * height, child_h, x, out, style, file_lines)
def emit(ir: dict, style, *, scale: float = SCALE, target_width: int = TARGET_W) -> str:
"""IR + Style -> SVG text. No Graphviz, no font measurement."""
files = _files(ir)
if not files:
raise ValueError(
"nothing to map — a minimap needs modules carrying `attrs.lines`. "
"Was this extracted with the python reader?"
)
s = style.slot
placed, labels, marks = [], [], []
x, y, shelf_h, max_x = PAD, PAD + LABEL_H, 0, 0
current_package = None
# One continuous flow, files ordered by package, wrapping at the target
# width — rather than a row per package. A real tree has many small
# packages (soleprint: 234 files in 70) and a row each gave a 1196x10165
# ribbon. The aspect ratio has to be chosen, not left to emerge; the
# grouping stays legible because a package's files are still adjacent.
ordered = [(pkg, f) for pkg, members in _bands(files) for f in members]
for package, f in ordered:
height = max(MIN_H, f["total"] * scale)
starts_package = package != current_package
gap = COL_GAP + (PKG_GAP if starts_package and x > PAD else 0)
if x + gap + COL_W > target_width and x > PAD:
y += shelf_h + ROW_GAP
x, shelf_h, gap = PAD, 0, 0
starts_package = True # a wrap re-labels, so a row is readable alone
elif x > PAD:
x += gap
if starts_package:
marks.append({"x": x, "y": y - 6, "text": package})
current_package = package
placed.append({
"x": x, "y": y, "w": COL_W, "h": height,
"fill": s("surface-1", s("surface-2")), "depth": -1,
"title": f'{f["id"]}{f["total"]} lines'
+ (f' — NOT PARSED: {f["error"]}' if f["error"] else ""),
"outline": s("border"),
})
for child in f["children"]:
offset = (child["line"] - 1) / max(f["total"], 1)
_blocks(child, y + offset * height,
child["lines"] / max(f["total"], 1) * height,
x, placed, style, f["total"])
labels.append({"x": x, "y": y + height + 9, "text": f["label"][:11], "band": False})
shelf_h = max(shelf_h, height + 12)
max_x = max(max_x, x + COL_W)
x += COL_W
y += shelf_h + PAD
labels = marks_to_labels(marks) + labels
width = max(max_x + PAD, 420)
height = y + 30
out = [
'<?xml version="1.0" encoding="UTF-8" standalone="no"?>',
f'<svg xmlns="http://www.w3.org/2000/svg" width="{width:.0f}pt" '
f'height="{height:.0f}pt" viewBox="0 0 {width:.0f} {height:.0f}">',
f'<rect width="{width:.0f}" height="{height:.0f}" fill="{s("surface-0")}"/>',
]
for b in placed:
extra = (f' stroke="{b["outline"]}" stroke-width="1"' if b.get("outline")
else ' stroke="none"')
# Nested blocks sit on their parent, so a little transparency keeps the
# containment readable instead of hiding it.
opacity = "" if b["depth"] < 0 else f' opacity="{0.95 if b["depth"] <= 1 else 0.8}"'
ident = (f' data-id="{escape(b["id"])}" data-kind="{b.get("kind", "")}" '
f'class="blk"' if b.get("id") else "")
out.append(
f'<rect x="{b["x"]:.1f}" y="{b["y"]:.1f}" width="{b["w"]:.1f}" '
f'height="{b["h"]:.1f}" rx="2" fill="{b["fill"]}"{extra}{opacity}{ident}>'
f'<title>{escape(b["title"])}</title></rect>'
)
for lab in labels:
if lab["band"]:
out.append(
f'<text x="{lab["x"]:.0f}" y="{lab["y"]:.0f}" font-family="Helvetica,sans-Serif" '
f'font-size="10" font-weight="bold" fill="{s("text-muted")}">'
f'{escape(lab["text"])}</text>'
)
else:
out.append(
f'<text x="{lab["x"]:.0f}" y="{lab["y"]:.0f}" font-family="Helvetica,sans-Serif" '
f'font-size="7" fill="{s("text-dim")}">{escape(lab["text"])}</text>'
)
# A legend, because the whole claim is that the colours carry the meaning.
lx = PAD
ly = height - 14
for kind in ("module", "class", "interface", "function"):
slot = KIND_SLOT.get(kind, "border")
out.append(
f'<rect x="{lx}" y="{ly - 7}" width="9" height="9" rx="2" '
f'fill="{s(slot, s("border"))}"/>'
)
out.append(
f'<text x="{lx + 13}" y="{ly + 1}" font-family="Helvetica,sans-Serif" '
f'font-size="9" fill="{s("text-dim")}">{kind}</text>'
)
lx += 22 + len(kind) * 6
if lx > width - 220:
ly += 13 # the legend ran into the summary; put it on its own line
out.append(
f'<text x="{width - PAD:.0f}" y="{ly + 1}" text-anchor="end" '
f'font-family="Helvetica,sans-Serif" font-size="9" fill="{s("text-dim")}">'
f'{len(files)} files · {sum(f["total"] for f in files):,} lines · '
f'1px ≈ {1 / scale:.1f} lines</text>'
)
out.append("</svg>")
return "\n".join(out) + "\n"
def marks_to_labels(marks: list) -> list:
"""Package names, as band labels above the first file of each group."""
return [{"x": m["x"], "y": m["y"], "text": m["text"], "band": True} for m in marks]

View File

@@ -0,0 +1,278 @@
"""
IR -> a Jupyter notebook. Generated, never hand-authored.
## The frame, because it is the whole argument
A notebook is normally a **source file** that someone confects by hand — prose,
code and stored output braided together, diffing badly, drifting from whatever
it documents the moment either moves, with no way to tell by looking. jupytext
addresses the diffing and leaves the rest: it makes the notebook editable as
text, so you still hand-author it.
Here a notebook is a **build artifact**. The source is the OpenAPI document —
the same file the server is built from — and the notebook is regenerated from
it. Nobody edits the `.ipynb`, the same way nobody edits a `.o` file. "Is this
document current" stops being a question about somebody's diligence and becomes
a question about whether the build ran.
That is also why it suits a mixed audience rather than a data-science one. The
endpoints, their methods, their payload shapes and their status codes are facts
taken from the spec, so a PM reading it is reading the API, not somebody's
recollection of it.
## Reproducible in the strict sense
Cell ids come from position, `execution_count` is null, `outputs` is empty, and
the JSON is key-sorted. **The same IR produces byte-identical bytes.** A
notebook that changes on every build cannot be reviewed, and one that cannot be
reviewed will not be trusted.
Written against the nbformat 4 schema directly. `nbformat` is not installed on
the machines this runs on, and the schema has six required keys.
"""
import json
from pathlib import Path
METADATA = {
"kernelspec": {"display_name": "Python 3", "language": "python", "name": "python3"},
"language_info": {"name": "python"},
}
def _cell(index: int, kind: str, text: str) -> dict:
lines = text.split("\n")
source = [ln + "\n" for ln in lines[:-1]] + ([lines[-1]] if lines[-1] else [])
cell = {
"cell_type": "markdown" if kind == "md" else "code",
"id": f"cell-{index:03d}",
"metadata": {},
"source": source,
}
if kind == "code":
cell["execution_count"] = None
cell["outputs"] = []
return cell
def _example(fields: list[dict]) -> str:
"""A request body shaped like the schema, with placeholder values."""
sample = {}
for f in fields:
if f.get("pk"):
continue # the server assigns it
t = str(f.get("type", "str")).lower()
name = f["name"]
if "int" in t:
sample[name] = 0
elif "float" in t or "decimal" in t:
sample[name] = 0.0
elif "bool" in t:
sample[name] = False
elif "date" in t or "time" in t:
sample[name] = "2026-01-01T00:00:00Z"
elif "list" in t:
sample[name] = []
else:
sample[name] = f"<{name}>"
# A Python literal, not JSON. `json.dumps` writes `false`/`true`/`null`,
# which are valid *identifiers* in Python — so the cell compiles and then
# raises NameError the moment anyone runs it. Compiling is not enough of a
# check; the notebook selftest executes these cells for exactly this reason.
body = ",\n".join(f" {k!r}: {v!r}" for k, v in sorted(sample.items()))
return "{\n" + body + ",\n}" if body else "{}"
CLIENT = '''def call(method, path, params=None, body=None):
"""One request. Returns (status, parsed_body).
A non-2xx is returned rather than raised: the error body usually names the
field that was wrong, and an exception throws that away.
"""
url = BASE_URL.rstrip("/") + "/" + path.lstrip("/")
if params:
url += "?" + urllib.parse.urlencode(params)
data = json.dumps(body).encode() if body is not None else None
headers = {"Accept": "application/json", **AUTH}
if data:
headers["Content-Type"] = "application/json"
req = urllib.request.Request(url, data=data, headers=headers, method=method)
print(f"-> {method} {url}")
try:
with urllib.request.urlopen(req, timeout=TIMEOUT) as r:
status, raw = r.status, r.read()
except urllib.error.HTTPError as e:
status, raw = e.code, e.read()
except urllib.error.URLError as e:
print(f"<- unreachable: {e.reason}")
return None, None
try:
parsed = json.loads(raw) if raw else None
except json.JSONDecodeError:
parsed = raw.decode("utf-8", "replace")
print(f"<- {status}")
return status, parsed
def show(result, limit=1500):
status, body = result
if status is None:
return
text = body if isinstance(body, str) else json.dumps(body, indent=2)
print(text[:limit] + (f"\\n… {len(text) - limit} more" if len(text) > limit else ""))'''
def _params_cell(step: dict) -> str:
env = step.get("env_var", "API_TOKEN")
return f'''import json
import os
import urllib.error
import urllib.parse
import urllib.request
BASE_URL = os.environ.get("API_BASE_URL", "{step.get("base_url", "")}")
TOKEN = os.environ.get("{env}", "")
TIMEOUT = 30
# Read from the environment, never written here: a token pasted into a cell
# travels with every copy of this notebook from then on.
AUTH = {{"Authorization": f"Bearer {{TOKEN}}"}} if TOKEN else {{}}
print("base ", BASE_URL)
print("token", f"set ({{len(TOKEN)}} chars)" if TOKEN else "NOT SET — export {env}=…")'''
def _call_cell(step: dict) -> str:
"""The generated call. An overlay's `code` replaces this wholesale."""
method, path = step.get("method", "GET"), step.get("path", "/")
params = step.get("path_params") or []
if step.get("graphql"):
# One endpoint carries every operation, so the operation name is the
# thing worth showing, not the path.
variables = {f["name"]: f"<{f['name']}>" for f in (step.get("body_fields") or [])}
return (
f'QUERY = """{step.get("title", "query")} {{ ... }}""" '
"# fill in the selection set\n"
+ (f"VARIABLES = {variables!r}\n" if variables else "")
+ f'show(call("POST", "{path}", body={{"query": QUERY'
+ (", \"variables\": VARIABLES" if variables else "")
+ '}))'
)
lines = [f'{p.upper()} = "<{p}>" # path parameter' for p in params]
call_path = path
for p in params:
call_path = call_path.replace("{" + p + "}", f'" + str({p.upper()}) + "')
# Trim the empty concatenations a placeholder at either end leaves behind.
expr = f'"{call_path}"' if params else f'"{path}"'
expr = expr.replace(' + ""', "").replace('"" + ', "")
# Parameters that were *always* sent are not optional in practice, whatever
# the spec calls them.
always = step.get("params_always") or []
if always:
lines.append("PARAMS = " + repr({p: f"<{p}>" for p in always}))
arg = ", params=PARAMS" if always else ""
if step.get("body_fields"):
lines.append("BODY = " + _example(step["body_fields"]))
lines.append("")
lines.append(f'show(call("{method}", {expr}{arg}, body=BODY))')
else:
if lines:
lines.append("")
lines.append(f'show(call("{method}", {expr}{arg}))')
return "\n".join(lines)
def _call_md(step: dict) -> str:
out = [f'## {step.get("method", "GET")} `{step.get("path", "/")}`']
if step.get("summary"):
out += ["", step["summary"]]
facts = []
if step.get("response_model"):
facts.append(f'returns **{step["response_model"]}**'
+ (" (a list)" if step.get("returns_list") else ""))
if step.get("request_model"):
facts.append(f'accepts **{step["request_model"]}**')
if step.get("status"):
facts.append(f'expects `{step["status"]}`')
if step.get("statuses"):
# What really came back, which is usually more than the spec promises.
facts.append("seen: " + ", ".join(f'`{c}`' for c in step["statuses"]))
if step.get("calls"):
facts.append(f'called {step["calls"]}×')
if step.get("params_sometimes"):
facts.append("sometimes sends " + ", ".join(f'`{p}`' for p in step["params_sometimes"]))
if step.get("id_formats"):
facts.append("id as " + "/".join(step["id_formats"]))
if facts:
out += ["", " · ".join(facts)]
if step.get("note"):
out += ["", step["note"]]
return "\n".join(out)
def build(spec: dict) -> dict:
"""A merged notebook spec -> the notebook, as a dict."""
blocks: list[tuple[str, str]] = []
for step in spec["steps"]:
kind = step.get("kind", "md")
before = step.get("before")
if before:
blocks.append(("md", before))
if kind == "md":
text = []
if step.get("title") and step["id"] != "intro":
text.append(f'## {step["title"]}')
elif step.get("title"):
text.append(f'# {step["title"]}')
if step.get("text"):
text += ["", step["text"]]
if step.get("table"):
text += ["", "| | fields |", "|---|---|"]
for row in step["table"]:
names = ", ".join(f'`{f}`' for f in row.get("fields", []))
text.append(f'| **{row["name"]}** | {names or ""} |')
blocks.append(("md", "\n".join(text)))
elif kind == "params":
if step.get("title"):
blocks.append(("md", f'## {step["title"]}'))
blocks.append(("code", step.get("code") or _params_cell(step)))
elif kind == "code":
if step.get("title"):
blocks.append(("md", f'## {step["title"]}'))
code = step.get("code")
if code is None and step.get("builtin") == "client":
code = CLIENT
blocks.append(("code", code or ""))
elif kind == "call":
blocks.append(("md", _call_md(step)))
blocks.append(("code", step.get("code") or _call_cell(step)))
after = step.get("after_text")
if after:
blocks.append(("md", after))
return {
"cells": [_cell(i, k, t) for i, (k, t) in enumerate(blocks)],
"metadata": METADATA,
"nbformat": 4,
"nbformat_minor": 5,
}
def emit(spec: dict) -> str:
"""The notebook as text. Key-sorted, so the same spec gives the same bytes."""
return json.dumps(build(spec), indent=1, sort_keys=True, ensure_ascii=False) + "\n"
def write(spec: dict, path) -> Path:
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(emit(spec))
return path

View File

@@ -0,0 +1,493 @@
"""
IR -> a self-contained documentation site: sidebar, content, graph viewer.
Not invented here. Five demos under `semester/` already converged on the same
two files, and this generates that arrangement rather than a sixth variant:
docs/index.html a 220px sticky sidebar beside a max-800px content column
docs/viewer.html `?src=` → fit to window, wheel-zoom at cursor, drag to pan
`sms`, `mpr`, `cht`, `unt` and `eth` each carry a copy of that viewer. They are
**the same 97 lines**, differing only in comments and one background colour —
which is the same eight-ways-to-do-one-thing this whole tool exists to end.
The handoff between them is already a convention, in both `spr/docs/docs.js:229`
and `sms/docs/index.html:311`, arrived at independently:
<a href="viewer.html?src=X.svg"><img src="X.svg" title="Click to expand"></a>
Inline and scaled to the column; click for the full thing.
## What is added
**A 1:1 toggle.** The copied viewer fits on load and resets to fit on
double-click, and has no way to say "actual size" — which is the one thing you
want the moment a diagram has small text in it. A click toggles fit ↔ 100%,
with the current scale shown in the corner so it is never ambiguous which you
are looking at. A click that moved the mouse is a drag and does not toggle.
## Colours
Baked from the same style slots as every diagram, so the page and the graph on
it match by construction. `--theme lucid` produces a light site and a light
diagram together; nothing has to be kept in sync by hand.
Self-contained and offline: no CDN, no build step, opens over `file://`.
"""
import json
from html import escape
from pathlib import Path
SIDEBAR_W = 220
CONTENT_W = 800
# The viewer, with the toggle the copied ones lack. Kept as one string because
# it is one file and its whole value is that there is exactly one of it.
VIEWER = """<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>__TITLE__</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body { background: __BG__; overflow: hidden; width: 100vw; height: 100vh;
font-family: __FONT__; }
#container { width: 100vw; height: 100vh; overflow: hidden; cursor: grab; }
#container.dragging { cursor: grabbing; }
img { transform-origin: 0 0; user-select: none; -webkit-user-drag: none; }
#hud {
position: fixed; bottom: 14px; left: 14px; display: flex; gap: 8px;
align-items: center; font-size: 11px; color: __MUTED__;
background: __SURFACE__; border: 1px solid __BORDER__;
border-radius: 6px; padding: 5px 9px; user-select: none;
}
#hud b { color: __TEXT__; font-weight: 600; font-variant-numeric: tabular-nums; }
#hud span { opacity: .7; }
a.back { position: fixed; top: 14px; left: 14px; font-size: 11px;
color: __MUTED__; text-decoration: none; background: __SURFACE__;
border: 1px solid __BORDER__; border-radius: 6px; padding: 5px 9px; }
a.back:hover { color: __TEXT__; }
</style>
</head>
<body>
<div id="container"><img id="img" alt=""></div>
<a class="back" href="index.html">&larr; docs</a>
<div id="hud"><b id="pct">100%</b><span id="mode">fit</span><span>&middot; click 1:1 &middot; drag &middot; wheel</span></div>
<script>
var src = new URLSearchParams(location.search).get('src');
var img = document.getElementById('img');
var container = document.getElementById('container');
var pct = document.getElementById('pct');
var modeEl = document.getElementById('mode');
if (src) { img.src = src; document.title = src + ' — __TITLE__'; }
var scale = 1, x = 0, y = 0, fitScale = 1, mode = 'fit';
var dragging = false, moved = false, startX, startY, startPanX, startPanY;
function apply() {
img.style.transform = 'translate(' + x + 'px,' + y + 'px) scale(' + scale + ')';
pct.textContent = Math.round(scale * 100) + '%';
modeEl.textContent = mode;
}
function fit() {
var sw = window.innerWidth / img.naturalWidth;
var sh = window.innerHeight / img.naturalHeight;
fitScale = Math.min(sw, sh) * 0.95;
scale = fitScale;
x = (window.innerWidth - img.naturalWidth * scale) / 2;
y = (window.innerHeight - img.naturalHeight * scale) / 2;
mode = 'fit';
apply();
}
// Zoom about a point in the viewport, so what is under the cursor stays there.
function zoomAt(px, py, factor) {
x = px - (px - x) * factor;
y = py - (py - y) * factor;
scale *= factor;
mode = Math.abs(scale - fitScale) < 0.001 ? 'fit'
: (Math.abs(scale - 1) < 0.001 ? '1:1' : 'free');
apply();
}
img.onload = fit;
window.addEventListener('resize', function () { if (mode === 'fit') fit(); });
container.addEventListener('wheel', function (e) {
e.preventDefault();
var rect = container.getBoundingClientRect();
zoomAt(e.clientX - rect.left, e.clientY - rect.top, e.deltaY < 0 ? 1.12 : 0.89);
}, { passive: false });
container.addEventListener('mousedown', function (e) {
if (e.button !== 0) return;
dragging = true; moved = false;
startX = e.clientX; startY = e.clientY; startPanX = x; startPanY = y;
container.classList.add('dragging');
e.preventDefault();
});
window.addEventListener('mousemove', function (e) {
if (!dragging) return;
if (Math.abs(e.clientX - startX) > 3 || Math.abs(e.clientY - startY) > 3) moved = true;
x = startPanX + (e.clientX - startX);
y = startPanY + (e.clientY - startY);
apply();
});
window.addEventListener('mouseup', function (e) {
if (!dragging) return;
dragging = false;
container.classList.remove('dragging');
// A click that moved the mouse was a drag, and must not also toggle.
if (moved) return;
if (mode === '1:1') { fit(); return; }
// Toggle to actual size about the point clicked, so the thing you aimed at
// is the thing you end up looking at.
var rect = container.getBoundingClientRect();
zoomAt(e.clientX - rect.left, e.clientY - rect.top, 1 / scale);
mode = '1:1';
apply();
});
container.addEventListener('dblclick', fit);
window.addEventListener('keydown', function (e) {
if (e.key === '0' || e.key === 'f') fit();
if (e.key === '1') { var r = container.getBoundingClientRect();
zoomAt(r.width / 2, r.height / 2, 1 / scale); mode = '1:1'; apply(); }
if (e.key === 'Escape') location.href = 'index.html';
});
</script>
</body>
</html>
"""
CSS = """/* Generated by docgen. The layout five demos converged on: a sticky sidebar
beside a bounded content column. Colours are baked from the style's theme, so
the page and the diagrams on it are one visual language. */
:root {
--bg: __BG__;
--surface: __SURFACE__;
--surface-2: __SURFACE2__;
--border: __BORDER__;
--text: __TEXT__;
--muted: __MUTED__;
--dim: __DIM__;
--accent: __ACCENT__;
}
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
background: var(--bg); color: var(--text);
font-family: __FONT__; font-size: 13px; line-height: 1.65;
}
.layout { display: flex; min-height: 100vh; }
.sidebar {
width: __SIDEBAR__px; flex-shrink: 0; background: var(--surface);
border-right: 1px solid var(--border);
position: sticky; top: 0; height: 100vh; overflow-y: auto;
padding: 1.25rem 0; scrollbar-width: none;
}
.sidebar::-webkit-scrollbar { display: none; }
.sidebar-header { padding: 0 1rem 1rem; border-bottom: 1px solid var(--border); }
.sidebar-header b { color: var(--text); font-size: 13px; }
.sidebar-header small { display: block; color: var(--dim); font-size: 10px; margin-top: 2px; }
.sidebar ul { list-style: none; }
/* Every link, however deep — a link inside a <summary> is still a link, and
selecting `li > a` quietly missed all of them. */
.sidebar a {
display: block; padding: 3px 1rem; color: var(--muted);
text-decoration: none; font-size: 12px; border-left: 2px solid transparent;
white-space: nowrap; overflow: hidden; text-overflow: ellipsis;
}
.sidebar a:hover { color: var(--text); background: var(--surface-2); }
.sidebar a.active { color: var(--accent); border-left-color: var(--accent); }
.sidebar .k { color: var(--dim); font-size: 9px; text-transform: uppercase;
letter-spacing: .04em; margin-left: .4em; font-weight: 400; }
/* Indent by nesting depth rather than by element, so it keeps working however
deep the tree goes. */
.sidebar ul ul a { padding-left: 1.8rem; }
.sidebar ul ul ul a { padding-left: 2.6rem; }
.sidebar ul ul ul ul a { padding-left: 3.4rem; }
.sidebar details > summary {
cursor: pointer; list-style: none; display: flex; align-items: center;
}
.sidebar details > summary::-webkit-details-marker { display: none; }
.sidebar details > summary::before {
content: ""; color: var(--dim); flex: 0 0 auto;
margin-left: .55rem; font-size: 9px; transition: transform .12s;
}
.sidebar details[open] > summary::before { transform: rotate(90deg); }
.sidebar details > summary > a { flex: 1 1 auto; padding-left: .45rem; }
.sidebar details > summary:hover::before { color: var(--text); }
.content { flex: 1; min-width: 0; max-width: __CONTENT__px; padding: 2rem 3rem; }
.content h1 { font-size: 22px; margin-bottom: .25rem; }
.content h2 { font-size: 15px; margin: 2rem 0 .5rem; padding-top: 1rem;
border-top: 1px solid var(--border); }
.content h3 { font-size: 13px; margin: 1.25rem 0 .35rem; color: var(--muted); }
.content p { margin-bottom: .75rem; color: var(--muted); }
.content code { font-family: ui-monospace, "Cascadia Mono", Consolas, monospace;
font-size: 11px; background: var(--surface); padding: 1px 5px;
border-radius: 3px; color: var(--text); }
.content table { border-collapse: collapse; margin: .75rem 0; font-size: 12px; }
.content th, .content td { text-align: left; padding: 4px 14px 4px 0;
border-bottom: 1px solid var(--border); color: var(--muted); }
.content th { color: var(--dim); font-weight: 600; font-size: 10px;
text-transform: uppercase; letter-spacing: .04em; }
.lede { color: var(--dim); font-size: 12px; margin-bottom: 1.5rem; }
/* The convention both spr/docs and sms/docs arrived at independently:
inline and scaled to the column, click for the full thing. */
.figure { margin: 1rem 0 1.5rem; }
.figure a { display: block; border: 1px solid var(--border); border-radius: 8px;
overflow: hidden; background: var(--surface); }
.figure a:hover { border-color: var(--accent); }
.figure img { display: block; width: 100%; height: auto; }
.figure figcaption { color: var(--dim); font-size: 10px; margin-top: .4rem; }
/* Both ends of the book, side by side and above the diagram. Above, because a
reader who scrolls past the picture has already formed an impression, and
"2 files could not be read" has to arrive before that and not after. */
.ledger { display: flex; gap: 1px; background: var(--border); border: 1px solid var(--border);
border-radius: 8px; overflow: hidden; margin: 0 0 1.5rem; }
.ledger > div { flex: 1 1 0; background: var(--surface); padding: .7rem .9rem; min-width: 0; }
.ledger dt { color: var(--dim); font-size: 9.5px; text-transform: uppercase;
letter-spacing: .07em; margin-bottom: .3rem; }
.ledger dd { margin: 0; color: var(--text); font-size: 12.5px; }
.ledger dd .sub { display: block; color: var(--muted); font-size: 11px; margin-top: .2rem;
overflow-wrap: anywhere; }
.ledger .lost { color: var(--artery); }
.ledger .kept { color: var(--ok); }
.gap { border: 1px solid var(--artery); border-left-width: 3px; border-radius: 6px;
background: var(--surface); padding: .7rem .9rem; margin: 0 0 1.5rem; font-size: 12px; }
.gap b { color: var(--artery); }
.gap ul { margin: .4rem 0 0 1.1rem; color: var(--muted); }
.gap code { font-size: 11px; }
"""
def _ledger(book: dict) -> str:
"""The two measures, and the gap between them if there is one.
This is the whole reason the site is the book's last step rather than just
another emitter: it is the one artifact somebody definitely opens, so it is
where "45 of 47 files" has to appear. A diagram cannot say it, and a log
nobody reads does not count as having said it.
"""
larder = book.get("larder") or {}
measure = book.get("book") or {}
failed = larder.get("failed") or []
kinds = measure.get("by_kind") or {}
out_summary = " · ".join(f"{v} {k}" for k, v in
sorted(kinds.items(), key=lambda kv: -kv[1])[:4])
unit = larder.get("unit", "unit")
read, seen = larder.get("read", 0), larder.get("seen", 0)
plural = unit if read == 1 else (unit[:-1] + "ies" if unit.endswith("y") else unit + "s")
in_line = f"{read} {plural} read"
if failed:
in_line += f' <span class="lost"{len(failed)} of {seen} could not be</span>'
panel = (
'<div class="ledger">'
f'<div><dt>what came in</dt><dd>{in_line}'
f'<span class="sub">{escape(larder.get("identity", "?"))}</span></dd></div>'
f'<div><dt>what came out</dt><dd>{escape(out_summary) or "nothing"}'
f'<span class="sub">{measure.get("edges", 0)} edges · '
f'{measure.get("external", 0)} external · '
# "step artifacts", not "artifacts": this page is written before the
# book's last step closes, so it cannot count itself. Saying `step`
# makes the number true rather than one short of book.json's.
f'{len(measure.get("artifacts") or [])} step artifacts</span></dd></div>'
"</div>"
)
# A reconciliation that failed is not a footnote. It means the document
# below is incomplete in a way the document below cannot show.
lost = [r for r in (book.get("reconciled") or []) if not r.get("ok")]
if lost or failed:
items = "".join(f"<li>{escape(r['claim'])}{escape(r['why'])}</li>" for r in lost)
items += "".join(
f"<li><code>{escape(f['name'])}</code> — {escape(f['error'])}</li>"
for f in failed[:12]
)
if len(failed) > 12:
items += f"<li>and {len(failed) - 12} more</li>"
panel += (
'<div class="gap"><b>This book is incomplete.</b> '
"What is drawn below is everything that could be read, which is not "
f"everything there is.<ul>{items}</ul></div>"
)
return panel
def _slots(style) -> dict:
s = style.slot
return {
"__BG__": s("surface-0"),
"__SURFACE__": s("surface-1") or s("surface-2"),
"__SURFACE2__": s("surface-2"),
"__BORDER__": s("border"),
"__TEXT__": s("text"),
"__MUTED__": s("text-muted"),
"__DIM__": s("text-dim"),
"__ACCENT__": s("accent"),
"__FONT__": '"Segoe UI", Inter, system-ui, -apple-system, Arial, sans-serif',
}
def _fill(template: str, values: dict) -> str:
for key, value in values.items():
template = template.replace(key, str(value))
return template
def _sidebar(items: list, depth: int = 0) -> str:
out = ["<ul>"]
for item in items:
label = escape(item.get("label", item["id"]))
kind = escape(item.get("kind", ""))
anchor = escape(item["id"])
link = f'<a href="#{anchor}" data-id="{anchor}">{label}<span class="k">{kind}</span></a>'
kids = item.get("children") or []
if kids:
out.append(
f"<li><details{' open' if depth == 0 else ''}>"
f"<summary>{link}</summary>{_sidebar(kids, depth + 1)}</details></li>"
)
else:
out.append(f"<li>{link}</li>")
out.append("</ul>")
return "".join(out)
def _sections(items: list, depth: int = 0) -> str:
out = []
for item in items:
tag = "h2" if depth == 0 else "h3"
attrs = item.get("attrs") or {}
out.append(f'<{tag} id="{escape(item["id"])}">{escape(item.get("label", ""))}'
f'<span class="k"> {escape(item.get("kind", ""))}</span></{tag}>')
if item.get("doc"):
out.append(f"<p>{escape(item['doc'])}</p>")
if item.get("href"):
out.append(f'<p><code>{escape(item["href"])}</code></p>')
kids = item.get("children") or []
if kids:
out.append(_sections(kids, depth + 1))
return "".join(out)
def emit(ir: dict, style, *, graph: str | None = None, title: str = "",
book: dict | None = None) -> dict:
"""IR + Style -> {filename: text}. Write them next to each other.
`book` is a book ledger (`book/__init__.py`). When present the page opens
with both measures — what went in, what came out — because a page that
shows only the result is the thing the measure exists to correct.
Optional, so the site emitter still works on a bare IR.
"""
from .index import to_sidebar
meta = ir.get("meta", {})
name = title or meta.get("root", "docs")
side = to_sidebar(ir)
values = _slots(style)
counts: dict[str, int] = {}
for n in ir["nodes"]:
counts[n["kind"]] = counts.get(n["kind"], 0) + 1
summary = " · ".join(f"{v} {k}" for k, v in sorted(counts.items(), key=lambda kv: -kv[1]))
ledger = _ledger(book) if book else ""
figure = ""
if graph:
figure = (
'<figure class="figure">'
f'<a href="viewer.html?src={escape(graph)}" title="Click to open — then click again for 1:1">'
f'<img src="{escape(graph)}" alt="{escape(name)}"></a>'
"<figcaption>Click to open the viewer · click again for actual size</figcaption>"
"</figure>"
)
external = side.get("external") or []
ext_html = ""
if external:
rows = "".join(f"<tr><td><code>{escape(e)}</code></td></tr>" for e in external[:40])
ext_html = (
'<h2 id="__external">Depends on, outside this tree</h2>'
"<p>Names that could not be resolved here — the dependency surface.</p>"
f"<table>{rows}</table>"
)
index = f"""<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{escape(name)}</title>
<link rel="stylesheet" href="site.css">
</head>
<body>
<div class="layout">
<nav class="sidebar">
<div class="sidebar-header"><b>{escape(name)}</b><small>{escape(summary)}</small></div>
{_sidebar(side["items"])}
</nav>
<main class="content">
<h1>{escape(name)}</h1>
<p class="lede">Generated from <code>{escape(meta.get("source", "?"))}</code> ·
{escape(summary)}. Regenerated, not edited.</p>
{ledger}
{figure}
{_sections(side["items"])}
{ext_html}
</main>
</div>
<script>
// Highlight the section being read. No dependency, no build step.
var links = [].slice.call(document.querySelectorAll('.sidebar a[data-id]'));
var byId = {{}};
links.forEach(function (a) {{ byId[a.dataset.id] = a; }});
var obs = new IntersectionObserver(function (entries) {{
entries.forEach(function (en) {{
var a = byId[en.target.id];
if (!a) return;
if (en.isIntersecting) {{
links.forEach(function (l) {{ l.classList.remove('active'); }});
a.classList.add('active');
}}
}});
}}, {{ rootMargin: '-10% 0px -80% 0px' }});
document.querySelectorAll('h2[id], h3[id]').forEach(function (h) {{ obs.observe(h); }});
</script>
</body>
</html>
"""
return {
"index.html": index,
"viewer.html": _fill(VIEWER.replace("__TITLE__", escape(name)), values),
"site.css": _fill(
CSS.replace("__SIDEBAR__", str(SIDEBAR_W)).replace("__CONTENT__", str(CONTENT_W)),
values,
),
}
def write(ir: dict, style, out_dir, *, graph: str | None = None, title: str = "",
book: dict | None = None) -> list[Path]:
out_dir = Path(out_dir)
out_dir.mkdir(parents=True, exist_ok=True)
written = []
for name, text in emit(ir, style, graph=graph, title=title, book=book).items():
path = out_dir / name
path.write_text(text)
written.append(path)
return written

View File

@@ -0,0 +1,67 @@
// Drive the explorer's logic under a stub DOM: select, walk, basket.
const fs = require('fs');
const html = fs.readFileSync(process.argv[2], 'utf8');
const script = html.split('<script>').pop().split('</script>')[0];
const detail = { innerHTML: '' };
const basketEl = { textContent: '' };
const nav = { addEventListener() {} };
const ids = { detail, basket: basketEl, nav };
global.document = {
getElementById: (i) => ids[i],
querySelectorAll: () => [],
};
global.CSS = { escape: (s) => s };
const api = new Function(script + '; return {select, toggleBasket, showBasket, clearBasket, FACTS, GRAPHS};')();
let ok = true;
const check = (name, cond) => {
console.log(` ${cond ? 'ok ' : 'FAIL'} ${name}`);
if (!cond) ok = false;
};
const ids_ = Object.keys(api.FACTS);
const withEdges = ids_.filter((i) => api.FACTS[i].out.length || api.FACTS[i]['in'].length);
check('facts are embedded for every node', ids_.length > 0);
check('some node has relationships', withEdges.length > 0);
const target = withEdges[0];
api.select(target);
check('selecting renders a detail pane', detail.innerHTML.includes('<h2>'));
check('it names what the thing is', detail.innerHTML.includes(api.FACTS[target].kind));
check('it lists what it reaches', detail.innerHTML.includes('reaches ('));
check('it lists what reaches it', detail.innerHTML.includes('reached by ('));
// Walking forward: every link in the pane must be a selectable id.
const links = [...detail.innerHTML.matchAll(/select\('([^']+)'\)/g)].map((m) => m[1]);
check('neighbours are offered as links to walk to', links.length > 0);
const reachable = links.every((i) => api.FACTS[i] !== undefined);
check('every link resolves to a real node', reachable);
if (links.length) {
api.select(links[0]);
check('walking forward re-renders on the neighbour',
detail.innerHTML.includes(api.FACTS[links[0]].label));
}
const withGraph = ids_.find((i) => api.GRAPHS[i]);
if (withGraph) {
api.select(withGraph);
check('a neighbourhood diagram is shown when one exists',
detail.innerHTML.includes('<img src="graphs/'));
} else {
check('neighbourhood diagrams were generated', false);
}
api.toggleBasket(target);
check('the basket counts a selection', basketEl.textContent === '1 selected');
api.showBasket();
check('the basket is a copyable list of paths', detail.innerHTML.includes('<textarea'));
check('...with a line count, to see it got too big',
/~\d+\s*lines/.test(detail.innerHTML));
api.toggleBasket(target);
check('toggling again removes it', basketEl.textContent === '0 selected');
process.exit(ok ? 0 : 1);

View File

@@ -0,0 +1 @@
"""Extractors: source artifacts -> IR. None of them has heard of SVG."""

View File

@@ -0,0 +1,25 @@
""" python3 -m docgen.extractors <db|openapi|usage|code> [options]"""
import sys
def main(argv=None):
argv = sys.argv[1:] if argv is None else argv
if argv and argv[0] == "openapi":
from .openapi_main import main as run
return run(argv[1:])
if argv and argv[0] == "code":
from .code_main import main as run
return run(argv[1:])
if argv and argv[0] == "usage":
from .usage_main import main as run
return run(argv[1:])
if argv and argv[0] == "db":
from .db_main import main as run
return run(argv[1:])
from .db_main import main as run # bare form stays the db one, as before
return run(argv)
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,281 @@
"""
Source in several languages -> IR structure, via **tree-sitter**.
python3 -m docgen.extractors code --root src/ --lang auto -o ir.json
Adopts a parser rather than writing one, which is the rule the brief sets and
the reason this handles generics, strings containing braces, nested types and
`#region` without any of them being a special case. `pip install tree_sitter
tree_sitter_c_sharp tree_sitter_typescript tree_sitter_python`.
**Optional, never required.** Python still goes through the stdlib `ast`
extractor, which does two-pass name resolution tree-sitter would have to
reimplement. Without tree-sitter installed, docgen loses C# and TypeScript and
nothing else — the import is lazy and the error says what to install.
## Structure only, and that is the point
This produces what is *where*: declarations, their nesting, and the lines each
occupies. It produces **no edges**. Resolving a C# `using` or a TypeScript
`import` to the thing it names is a different and much larger job, and the
consumer that needs this — the minimap — needs none of it.
Saying so matters: an extractor that quietly produced half a dependency graph
would be worse than one that produces none, because the half would look whole.
## Vocabulary
module a file, or a namespace
class class, struct, record, enum — a type with members
interface kept separate from class on purpose: in C# and TypeScript the
distinction is most of what reading a file tells you, and the
minimap's whole claim is that the pattern comes from the colours
function method, constructor, property, function, arrow function
"""
import sys
from pathlib import Path
from ..ir import Graph, Meta
# Extension -> (grammar module, language function). `.tsx` needs its own parser:
# the TSX grammar is a different language, not an option on the TypeScript one.
LANGUAGES = {
".cs": ("tree_sitter_c_sharp", "language"),
".ts": ("tree_sitter_typescript", "language_typescript"),
".mts": ("tree_sitter_typescript", "language_typescript"),
".tsx": ("tree_sitter_typescript", "language_tsx"),
".py": ("tree_sitter_python", "language"),
}
# Declaration node types, per grammar, mapped onto the IR's small vocabulary.
DECLARATIONS = {
"c_sharp": {
"namespace_declaration": "module",
"file_scoped_namespace_declaration": "module",
"class_declaration": "class",
"struct_declaration": "class",
"record_declaration": "class",
"record_struct_declaration": "class",
"enum_declaration": "class",
"interface_declaration": "interface",
"method_declaration": "function",
"constructor_declaration": "function",
"destructor_declaration": "function",
"property_declaration": "function",
"operator_declaration": "function",
"local_function_statement": "function",
},
"typescript": {
"module": "module",
"internal_module": "module",
"class_declaration": "class",
"abstract_class_declaration": "class",
"enum_declaration": "class",
"interface_declaration": "interface",
"type_alias_declaration": "interface",
"function_declaration": "function",
"generator_function_declaration": "function",
"method_definition": "function",
"public_field_definition": "function",
},
"python": {
"class_definition": "class",
"function_definition": "function",
"decorated_definition": None, # descend; the real node is inside
},
}
GRAMMAR_FAMILY = {
"tree_sitter_c_sharp": "c_sharp",
"tree_sitter_typescript": "typescript",
"tree_sitter_python": "python",
}
class MissingParser(ImportError):
"""tree-sitter, or one of its grammars, is not installed."""
def _parser(suffix: str):
"""(Parser, family) for a file extension. Lazy, so the import is optional."""
if suffix not in LANGUAGES:
raise MissingParser(f"no grammar registered for {suffix!r}")
module_name, fn = LANGUAGES[suffix]
try:
from tree_sitter import Language, Parser
except ImportError:
raise MissingParser(
"tree-sitter is not installed — C# and TypeScript need it.\n"
" pip install tree_sitter tree_sitter_c_sharp tree_sitter_typescript\n"
"Python does not: it uses the stdlib `ast` extractor."
) from None
try:
grammar = __import__(module_name)
except ImportError:
raise MissingParser(
f"{module_name} is not installed — needed for {suffix} files.\n"
f" pip install {module_name.replace('_', '-')}"
) from None
return Parser(Language(getattr(grammar, fn)())), GRAMMAR_FAMILY[module_name]
def _name(node, source: bytes) -> str | None:
"""A declaration's name, or None when it has none worth recording."""
field = node.child_by_field_name("name")
if field is not None:
return source[field.start_byte:field.end_byte].decode("utf-8", "replace")
# C# properties and TS fields sometimes carry the name as an identifier child.
for child in node.children:
if child.type in ("identifier", "property_identifier", "type_identifier"):
return source[child.start_byte:child.end_byte].decode("utf-8", "replace")
return None
def _walk(node, source: bytes, family: str, out: list, parent: str | None, prefix: str,
seen_ids: set | None = None):
"""Collect declarations, depth-first, keeping nesting in the id."""
table = DECLARATIONS[family]
seen_ids = seen_ids if seen_ids is not None else set()
for child in node.children:
kind = table.get(child.type, ...)
if kind is None:
# A wrapper — a decorated definition, say. Descend without naming it.
_walk(child, source, family, out, parent, prefix, seen_ids)
continue
if kind is ...:
_walk(child, source, family, out, parent, prefix, seen_ids)
continue
name = _name(child, source)
if not name:
_walk(child, source, family, out, parent, prefix, seen_ids)
continue
nid = f"{prefix}.{name}" if prefix else name
if nid in seen_ids:
# Same reason as the Python extractor: an overload, a partial class,
# or a name declared twice in different branches. The line keeps the
# id unique without making it unstable.
nid = f"{nid}#L{child.start_point[0] + 1}"
seen_ids.add(nid)
out.append({
"id": nid,
"kind": kind,
"label": name,
"parent": parent,
"line": child.start_point[0] + 1,
"lines": max(1, child.end_point[0] - child.start_point[0] + 1),
})
_walk(child, source, family, out, nid, nid, seen_ids)
def extract_file(path: Path, root: Path) -> tuple[dict, list] | None:
"""One file -> (module node, declarations). None when unparseable."""
parser, family = _parser(path.suffix)
try:
source = path.read_bytes()
except OSError as e:
return {"error": str(e)}, []
rel = path.relative_to(root)
module_id = ".".join([*rel.parts[:-1], rel.stem])
tree = parser.parse(source)
decls: list = []
_walk(tree.root_node, source, family, decls, module_id, module_id)
module = {
"id": module_id,
"kind": "module",
"label": rel.stem,
"parent": ".".join(rel.parts[:-1]) or None,
"lines": source.count(b"\n") + 1,
"file": rel.as_posix(),
# tree-sitter never fails to parse; it produces ERROR nodes instead. That
# is more useful than an exception, and worth recording rather than
# silently accepting a partial tree.
"errors": _count_errors(tree.root_node),
}
return module, decls
def _count_errors(node) -> int:
n = 1 if node.type == "ERROR" or node.is_missing else 0
for child in node.children:
n += _count_errors(child)
return n
def extract(root, suffixes=None, exclude=(), source: str = "code",
identity: str | None = None) -> Graph:
"""Walk a tree and map its structure. No edges — see the module docstring."""
root = Path(root).resolve()
if not root.is_dir():
raise NotADirectoryError(f"not a directory: {root}")
wanted = tuple(suffixes) if suffixes else tuple(LANGUAGES)
skip = {"__pycache__", ".git", ".venv", "venv", "node_modules", "dist", "build",
"bin", "obj", *exclude}
from ..book.larder import Larder
g = Graph(Meta(source=source, root=root.name))
packages: set[str] = set()
files = [
p for p in sorted(root.rglob("*"))
if p.suffix in wanted and p.is_file()
and not any(part in skip for part in p.relative_to(root).parts)
]
# The measure is built alongside the walk rather than recomputed after it.
# tree-sitter never raises on bad syntax — it produces ERROR nodes — so
# "failed" here means a file that could not be handled at all, and the
# partially-parsed ones are counted separately under `unparsed_regions`.
larder = Larder(kind="code", identity=identity or root.name, unit="file",
seen=len(files))
unparsed = 0
for path in files:
try:
module, decls = extract_file(path, root)
except MissingParser:
raise
except Exception as e: # noqa: BLE001 - one bad file must not cost the run
rel = path.relative_to(root)
larder.fail(rel.as_posix(), f"{type(e).__name__}: {e}")
g.node(".".join([*rel.parts[:-1], rel.stem]), "module", rel.stem,
attrs={"file": rel.as_posix(), "error": f"{type(e).__name__}: {e}"})
continue
parent = module["parent"]
if parent:
packages.add(parent)
attrs = {"file": module["file"], "lines": module["lines"]}
if module["errors"]:
attrs["error"] = f"{module['errors']} unparsed region(s)"
unparsed += 1
g.node(module["id"], "module", module["label"], parent=parent, attrs=attrs)
for d in decls:
g.node(d["id"], d["kind"], d["label"], parent=d["parent"],
attrs={"file": module["file"], "line": d["line"], "lines": d["lines"]})
# Directories that hold files but are not themselves files still need to
# exist, or every module in them is an orphan.
known = {n.id for n in g.nodes}
for pkg in sorted(packages):
parts = pkg.split(".")
for i in range(1, len(parts) + 1):
pid = ".".join(parts[:i])
if pid not in known:
g.node(pid, "module", parts[i - 1],
parent=".".join(parts[: i - 1]) or None,
attrs={"lines": 0, "directory": True})
known.add(pid)
larder.extra["languages"] = len({p.suffix for p in files})
if unparsed:
# Worth its own number rather than folding into `failed`: a file with an
# unparsed region still contributed structure, so calling it a failure
# would understate what the book contains.
larder.extra["unparsed_regions"] = unparsed
g.meta.larder = larder.to_dict()
return g

View File

@@ -0,0 +1,39 @@
""" python3 -m docgen.extractors code --root src/ [--ext .cs] [-o ir.json]"""
import argparse
import json
import sys
from pathlib import Path
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.extractors code")
p.add_argument("--root", "-s", required=True, type=Path)
p.add_argument("--output", "-o", type=Path)
p.add_argument("--ext", action="append", default=[],
help="Limit to these extensions. Default: every registered one.")
p.add_argument("--exclude", action="append", default=[])
args = p.parse_args(argv)
from .code import LANGUAGES, MissingParser, extract
try:
ir = extract(args.root, suffixes=args.ext or None, exclude=tuple(args.exclude))
except MissingParser as e:
print(f"Error: {e}", file=sys.stderr)
return 1
except (NotADirectoryError, OSError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
text = json.dumps(ir.to_dict(), indent=2) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
from collections import Counter
c = Counter(n.kind for n in ir.nodes)
print(f"{len(ir.nodes)} nodes -> {args.output} "
+ " · ".join(f"{v} {k}" for k, v in sorted(c.items(), key=lambda kv: -kv[1])))
else:
sys.stdout.write(text)
return 0

View File

@@ -0,0 +1,160 @@
"""
A database schema -> IR.
Reads the **published** `{models, relationships, source}` contract rather than
importing modelgen's Python. That contract is already emitted by
`modelgen/generator/jsonschema.py`, consumed by `graphgen/schema.py` and
`datagen`, and asserted by two modelgen tests — so it is the stable surface, and
reading it means this extractor works for every source modelgen supports
(Django, SQLAlchemy, OpenAPI, CSV/ODS, a live database) without knowing about
any of them.
python3 -m docgen.extractors.db --schema cfg/sample/.../graphgen/schema.json
Connecting to a live database is **not here**. `modelgen from-db --url ...`
reflects via SQLAlchemy's Inspector across dialects and writes the schema.json
this reads; that is its job and it is already done. The two-step is also the
safer one — the URL, and therefore the credentials, never enters this pipeline.
## The schema checkpoint
The brief requires the IR to survive a second domain without new top-level
fields. It does, and the mapping is not a squeeze:
table -> node, kind "table" column -> node, kind "column"
column -> parent is its table FK -> edge, kind "foreign_key"
M2M -> edge, kind "references"
`kind` carries the domain vocabulary, `parent` carries containment, `attrs`
carries what only this domain cares about — `pk`, `nullable`, the column type.
Nothing needed a field that `module`/`class`/`function` did not also use, which
is the result the checkpoint was there to confirm.
"""
import json
from pathlib import Path
from ..ir import Graph, Meta
def from_schema_dict(data: dict, root: str = "schema", source: str = "db",
identity: str | None = None) -> Graph:
"""A graphgen-compatible schema dict -> IR.
Accepts both spellings of the contract: the on-disk `schema.json` form,
where `models` is a mapping of name to definition, and the loaded form that
`graphgen.schema.load_graph_schema` returns, where it is a list. They are
the same data and callers have both.
"""
from ..book.larder import Larder
g = Graph(Meta(source=source, root=root))
models = data.get("models", {})
listed = models.values() if isinstance(models, dict) else models
names = (
set(models)
if isinstance(models, dict)
else {m.get("id") or m.get("name") for m in models}
)
for name, model in (
models.items() if isinstance(models, dict)
else ((m.get("id") or m.get("name"), m) for m in models)
):
attrs = {}
if model.get("doc"):
attrs["doc"] = model["doc"]
g.node(name, "table", name, attrs=attrs)
fields = model.get("fields", {})
pairs = fields.items() if isinstance(fields, dict) else (
(f.get("name"), f) for f in fields
)
for field_name, field in pairs:
type_str = field.get("type", "str")
target, kind = _relation(type_str, field)
a = {"type": _plain_type(type_str)}
if field.get("pk"):
a["pk"] = True
if field.get("nullable"):
a["nullable"] = True
if target:
# Kept on the column so the index can say what it points at
# without walking the edge list.
a["references"] = target
g.node(f"{name}.{field_name}", "column", field_name, parent=name, attrs=a)
if target and kind:
# The edge is table -> table: a diagram of forty columns joined
# column-to-column is unreadable, and the relationship is
# between the tables. Which column carries it is in `attrs`.
g.edge(name, target, kind, attrs={"label": field_name})
# Some producers give `relationships` alongside the fields; take them too,
# and let the dedupe below settle it.
for rel in data.get("relationships", []):
src, dst = rel.get("from_model"), rel.get("to_model")
if src in names and dst in names:
kind = "references" if rel.get("type") == "M2M" else "foreign_key"
g.edge(src, dst, kind, attrs={"label": rel.get("from_field", "")})
# A schema is a published contract, so there is nothing here that can fail
# to be read — every table it declares is a table. `seen == read` always,
# and saying so is more useful than omitting the measure: it distinguishes
# "no failures" from "not measured".
larder = Larder(kind="db", identity=identity or root, unit="table",
seen=len(names))
rels = data.get("relationships") or []
larder.extra["relationships"] = len(rels)
if data.get("source"):
larder.extra["dialect"] = str(data["source"])
g.meta.larder = larder.to_dict()
_dedupe(g)
for missing in sorted(
{e.target for e in g.edges} - {n.id for n in g.nodes}
):
# A foreign key naming a table the schema does not define. Recorded
# rather than dropped, for the same reason an unresolved import is.
g.node(missing, "external", missing, attrs={"unresolved": True})
return g
def _relation(type_str: str, field: dict) -> tuple[str | None, str | None]:
"""(target table, edge kind) for a field, or (None, None)."""
if isinstance(type_str, str):
if type_str.startswith("FK:"):
return type_str[3:], "foreign_key"
if type_str.startswith("M2M:"):
return type_str[4:], "references"
if field.get("fk"):
return field["fk"], "references" if field.get("m2m") else "foreign_key"
return None, None
def _plain_type(type_str) -> str:
if isinstance(type_str, str):
for prefix in ("FK:", "M2M:"):
if type_str.startswith(prefix):
return prefix.rstrip(":")
return type_str
def _dedupe(g: Graph) -> None:
seen, keep = set(), []
for e in g.edges:
key = (e.source, e.target, e.kind)
if key in seen:
continue
seen.add(key)
keep.append(e)
g.edges = keep
def extract(schema_path, source: str = "db", identity: str | None = None) -> Graph:
"""Read a schema.json from disk."""
path = Path(schema_path)
data = json.loads(path.read_text())
return from_schema_dict(data, root=path.parent.name or path.stem, source=source,
identity=identity or str(schema_path))

View File

@@ -0,0 +1,31 @@
""" python3 -m docgen.extractors.db --schema path/to/schema.json [-o ir.json]"""
import argparse
import json
import sys
from pathlib import Path
from .db import extract
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.extractors.db")
p.add_argument("--schema", "-s", required=True, type=Path,
help="A graphgen-compatible schema.json, as modelgen emits.")
p.add_argument("--output", "-o", type=Path)
args = p.parse_args(argv)
try:
ir = extract(args.schema)
except (OSError, json.JSONDecodeError, KeyError) as e:
print(f"Error: could not read {args.schema}: {e}", file=sys.stderr)
return 1
text = json.dumps(ir.to_dict(), indent=2) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
print(f"{len(ir.nodes)} nodes, {len(ir.edges)} edges -> {args.output}")
else:
sys.stdout.write(text)
return 0

View File

@@ -0,0 +1,190 @@
"""
An OpenAPI document -> IR: the endpoints, and the shapes they carry.
Adapts `modelgen/loader/extract/openapi.py`, which already parses OpenAPI 3.x
and Swagger 2.0 and resolves `$ref`. This turns its output into IR nodes; it
does not re-parse anything.
python3 -m docgen.extractors.openapi --spec petstore.yaml -o ir.json
## Why this one matters more than it looks
A hand-written API notebook is the thing nobody can keep current: the spec moves
and the document does not, and there is no way to tell by looking. Extracting
the endpoints means the document is **generated from the same file the server is
built from**, so "is this current" becomes a question about a build rather than
about somebody's diligence.
It emits schemas as `table`/`column`, the same vocabulary the database extractor
uses. That is deliberate: an API's data model and a database's are the same kind
of thing, so the ER emitter draws either without knowing which it got. Endpoints
are a separate `kind`, so a view can ask for one or the other.
only_kinds(ir, {"table", "column"}) -> the data model, as an ER diagram
only_kinds(ir, {"endpoint"}) -> the surface, as a notebook
"""
from pathlib import Path
from ..ir import Graph, Meta
def _modelgen():
"""modelgen's OpenAPI reader, from wherever the reference repo is.
Imported lazily and by path rather than as a hard dependency. This is the
**only** seam between docgen and the wider repo — see `reference.py`, which
resolves it from `$DOCGEN_REFERENCE` or by walking up. Every other extractor
and every emitter works with nothing above `docgen/`.
Not reimplemented here on purpose: modelgen already parses the spec and
resolves `$ref`, and a second OpenAPI reader in the same repo is two things
to keep correct.
"""
from .. import reference
if reference.on_path() is None:
raise reference.missing(
"modelgen",
"OpenAPI is read through station/tools/modelgen/loader/extract/"
"openapi.py, which parses the spec and resolves $ref",
)
try:
from modelgen.loader.extract.openapi import OpenAPIExtractor
except ImportError as e:
raise reference.missing(
"modelgen.loader.extract.openapi",
f"the reference repo was found but the module did not import ({e})",
) from None
return OpenAPIExtractor
def _type_name(hint) -> str:
if hint is None:
return "Any"
if isinstance(hint, str):
return hint
return getattr(hint, "__name__", str(hint))
def _refs(path: Path) -> dict[str, dict[str, str]]:
"""{schema: {field: referenced schema}} — the relationships, recovered.
modelgen resolves an inter-schema `$ref` to the literal string `dict`, so by
the time its fields reach us the *target* is gone. Without this, an API's
data model draws as disconnected cards: three tables, no foreign keys, and
nothing saying the relationships were lost. Which is exactly the failure
this tool is otherwise built to prevent.
So one key is read directly, and one only: `$ref` under a schema's
`properties`. That is not parsing OpenAPI — no paths, no bodies, no
responses, no `$ref` resolution, no composition keywords. modelgen still
does all of the reading that matters, and this recovers the single fact its
type mapping cannot carry.
Returns `{}` on anything unexpected. A missing relationship is a worse
diagram; a raised exception here would be no diagram at all.
"""
try:
import yaml # available: modelgen just used it
except ImportError:
return {}
try:
doc = yaml.safe_load(path.read_text()) or {}
schemas = ((doc.get("components") or {}).get("schemas")) or {}
except Exception: # noqa: BLE001 - see the docstring
return {}
out: dict[str, dict[str, str]] = {}
for name, schema in schemas.items():
if not isinstance(schema, dict):
continue
for field, spec in (schema.get("properties") or {}).items():
if not isinstance(spec, dict):
continue
# A direct reference, or an array of them — `lines: [OrderLine]` is
# the same relationship as `order: Order`, pointing the other way.
ref = spec.get("$ref")
if not ref and isinstance(spec.get("items"), dict):
ref = spec["items"].get("$ref")
if isinstance(ref, str) and ref.startswith("#/components/schemas/"):
out.setdefault(name, {})[field] = ref.rsplit("/", 1)[-1]
return out
def extract(spec_path, source: str = "openapi", identity: str | None = None) -> Graph:
"""An OpenAPI file -> IR."""
from ..book.larder import Larder
OpenAPIExtractor = _modelgen()
path = Path(spec_path)
extractor = OpenAPIExtractor(path)
models, enums = extractor.extract()
endpoints = extractor.endpoints()
g = Graph(Meta(source=source, root=path.name))
known = {m.name for m in models}
refs = _refs(path)
for model in models:
attrs = {}
if getattr(model, "docstring", None):
attrs["doc"] = model.docstring.strip().split("\n")[0]
g.node(model.name, "table", model.name, attrs=attrs)
for field in model.fields:
a = {"type": _type_name(field.type_hint)}
if getattr(field, "optional", False):
a["nullable"] = True
if field.name in ("id", "uuid"):
a["pk"] = True
# The recovered $ref target wins over the mapped type name: modelgen
# says `dict` where the spec said which schema.
target = refs.get(model.name, {}).get(field.name) or _type_name(field.type_hint)
if target in known and target != model.name:
a["references"] = target
g.edge(model.name, target, "foreign_key", attrs={"label": field.name})
g.node(
f"{model.name}.{field.name}", "column", field.name,
parent=model.name, attrs=a,
)
for e in endpoints:
eid = f"{e.method} {e.path}"
attrs = {
"method": e.method,
"path": e.path,
"status": getattr(e, "status", None) or 200,
}
for key in ("summary", "operation_id", "envelope_key"):
value = getattr(e, key, None)
if value:
attrs[key] = value
if getattr(e, "response_is_list", False):
attrs["returns_list"] = True
if getattr(e, "path_params", None):
attrs["path_params"] = list(e.path_params)
if getattr(e, "request_model", None):
attrs["request_model"] = e.request_model
if getattr(e, "response_model", None):
attrs["response_model"] = e.response_model
g.node(eid, "endpoint", eid, attrs=attrs)
# `accepts` and `returns` rather than one `uses`: which direction a
# shape travels is the thing a reader wants to know.
if getattr(e, "request_model", None) in known:
g.edge(eid, e.request_model, "accepts")
if getattr(e, "response_model", None) in known:
g.edge(eid, e.response_model, "returns")
# A spec is the larder here, and `path` is the unit because that is what a
# spec is an inventory of. Schemas and enums are counted separately: a spec
# with 40 schemas and 3 endpoints is a data model, and the measure should
# make that visible before the diagram does.
larder = Larder(kind="openapi", identity=identity or path.name, unit="path",
seen=len({e.path for e in endpoints}))
larder.extra["operations"] = len(endpoints)
larder.extra["schemas"] = len(models)
if enums:
larder.extra["enums"] = len(enums)
g.meta.larder = larder.to_dict()
return g

View File

@@ -0,0 +1,29 @@
""" python3 -m docgen.extractors.openapi --spec petstore.yaml [-o ir.json]"""
import argparse
import json
import sys
from pathlib import Path
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.extractors.openapi")
p.add_argument("--spec", "-s", required=True, type=Path)
p.add_argument("--output", "-o", type=Path)
args = p.parse_args(argv)
from .openapi import extract
try:
ir = extract(args.spec)
except (OSError, ImportError, ValueError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
text = json.dumps(ir.to_dict(), indent=2) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
eps = sum(1 for n in ir.nodes if n.kind == "endpoint")
print(f"{len(ir.nodes)} nodes ({eps} endpoints), {len(ir.edges)} edges -> {args.output}")
else:
sys.stdout.write(text)
return 0

View File

@@ -0,0 +1,36 @@
"""
Python source -> IR, by `ast`.
from docgen.extractors.python import extract
ir = extract(Path("app/"))
Deterministic. A diagram built from this cannot be out of date with the code,
which is the whole reason the structural path has no model in it.
Two passes, because `ast` resolves nothing on its own — see `collect.py` and
`resolve.py`.
"""
from pathlib import Path
from .collect import collect
from .resolve import to_ir
def extract(root, exclude=(), source="python", identity=None):
"""Walk `root`, return an IR Graph. Never raises on a bad file.
`identity` is what the larder measure calls this source — the path as the
caller wrote it, which is what they will recognise. It defaults to the
directory name rather than the resolved path, for the same reason
`meta.root` does: an absolute path is not a secret but it is machine
specific, and the measure should mean the same thing on two machines.
"""
root = Path(root).resolve()
if not root.is_dir():
raise NotADirectoryError(f"not a directory: {root}")
modules = collect(root, exclude=exclude)
return to_ir(modules, root=root.name, source=source, identity=identity)
__all__ = ["extract", "collect", "to_ir"]

View File

@@ -0,0 +1,37 @@
""" python3 -m docgen.extractors.python --root PATH [--exclude NAME ...]"""
import argparse
import json
import sys
from pathlib import Path
from . import extract
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.extractors.python")
p.add_argument("--root", "-s", required=True, type=Path, help="Tree to read.")
p.add_argument("--output", "-o", type=Path, help="Where to write. Default stdout.")
p.add_argument("--exclude", action="append", default=[], help="Directory name to skip.")
args = p.parse_args(argv)
try:
ir = extract(args.root, exclude=tuple(args.exclude))
except (NotADirectoryError, OSError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
text = json.dumps(ir.to_dict(), indent=2) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
skipped = sum(1 for n in ir.nodes if n.attrs.get("error"))
print(f"{len(ir.nodes)} nodes, {len(ir.edges)} edges -> {args.output}"
+ (f" ({skipped} file(s) unparsed)" if skipped else ""))
else:
sys.stdout.write(text)
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,235 @@
"""
Pass one: read each module, record what it defines and what it imports.
Nothing is resolved here. `class User(Base)` is recorded as the literal string
`"Base"`, because that is genuinely all `ast` knows — it has no idea the name
came from `from .db import Base` three lines up. Resolving it needs every
module's import table, which is why there is a second pass.
Keeping the two apart is what makes the analysis testable: pass one is a pure
function of one file, pass two is a pure function of the collected tables.
## What `ast` gives, and what it costs
`ast.NodeVisitor` dispatches on node type. Every `visit_*` must end in
`generic_visit(node)` or traversal stops there and nested definitions are lost —
a class inside a function, a method inside a class. That one missing call is the
classic silent hole in an AST walker, so it is at the end of every visitor here.
Every node carries `lineno`, which lands in `attrs` and is what later lets a UI
link a box to a line.
`ast.parse` uses the **running interpreter's grammar**. A file using syntax newer
than this Python raises `SyntaxError`; it is recorded as a skipped file rather
than crashing the run, because one unparseable file should not cost you the
other four hundred.
"""
import ast
from dataclasses import dataclass, field
from pathlib import Path
@dataclass
class Definition:
"""Something a module defines, before it has an id."""
name: str # local name, as written
qualname: str # dotted within the module: "User.save"
kind: str # class | function
lineno: int
end_lineno: int = 0
doc: str | None = None
bases: list[str] = field(default_factory=list) # raw, unresolved
@dataclass
class Module:
"""One file's worth of collected facts."""
name: str # dotted, root-relative
path: str # root-relative posix path
doc: str | None = None
lines: int = 0
package: str = "" # the package it lives in
defines: list[Definition] = field(default_factory=list)
imports: dict[str, str] = field(default_factory=dict) # local -> dotted target
import_order: list[str] = field(default_factory=list) # modules imported, in order
error: str | None = None
class _Collector(ast.NodeVisitor):
"""Walks one module. Instance state tracks the enclosing scope."""
def __init__(self, module: Module):
self.module = module
self._scope: list[str] = [] # enclosing class/function names
# -- definitions ------------------------------------------------------
def _define(self, node, kind: str, bases=()):
qualname = ".".join([*self._scope, node.name])
self.module.defines.append(
Definition(
name=node.name,
qualname=qualname,
kind=kind,
lineno=node.lineno,
# How many lines a construct occupies is structure, not styling:
# it is what a density map sizes a block by, and what "this class
# is 400 lines" means. `ast` carries it, so there is no reason to
# record only where something starts.
end_lineno=getattr(node, "end_lineno", node.lineno) or node.lineno,
doc=_first_line(ast.get_docstring(node)),
bases=list(bases),
)
)
def visit_ClassDef(self, node: ast.ClassDef):
self._define(node, "class", bases=[_name_of(b) for b in node.bases])
self._scope.append(node.name)
self.generic_visit(node)
self._scope.pop()
def visit_FunctionDef(self, node: ast.FunctionDef):
self._define(node, "function")
self._scope.append(node.name)
self.generic_visit(node)
self._scope.pop()
# `async def` is a different AST node with the same meaning here.
visit_AsyncFunctionDef = visit_FunctionDef
# -- imports ----------------------------------------------------------
def visit_Import(self, node: ast.Import):
for alias in node.names:
# `import a.b.c` binds `a`; `import a.b.c as x` binds `x` to a.b.c.
local = alias.asname or alias.name.split(".")[0]
target = alias.name
self.module.imports[local] = target
self.module.import_order.append(target)
self.generic_visit(node)
def visit_ImportFrom(self, node: ast.ImportFrom):
base = _resolve_relative(self.module, node.module, node.level)
for alias in node.names:
if alias.name == "*":
# A star import binds names this pass cannot know. Recorded as a
# module edge; the names it brought in stay unresolved, which is
# the honest outcome rather than a guess.
self.module.import_order.append(base)
continue
local = alias.asname or alias.name
self.module.imports[local] = f"{base}.{alias.name}" if base else alias.name
if base:
self.module.import_order.append(base)
self.generic_visit(node)
def _first_line(doc: str | None) -> str | None:
if not doc:
return None
line = doc.strip().split("\n", 1)[0].strip()
return line or None
def _name_of(node: ast.expr) -> str:
"""The dotted source text of a name expression, or "" if it is not one.
`Base` -> "Base"; `db.Base` -> "db.Base"; `Generic[T]` -> "Generic".
Anything genuinely computed returns "" and is dropped — a base class that is
a function call is not a name any resolver could follow.
"""
if isinstance(node, ast.Name):
return node.id
if isinstance(node, ast.Attribute):
prefix = _name_of(node.value)
return f"{prefix}.{node.attr}" if prefix else ""
if isinstance(node, ast.Subscript):
return _name_of(node.value)
return ""
def _resolve_relative(module: Module, target: str | None, level: int) -> str:
"""`from ..x import y` inside a.b.c -> "a.x".
level 0 is absolute. level 1 is the current package, level 2 its parent, and
so on. Getting this wrong silently attaches edges to the wrong module, so it
is computed from the package rather than from the module name.
"""
if not level:
return target or ""
parts = module.package.split(".") if module.package else []
if level > 1:
parts = parts[: -(level - 1)] if level - 1 <= len(parts) else []
base = ".".join(parts)
if target:
return f"{base}.{target}" if base else target
return base
def module_name(path: Path, root: Path, prefix: str = "") -> tuple[str, str]:
"""(dotted module name, package) for a file, relative to root.
`prefix` is the root's own name, supplied when the root directory is itself
a package. Without it, pointing at `histgen/` makes `histgen/__init__.py`
resolve to the empty string — an unnamed root that every sibling then fails
to claim as its parent. Found by running this over the tools next door,
which is the case a fixture tree does not cover.
"""
rel = path.relative_to(root)
parts = list(rel.parts)
is_init = parts[-1] == "__init__.py"
if is_init:
parts = parts[:-1] # a package is named by its directory
else:
parts[-1] = parts[-1][: -len(".py")]
if prefix:
parts = [prefix, *parts]
name = ".".join(parts)
# An `__init__.py` *is* its package, so `from .x import y` inside it resolves
# against itself, not against its parent. Taking parts[:-1] here sent every
# relative import in a package root one level too high, where it resolved to
# nothing and became a bogus `external` node sitting next to the real module
# of the same name.
package = name if is_init else ".".join(parts[:-1])
return name, package
def collect_file(path: Path, root: Path, prefix: str = "") -> Module:
"""Everything pass two needs from one file. Never raises on bad input."""
name, package = module_name(path, root, prefix)
module = Module(name=name, path=path.relative_to(root).as_posix(), package=package)
try:
source = path.read_text(encoding="utf-8")
except (OSError, UnicodeDecodeError) as e:
module.error = f"unreadable: {e}"
return module
try:
tree = ast.parse(source, filename=str(path))
except SyntaxError as e:
# Newer syntax than this interpreter, or a genuinely broken file. One
# bad file must not cost the run.
module.error = f"syntax: line {e.lineno}: {e.msg}"
return module
module.doc = _first_line(ast.get_docstring(tree))
module.lines = source.count("\n") + 1
_Collector(module).visit(tree)
return module
def collect(root: Path, exclude: tuple[str, ...] = ()) -> list[Module]:
"""Every .py under root, in sorted order so the result is stable."""
skip = {"__pycache__", ".git", ".venv", "venv", "node_modules", "site-packages"}
skip.update(exclude)
# A root that is itself a package is named by its own directory, the way it
# would be imported. A plain directory of packages is not.
prefix = root.name if (root / "__init__.py").exists() else ""
modules = []
for path in sorted(root.rglob("*.py")):
if any(part in skip for part in path.relative_to(root).parts):
continue
modules.append(collect_file(path, root, prefix))
return modules

View File

@@ -0,0 +1,183 @@
"""
Pass two: turn collected names into ids, and collected facts into an IR.
This is the part that decides whether the output is analysis or decoration.
`ast` resolves nothing — pass one recorded the literal string `"Base"`. Here we
know every module's import table, so `"Base"` in `app.models` becomes
`app.db.Base` and the edge points at a real node.
## Unresolved names become nodes, never nothing
A name that cannot be resolved — a third-party import, a star-import binding, a
dynamically built base — becomes a node with `kind: "external"` and keeps its
edge. Dropping it would be the worse failure: the diagram would silently lose a
dependency and look complete, and nobody would know to go looking. An `external`
box is visible and can be styled as the boundary it is.
## What is deliberately not attempted
`calls` edges. Resolving `self.foo()` or `obj.method()` needs type inference,
and a call graph that is quietly 60% right is worse than none — it reads as
authoritative. `imports` and `inherits` are both syntactically decidable from
the import table, which is why they are here and `calls` is not.
"""
from .collect import Module
from ...ir import Graph, Meta
def _id_for(module: Module, qualname: str) -> str:
return f"{module.name}.{qualname}" if module.name else qualname
def _resolve(name: str, module: Module, known: set[str]) -> str | None:
"""A raw name, as an id — or None when it cannot be resolved.
Order matters, and mirrors Python's own: an imported name shadows a
module-level definition of the same name, because the import ran later.
"""
if not name:
return None
head, _, rest = name.partition(".")
# 1. the whole name was imported: `from .db import Base` -> Base
if name in module.imports:
return module.imports[name]
# 2. a dotted name whose head was imported: `import db` -> db.Base
if head in module.imports:
target = module.imports[head]
return f"{target}.{rest}" if rest else target
# 3. defined in this module
local = _id_for(module, name)
if local in known:
return local
# 4. a sibling in the same package: `models.User` inside `app.db`
if module.package:
sibling = f"{module.package}.{name}"
if sibling in known:
return sibling
return None
def larder_of(modules: list[Module], identity: str) -> "Larder":
"""The input measure for a tree of Python files.
Every fact here was already computed and then discarded: `collect_file`
records `Module.error` for a file it could not read or parse, and nothing
downstream ever reported how many there were. A run that parsed 45 of 47
files produced the same document as one that parsed all 47.
"""
from ...book.larder import Larder
larder = Larder(kind="python", identity=identity, unit="file",
seen=len(modules))
for m in modules:
if m.error:
larder.fail(m.path, m.error)
larder.extra["packages"] = len({m.package for m in modules if m.package})
return larder
def to_ir(modules: list[Module], root: str, source: str = "python",
identity: str | None = None) -> Graph:
"""Collected modules -> a validated-shaped IR graph."""
g = Graph(Meta(source=source, root=root,
larder=larder_of(modules, identity or root).to_dict()))
# -- nodes we own ------------------------------------------------------
known: set[str] = set()
for m in modules:
known.add(m.name)
for d in m.defines:
known.add(_id_for(m, d.qualname))
module_names = {m.name for m in modules}
for m in modules:
attrs = {"file": m.path}
if m.lines:
attrs["lines"] = m.lines
if m.doc:
attrs["doc"] = m.doc
if m.error:
# Kept rather than dropped: a file that could not be parsed is a
# hole in the analysis, and the IR should say so out loud.
attrs["error"] = m.error
# Containment is the *enclosing* package, which is not the same as the
# package relative imports resolve against: inside `a/b/__init__.py`,
# `from .x` means `a.b.x` (so package is `a.b`) while the module is
# contained by `a`. Using one for both floats every package root out of
# its own parent.
enclosing = m.name.rsplit(".", 1)[0] if "." in m.name else None
g.node(
m.name,
"module",
m.name.rsplit(".", 1)[-1],
parent=enclosing if enclosing in module_names else None,
attrs=attrs,
)
taken: set[str] = set()
for d in m.defines:
nid = _id_for(m, d.qualname)
if nid in taken:
# Two definitions can share a qualified name — the same helper
# defined in both branches of an `if`, or a name rebound later.
# That is legal Python and the id has to stay unique, so the
# line disambiguates. Stable across runs, and it says which one.
nid = f"{nid}#L{d.lineno}"
taken.add(nid)
parent_qual = d.qualname.rsplit(".", 1)[0] if "." in d.qualname else None
parent = _id_for(m, parent_qual) if parent_qual else m.name
a = {"file": m.path, "line": d.lineno, "lines": max(1, d.end_lineno - d.lineno + 1)}
if d.doc:
a["doc"] = d.doc
g.node(nid, d.kind, d.name, parent=parent, attrs=a)
# -- edges, and the external nodes they force into existence -----------
external: dict[str, str] = {}
def _point_at(raw: str, module: Module) -> str | None:
"""The id an edge should target, creating an `external` node if needed."""
resolved = _resolve(raw, module, known)
if resolved and resolved in known:
return resolved
target = resolved or raw
if target not in known:
external.setdefault(target, target)
return target
for m in modules:
for target in dict.fromkeys(m.import_order): # dedupe, keep order
if not target:
continue
dest = target if target in known else None
if dest is None:
external.setdefault(target, target)
dest = target
if dest != m.name:
g.edge(m.name, dest, "imports")
for d in m.defines:
if d.kind != "class":
continue
src = _id_for(m, d.qualname)
for base in d.bases:
dest = _point_at(base, m)
if dest and dest != src:
g.edge(src, dest, "inherits")
for eid in sorted(external):
g.node(
eid,
"external",
eid.rsplit(".", 1)[-1],
attrs={"unresolved": True},
)
return g

View File

@@ -0,0 +1,268 @@
"""
Recorded traffic -> IR. What callers actually do, rather than what exists.
Reads a **HAR** file — the format browser devtools, mitmproxy, Charles and
Insomnia all export. Standard, JSON, stdlib-parseable, and already sitting on
most people's disk after ten minutes of using the thing they want documented.
python3 -m docgen.extractors usage --har session.har -o ir.json
## Why this exists
An OpenAPI document says what endpoints *are*. It does not say how to use them,
and the gap is where the whole difficulty lives:
- **The order.** Which call has to happen first. A spec is a set; usage is a
sequence, and the sequence is most of what a newcomer needs.
- **Which parameters matter.** A spec lists twenty optional query parameters.
Traffic shows the two that are always sent.
- **What a real payload looks like** — as opposed to a shape with every field
present and none of them meaning anything.
- **The endpoints that are not in the document at all**, which for a GraphQL
endpoint beside a REST surface is the normal case rather than an oversight.
- **Which responses actually happen.** A spec promises 200 and 404; traffic
shows the 422 that everyone hits.
None of that is recoverable by reading harder. It is only in the traffic.
## The two inferences, and their limits
**Path templating.** `/pets/123` and `/pets/456` are one endpoint. Any segment
that looks like an identifier — digits, a UUID, a long hex string — becomes
`{id}`, *whatever its format*: a route taking a numeric id on one call and a
UUID on the next is one route, and splitting it by format invents an endpoint
that does not exist. This is still a guess — a genuine path segment that happens
to be numeric gets templated wrongly — so `attrs.observed_paths` and
`attrs.id_formats` keep what was actually seen beside it.
**Sequence.** Consecutive calls become `follows` edges carrying how often that
pair occurred. Consecutive is not *caused by*, and one session's order is not
the only order — the weight is what separates a habit from an accident, and a
single recording will not tell you which.
Nothing here reads a response body's values beyond its shape, and no header is
copied into the IR: a HAR is full of cookies and bearer tokens, and none of them
belong in a document that gets committed.
"""
import json
import re
from collections import Counter
from pathlib import Path
from urllib.parse import parse_qs, urlsplit
from ..ir import Graph, Meta
UUID = re.compile(r"^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$", re.I)
LONG_HEX = re.compile(r"^[0-9a-f]{16,}$", re.I)
DIGITS = re.compile(r"^\d+$")
# Never copied into the IR. A HAR carries live credentials, and a document that
# gets committed must not.
SECRET_HEADERS = {"authorization", "cookie", "set-cookie", "x-api-key", "proxy-authorization"}
def _template(path: str) -> tuple[str, set[str]]:
"""`/pets/123` -> `/pets/{id}`. Returns (templated, id formats seen).
Every identifier-looking segment becomes `{id}`, whatever its format. A
caller that passes a numeric id on one call and a UUID on the next is using
**one** route, and templating them to `{id}` and `{uuid}` splits it into two
endpoints that do not exist — which is worse than the imprecision it avoids.
The formats are recorded instead, so "this route takes both" stays visible.
"""
out, formats = [], set()
for seg in path.split("/"):
if not seg:
out.append(seg)
continue
fmt = None
if DIGITS.match(seg):
fmt = "numeric"
elif UUID.match(seg):
fmt = "uuid"
elif LONG_HEX.match(seg):
fmt = "hash"
if fmt:
out.append("{id}")
formats.add(fmt)
else:
out.append(seg)
return "/".join(out), formats
def _body(entry: dict) -> dict | None:
post = (entry.get("request") or {}).get("postData") or {}
text = post.get("text")
if not text:
return None
try:
parsed = json.loads(text)
except (json.JSONDecodeError, TypeError):
return None
return parsed if isinstance(parsed, dict) else None
def _shape(value) -> str:
"""The type of a value, never the value. A payload is full of real data."""
if isinstance(value, bool):
return "bool"
if isinstance(value, int):
return "int"
if isinstance(value, float):
return "float"
if isinstance(value, list):
return "list"
if isinstance(value, dict):
return "object"
if value is None:
return "null"
return "str"
def _graphql(body: dict | None) -> tuple[str | None, str | None]:
"""(operation name, operation type) if this is a GraphQL call."""
if not body or "query" not in body:
return None, None
query = body.get("query")
if not isinstance(query, str):
return None, None
name = body.get("operationName")
m = re.match(r"\s*(query|mutation|subscription)?\s*([A-Za-z_]\w*)?", query)
op_type = (m.group(1) if m else None) or "query"
return (name or (m.group(2) if m and m.group(2) else "anonymous")), op_type
def extract(har_path, source: str = "usage", identity: str | None = None) -> Graph:
"""A HAR file -> IR of what was actually called."""
from ..book.larder import Larder
path = Path(har_path)
har = json.loads(path.read_text())
entries = (har.get("log") or {}).get("entries") or []
entries = sorted(entries, key=lambda e: e.get("startedDateTime", ""))
calls = [] # (key, kind, facts) in order
seen: dict[str, dict] = {}
# The unit is the HAR entry — one recorded request — not the distinct call.
# A capture's size is what it is, and the collapse from many entries to few
# endpoints is a fact worth being able to show against this number.
larder = Larder(kind="usage", identity=identity or path.name, unit="entry",
seen=len(entries))
for i, entry in enumerate(entries):
req = entry.get("request") or {}
res = entry.get("response") or {}
method = (req.get("method") or "GET").upper()
url = req.get("url") or ""
if not url:
# Without a URL there is nothing to template, and `urlsplit("")`
# yields an empty path that `_template` turns into "/". That made a
# malformed entry indistinguishable from a real call to the root.
# Recorded by name and skipped, because a capture is somebody's
# session and quietly inventing a request in it is worse than
# reporting a gap.
larder.fail(f"entries[{i}]", "no request URL")
continue
split = urlsplit(url)
templated, id_formats = _template(split.path or "/")
body = _body(entry)
op, op_type = _graphql(body)
if op:
key = f"{op_type} {op}"
kind = "operation"
else:
key = f"{method} {templated}"
kind = "endpoint"
record = seen.setdefault(
key,
{
"kind": kind,
"calls": 0,
"statuses": Counter(),
"params": Counter(),
"body_fields": Counter(),
"field_types": {},
"paths": Counter(),
"method": method,
"path": templated,
"id_formats": set(),
"op_type": op_type,
"host": split.netloc,
},
)
record["calls"] += 1
record["id_formats"] |= id_formats
status = res.get("status")
if status:
record["statuses"][int(status)] += 1
if split.path:
record["paths"][split.path] += 1
for name in parse_qs(split.query or ""):
record["params"][name] += 1
if body and not op:
for name, value in body.items():
record["body_fields"][name] += 1
record["field_types"].setdefault(name, _shape(value))
if op and isinstance(body.get("variables"), dict):
for name, value in body["variables"].items():
record["body_fields"][name] += 1
record["field_types"].setdefault(name, _shape(value))
calls.append(key)
g = Graph(Meta(source=source, root=path.name))
larder.extra["hosts"] = len({r["host"] for r in seen.values() if r.get("host")})
larder.extra["calls"] = len(seen)
g.meta.larder = larder.to_dict()
order = {}
for i, key in enumerate(calls):
order.setdefault(key, i)
for key, r in sorted(seen.items(), key=lambda kv: order[kv[0]]):
n = r["calls"]
attrs = {
"calls": n,
"statuses": sorted(r["statuses"]),
"first_seen": order[key],
}
if r["kind"] == "endpoint":
attrs["method"] = r["method"]
attrs["path"] = r["path"]
else:
attrs["protocol"] = "graphql"
attrs["operation"] = r["op_type"]
if r["host"]:
attrs["host"] = r["host"]
# Always sent vs sometimes sent is the distinction a spec cannot make.
always = sorted(p for p, c in r["params"].items() if c == n)
sometimes = sorted(p for p, c in r["params"].items() if c < n)
if always:
attrs["params_always"] = always
if sometimes:
attrs["params_sometimes"] = sometimes
if r["body_fields"]:
attrs["body_fields"] = [
{"name": f, "type": r["field_types"].get(f, "str"),
"always": r["body_fields"][f] == n}
for f in sorted(r["body_fields"])
]
if r["id_formats"]:
# The templating is a guess; keep what was actually seen beside it.
attrs["id_formats"] = sorted(r["id_formats"])
attrs["observed_paths"] = [p for p, _ in r["paths"].most_common(5)]
g.node(key, r["kind"], key, attrs=attrs)
# The sequence. Consecutive is not caused-by, so the weight is the signal.
pairs = Counter(zip(calls, calls[1:]))
for (a, b), weight in pairs.items():
if a == b:
continue # a repeated call is polling, not a step
g.edge(a, b, "follows", attrs={"weight": weight})
return g

View File

@@ -0,0 +1,32 @@
""" python3 -m docgen.extractors usage --har session.har [-o ir.json]"""
import argparse
import json
import sys
from pathlib import Path
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.extractors usage")
p.add_argument("--har", "-s", required=True, type=Path,
help="A HAR recording, as devtools/mitmproxy/Charles export.")
p.add_argument("--output", "-o", type=Path)
args = p.parse_args(argv)
from .usage import extract
try:
ir = extract(args.har)
except (OSError, json.JSONDecodeError, KeyError) as e:
print(f"Error: could not read {args.har}: {e}", file=sys.stderr)
return 1
text = json.dumps(ir.to_dict(), indent=2) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
eps = sum(1 for n in ir.nodes if n.kind == "endpoint")
ops = sum(1 for n in ir.nodes if n.kind == "operation")
print(f"{len(ir.nodes)} nodes ({eps} endpoints, {ops} graphql), "
f"{len(ir.edges)} sequence edges -> {args.output}")
else:
sys.stdout.write(text)
return 0

View File

@@ -0,0 +1,154 @@
# docgen's own OpenAPI fixture.
#
# Small, and shipped here rather than borrowed from station/tools/shuntgen so the
# framework test does not need the repo to be next door. A test's fixtures belong
# to the test — the suite still skips without modelgen, because reading OpenAPI
# genuinely needs it, but it no longer skips for want of a file.
#
# What it exercises, deliberately:
# - schemas that reference each other, so `foreign_key` edges have somewhere
# to come from and the ERD emitter has something to draw
# - an enum, which is a separate path in modelgen's reader
# - more than three paths, and one non-collection path with a parameter
# - a create body whose `id` is server-assigned, which is what the notebook
# emitter's "primary key is left out of a create body" check is about
openapi: 3.0.3
info:
title: Orders
version: "1.0.0"
paths:
/customers:
get:
operationId: listCustomers
responses:
"200":
description: Every customer.
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/Customer"
post:
operationId: createCustomer
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/Customer"
responses:
"201":
description: Created.
content:
application/json:
schema:
$ref: "#/components/schemas/Customer"
/customers/{customerId}:
get:
operationId: getCustomer
parameters:
- name: customerId
in: path
required: true
schema:
type: integer
responses:
"200":
description: One customer.
content:
application/json:
schema:
$ref: "#/components/schemas/Customer"
/orders:
get:
operationId: listOrders
parameters:
- name: status
in: query
required: false
schema:
$ref: "#/components/schemas/OrderStatus"
responses:
"200":
description: Orders, newest first.
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/Order"
post:
operationId: createOrder
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/Order"
responses:
"201":
description: Created.
content:
application/json:
schema:
$ref: "#/components/schemas/Order"
/orders/{orderId}/lines:
get:
operationId: listOrderLines
parameters:
- name: orderId
in: path
required: true
schema:
type: integer
responses:
"200":
description: The lines on one order.
content:
application/json:
schema:
type: array
items:
$ref: "#/components/schemas/OrderLine"
components:
schemas:
OrderStatus:
type: string
enum: [pending, paid, shipped, cancelled]
Customer:
type: object
required: [id, email]
properties:
id:
type: integer
description: Server-assigned.
email:
type: string
name:
type: string
Order:
type: object
required: [id, customer]
properties:
id:
type: integer
description: Server-assigned.
customer:
$ref: "#/components/schemas/Customer"
status:
$ref: "#/components/schemas/OrderStatus"
total:
type: number
OrderLine:
type: object
required: [id, order, sku]
properties:
id:
type: integer
order:
$ref: "#/components/schemas/Order"
sku:
type: string
quantity:
type: integer

View File

@@ -0,0 +1,6 @@
"""The IR: what a graph is. Structure only, no visual information."""
from .model import Edge, Graph, Meta, Node, SCHEMA_VERSION
from .validate import IRError, check, validate
__all__ = ["Graph", "Node", "Edge", "Meta", "SCHEMA_VERSION", "check", "validate", "IRError"]

View File

@@ -0,0 +1,8 @@
""" python3 -m docgen.ir <ir.json> — validate a document at the boundary."""
import sys
from .validate import main
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,164 @@
"""
The IR, as Python. Mirrors `schema.json`, which is the contract.
Stdlib dataclasses, not Pydantic. The IR's whole value is being a plain JSON
document that anything can read — a format that needs a library installed to be
opened is not a format, it is an API. Validation is a function at the boundary
(`validate.py`), not a property of the type.
The duplication between this file and `schema.json` is deliberate and is the one
place it is allowed. `validate.py::check_model_matches_schema` asserts the two
agree, so they cannot drift silently.
ir = Graph(meta=Meta(source="python", root="app/"))
ir.node("app.models.User", "class", "User", parent="app.models")
ir.edge("app.models.User", "app.db.Base", "inherits")
json.dumps(ir.to_dict())
## What must never appear here
No colour, no shape, no size, no position. `shape="cylinder"` is not a field —
it is `kind="datastore"` plus a style rule, which is what lets the same IR render
in a theme that has no cylinders. The previous iteration of this model carried
`cls`, `shape` and `style`, and that is exactly the mistake this replaces.
"""
from copy import deepcopy
from dataclasses import dataclass, field
from typing import Any
SCHEMA_VERSION = "1"
@dataclass
class Meta:
"""Where this graph came from.
`generated_at` defaults to None on purpose. A timestamp makes two
extractions of an unchanged tree differ, which destroys the diff emitter's
only useful property. Set it when the run time is the fact being recorded,
not by habit.
`larder` is what was actually read — see `book/larder.py`. It is optional so
that an IR written by hand, or by an extractor that cannot count its input,
is still valid; what it must never be is *wrong*. Absent means "not
measured", which a reader can act on. A zero would be a claim.
"""
source: str
root: str
schema_version: str = SCHEMA_VERSION
generated_at: str | None = None
larder: dict | None = None
def to_dict(self) -> dict:
out = {
"source": self.source,
"root": self.root,
"schema_version": self.schema_version,
"generated_at": self.generated_at,
}
# Omitted rather than null when unmeasured: `meta.larder` present with a
# null inside is two ways to say the same thing, and consumers end up
# checking for both.
#
# Deep-copied because the rest of this model copies too — `Node.to_dict`
# does `dict(self.attrs)`. A shallow copy is not enough here: `failed` is
# a list of dicts, so aliasing it would let a caller editing a serialised
# document reach back into the live measure.
if self.larder is not None:
out["larder"] = deepcopy(self.larder)
return out
@dataclass
class Node:
"""One thing. `kind` is the only field style and layout may key on."""
id: str
kind: str
label: str = ""
parent: str | None = None
attrs: dict[str, Any] = field(default_factory=dict)
def __post_init__(self):
if not self.label:
self.label = self.id.rsplit(".", 1)[-1]
def to_dict(self) -> dict:
return {
"id": self.id,
"kind": self.kind,
"label": self.label,
"parent": self.parent,
"attrs": dict(self.attrs),
}
@dataclass
class Edge:
"""One relationship. Containment is not one — that is `Node.parent`."""
source: str
target: str
kind: str
attrs: dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> dict:
return {
"source": self.source,
"target": self.target,
"kind": self.kind,
"attrs": dict(self.attrs),
}
@dataclass
class Graph:
"""The whole document."""
meta: Meta
nodes: list[Node] = field(default_factory=list)
edges: list[Edge] = field(default_factory=list)
# -- building ---------------------------------------------------------
def node(self, id: str, kind: str, label: str = "", **kw) -> Node:
n = Node(id, kind, label, **kw)
self.nodes.append(n)
return n
def edge(self, source: str, target: str, kind: str, **kw) -> Edge:
e = Edge(source, target, kind, **kw)
self.edges.append(e)
return e
def has(self, node_id: str) -> bool:
return any(n.id == node_id for n in self.nodes)
# -- serialising ------------------------------------------------------
def to_dict(self) -> dict:
"""Sorted, so the same tree twice produces the same bytes.
Sorting here rather than asking every extractor to emit in order: the
stability guarantee belongs to the format, not to each producer's
traversal order, and an extractor that visits files in directory order
is otherwise at the mercy of the filesystem.
"""
return {
"meta": self.meta.to_dict(),
"nodes": [n.to_dict() for n in sorted(self.nodes, key=lambda n: n.id)],
"edges": [
e.to_dict()
for e in sorted(self.edges, key=lambda e: (e.source, e.target, e.kind))
],
}
@classmethod
def from_dict(cls, data: dict) -> "Graph":
meta = Meta(**data["meta"])
g = cls(meta=meta)
g.nodes = [Node(**n) for n in data["nodes"]]
g.edges = [Edge(**e) for e in data["edges"]]
return g

View File

@@ -0,0 +1,114 @@
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"$id": "https://soleprint.local/atlas2/docgen/ir/schema.json",
"title": "docgen graph IR",
"description": "What a graph is. Structure only: no colour, no shape, no position. If a field would change between light and dark theme, it does not belong here. Extractors write this; emitters read it; neither knows the other exists.",
"type": "object",
"required": ["meta", "nodes", "edges"],
"additionalProperties": false,
"properties": {
"meta": {
"type": "object",
"required": ["source", "root", "schema_version"],
"additionalProperties": false,
"properties": {
"source": {
"type": "string",
"description": "Which extractor produced this: python, db, ..."
},
"root": {
"type": "string",
"description": "What it was pointed at. Relative where possible, so the IR is not machine-specific."
},
"schema_version": {
"type": "string",
"description": "This document's version. Bumped when the shape changes, so a consumer can refuse what it cannot read."
},
"generated_at": {
"type": ["string", "null"],
"description": "ISO-8601, or null. Null is the default and the honest one: a timestamp makes two extractions of the same tree differ, which breaks diffing. Set it only when the run time is itself the fact being recorded."
},
"larder": {
"type": ["object", "null"],
"description": "What was actually read: the first step of a book, and the only measure of the input this document carries. Absent means not measured, which a reader can act on; a zero would be a claim. Provenance lives here and never in nodes or attrs, the same split that keeps colour out of the IR.",
"required": ["kind", "identity", "unit", "seen", "read", "failed"],
"additionalProperties": false,
"properties": {
"kind": { "type": "string", "description": "Which extractor stocked it: python, code, db, openapi, usage." },
"identity": { "type": "string", "description": "The path, or a connection string with its credentials masked. Scrubbed at construction in book/larder.py and swept for in validate.py — the one field in the IR that could carry a secret." },
"unit": { "type": "string", "description": "What `seen` counts: file, table, path, entry, document. Closed vocabulary, so two larders can be compared." },
"seen": { "type": "integer", "description": "How many units the larder offered." },
"read": { "type": "integer", "description": "How many were consumed. Always seen - len(failed); derived on the dataclass so the two cannot disagree." },
"failed": {
"type": "array",
"description": "What could not be consumed, by name and reason. Named rather than counted: a count says a book is incomplete, a name says which part of it to distrust.",
"items": {
"type": "object",
"required": ["name", "error"],
"additionalProperties": false,
"properties": {
"name": { "type": "string" },
"error": { "type": "string" }
}
}
},
"extra": { "type": "object", "description": "Per-source facts that do not generalise: packages for a tree, dialect for a database, hosts for a capture." }
}
}
}
},
"nodes": {
"type": "array",
"items": {
"type": "object",
"required": ["id", "kind", "label"],
"additionalProperties": false,
"properties": {
"id": {
"type": "string",
"description": "Fully qualified and stable across runs. Stability is what makes two graphs from two commits diffable."
},
"kind": {
"type": "string",
"description": "The hinge of the whole system, and the ONLY field style and layout may key on. Small closed vocabulary per domain: module/class/function, table/column/view. `external` means a name that could not be resolved."
},
"label": {
"type": "string",
"description": "Human-facing short name. Distinct from id on purpose."
},
"parent": {
"type": ["string", "null"],
"description": "Containment only — a module contains a class. Not a relationship; relationships are edges. Must name another node or be null."
},
"attrs": {
"type": "object",
"description": "Open bag for domain data. file/line live here and are what let a UI link a box to a line. Anything not needed by every consumer goes here.",
"default": {}
}
}
}
},
"edges": {
"type": "array",
"items": {
"type": "object",
"required": ["source", "target", "kind"],
"additionalProperties": false,
"properties": {
"source": { "type": "string", "description": "A node id." },
"target": { "type": "string", "description": "A node id." },
"kind": {
"type": "string",
"description": "Small closed vocabulary per domain: imports, inherits, calls / foreign_key, references."
},
"attrs": { "type": "object", "default": {} }
}
}
}
},
"$comment": "Field names are shaped toward Cytoscape's {data: {id, source, target}} so an interactive canvas is nearly free. The banned-field list lives in validate.py, not here, because JSON Schema can say what is allowed but not why."
}

View File

@@ -0,0 +1,312 @@
"""
Check an IR document at the boundary.
Called wherever the IR crosses between layers: after an extractor writes one,
before an emitter reads one. That is the whole point of having one format — the
check is in one place instead of every consumer defending itself.
python3 -m docgen.ir path/to/ir.json
Stdlib only. This is not a general JSON Schema engine and does not want to be;
it reads the field lists **out of `schema.json`** so the two cannot drift, then
checks the handful of things that actually go wrong. `jsonschema` would validate
the shape and still miss every item below the first section, which are the ones
that produce a broken diagram.
## What it catches that a schema cannot
- an edge naming a node that does not exist
- a `parent` naming a node that does not exist, or a containment cycle
- duplicate ids
- **a visual field smuggled into the IR** — the architectural rule, as a check
"""
import json
import re
import sys
from pathlib import Path
HERE = Path(__file__).resolve().parent
SCHEMA_PATH = HERE / "schema.json"
# Not "fields we forgot to allow" — fields that mean the layering has broken.
# A colour in the IR means an extractor decided what something looks like, and
# from then on the graph renders one way forever. Named here rather than in
# schema.json because a schema can say a key is disallowed but not why.
VISUAL_KEYS = {
"color", "colour", "fill", "fillcolor", "bgcolor", "background",
"shape", "style", "penwidth", "stroke", "width", "height",
"font", "fontname", "fontsize", "fontcolor",
"pos", "x", "y", "rank", "layout", "theme", "class", "cls",
}
class IRError(ValueError):
"""An IR document that a consumer cannot safely read."""
def _schema() -> dict:
return json.loads(SCHEMA_PATH.read_text())
def _props(schema: dict, *path: str) -> dict:
node = schema
for step in path:
node = node["properties"][step] if "properties" in node else node[step]
return node
def _fields(schema: dict, section: str) -> tuple[set, set]:
"""(required, allowed) for `nodes`/`edges` items, read from the schema."""
item = schema["properties"][section]["items"]
return set(item.get("required", [])), set(item.get("properties", {}))
def check(data: dict, *, strict_visual: bool = True) -> list[str]:
"""Every problem, as a list. Empty means the document is sound.
A list rather than raising on the first: an extractor with four dangling
edges should report four, not make you run it four times.
"""
problems: list[str] = []
schema = _schema()
# -- top level --------------------------------------------------------
for key in schema["required"]:
if key not in data:
problems.append(f"missing top-level {key!r}")
if problems:
return problems # nothing below is meaningful without these
meta_schema = schema["properties"]["meta"]
for key in meta_schema["required"]:
if key not in data["meta"]:
problems.append(f"meta is missing {key!r}")
for key in data["meta"]:
if key not in meta_schema["properties"]:
problems.append(f"meta has unknown key {key!r}")
version = data["meta"].get("schema_version")
expected = _props(schema, "meta")["properties"]["schema_version"]
if version is not None and not isinstance(version, str):
problems.append(f"meta.schema_version must be a string, got {type(version).__name__}")
problems.extend(_check_larder(data["meta"].get("larder"), meta_schema))
# -- nodes ------------------------------------------------------------
node_required, node_allowed = _fields(schema, "nodes")
seen: set[str] = set()
for i, n in enumerate(data["nodes"]):
where = f"nodes[{i}]"
if not isinstance(n, dict):
problems.append(f"{where} is not an object")
continue
for key in node_required - set(n):
problems.append(f"{where} is missing {key!r}")
for key in set(n) - node_allowed:
problems.append(f"{where} has unknown key {key!r}")
nid = n.get("id")
if isinstance(nid, str):
if nid in seen:
problems.append(f"{where} id {nid!r} is declared more than once")
seen.add(nid)
# -- edges ------------------------------------------------------------
edge_required, edge_allowed = _fields(schema, "edges")
for i, e in enumerate(data["edges"]):
where = f"edges[{i}]"
if not isinstance(e, dict):
problems.append(f"{where} is not an object")
continue
for key in edge_required - set(e):
problems.append(f"{where} is missing {key!r}")
for key in set(e) - edge_allowed:
problems.append(f"{where} has unknown key {key!r}")
for end in ("source", "target"):
ref = e.get(end)
if isinstance(ref, str) and ref not in seen:
problems.append(f"{where} {end} names unknown node {ref!r}")
# -- containment ------------------------------------------------------
parents = {
n["id"]: n.get("parent")
for n in data["nodes"]
if isinstance(n, dict) and isinstance(n.get("id"), str)
}
for nid, parent in parents.items():
if parent is None:
continue
if parent not in seen:
problems.append(f"node {nid!r} has parent {parent!r}, which is not a node")
continue
# A containment cycle makes any tree walk non-terminating, and the index
# emitter is a tree walk.
slow, fast = nid, parent
while fast is not None and fast in parents:
if slow == fast:
problems.append(f"node {nid!r} is in a containment cycle")
break
fast = parents[fast]
if fast is None or fast not in parents:
break
fast = parents[fast]
slow = parents[slow]
# -- the architectural rule -------------------------------------------
if strict_visual:
for i, n in enumerate(data["nodes"]):
if isinstance(n, dict):
for key in set(n.get("attrs") or {}) & VISUAL_KEYS:
problems.append(
f"nodes[{i}].attrs.{key!r} is a visual field — "
"style belongs in style/*.json keyed on `kind`, not in the IR"
)
for i, e in enumerate(data["edges"]):
if isinstance(e, dict):
for key in set(e.get("attrs") or {}) & VISUAL_KEYS:
problems.append(
f"edges[{i}].attrs.{key!r} is a visual field — "
"style belongs in style/*.json keyed on `kind`, not in the IR"
)
return problems
# A credential that survived redaction, detected independently of the code that
# does the redacting. `book/larder.py` has its own key list; this has a second
# one on purpose. If the two ever disagree the check fires, which is the point —
# a scrubber that is graded by its own word is not graded. Same reason
# VISUAL_KEYS lives here and not in schema.json.
_DSN_PASSWORD = re.compile(r"[a-zA-Z][a-zA-Z0-9+.\-]*://[^:/@\s]+:(?P<secret>[^@/\s]+)@")
_SECRET_PARAM = re.compile(
r"(?i)\b(?:password|passwd|pwd|secret|token|access_token|refresh_token"
r"|api_key|apikey|sig|signature|credentials)\s*=\s*(?P<secret>[^&;\s]+)"
)
_MASKED = {"***", "xxx", "redacted", "[redacted]", "masked"}
def _check_larder(larder, meta_schema: dict) -> list[str]:
"""The input measure, checked — shape, arithmetic, and no leaked secret.
Absent is fine and means "not measured". Present and wrong is not, because
the whole value of the measure is that the number can be trusted against the
result; a larder claiming 47 reads when it read 45 is worse than no larder.
"""
if larder is None:
return []
if not isinstance(larder, dict):
return [f"meta.larder must be an object, got {type(larder).__name__}"]
problems = []
spec = meta_schema["properties"]["larder"]
for key in spec["required"]:
if key not in larder:
problems.append(f"meta.larder is missing {key!r}")
for key in larder:
if key not in spec["properties"]:
problems.append(f"meta.larder has unknown key {key!r}")
failed = larder.get("failed")
if failed is not None and not isinstance(failed, list):
problems.append("meta.larder.failed must be an array")
failed = None
elif failed:
for i, f in enumerate(failed):
if not isinstance(f, dict) or "name" not in f or "error" not in f:
problems.append(f"meta.larder.failed[{i}] needs both 'name' and 'error'")
# read is derived. Stored, it can disagree with itself, and this is where
# that shows up rather than in a book measure nobody can reconcile.
seen, read = larder.get("seen"), larder.get("read")
if isinstance(seen, int) and isinstance(read, int) and isinstance(failed, list):
if read != max(0, seen - len(failed)):
problems.append(
f"meta.larder.read is {read} but seen={seen} with {len(failed)} failed "
f"implies {max(0, seen - len(failed))} — the measure disagrees with itself"
)
if isinstance(read, int) and isinstance(seen, int) and read > seen:
problems.append(f"meta.larder.read ({read}) exceeds seen ({seen})")
unit = larder.get("unit")
if unit is not None and not isinstance(unit, str):
problems.append(f"meta.larder.unit must be a string, got {type(unit).__name__}")
identity = larder.get("identity")
if isinstance(identity, str):
for pattern, what in ((_DSN_PASSWORD, "a DSN password"),
(_SECRET_PARAM, "a secret parameter")):
m = pattern.search(identity)
if m and m.group("secret").lower() not in _MASKED:
problems.append(
f"meta.larder.identity still contains {what}"
"redact() in book/larder.py did not mask it, and this document "
"must not be written anywhere"
)
return problems
def validate(data: dict, **kw) -> dict:
"""check(), but raises. For use at a boundary where carrying on is wrong."""
problems = check(data, **kw)
if problems:
raise IRError(f"{len(problems)} problem(s):\n " + "\n ".join(problems))
return data
def check_model_matches_schema() -> list[str]:
"""The sanctioned duplication, asserted.
`model.py` mirrors `schema.json` by hand. This is what stops the two from
drifting: every field the schema declares must exist on the dataclass, and
every dataclass field must be declared in the schema.
"""
from .model import Edge, Meta, Node
schema = _schema()
problems = []
pairs = [
("meta", Meta, set(schema["properties"]["meta"]["properties"])),
("nodes", Node, _fields(schema, "nodes")[1]),
("edges", Edge, _fields(schema, "edges")[1]),
]
for name, cls, declared in pairs:
actual = set(cls.__dataclass_fields__)
for missing in declared - actual:
problems.append(f"schema declares {name}.{missing!r}; {cls.__name__} has no such field")
for extra in actual - declared:
problems.append(f"{cls.__name__} has {extra!r}; schema does not declare it")
return problems
def main(argv=None) -> int:
argv = sys.argv[1:] if argv is None else argv
if not argv:
print("usage: python3 -m docgen.ir <ir.json>", file=sys.stderr)
return 2
drift = check_model_matches_schema()
if drift:
print("model.py and schema.json disagree:", file=sys.stderr)
for d in drift:
print(f" {d}", file=sys.stderr)
return 1
path = Path(argv[0])
try:
data = json.loads(path.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {path}: {e}", file=sys.stderr)
return 1
problems = check(data)
if problems:
print(f"{path}: {len(problems)} problem(s)", file=sys.stderr)
for p in problems:
print(f" {p}", file=sys.stderr)
return 1
print(
f"{path}: ok — {len(data['nodes'])} nodes, {len(data['edges'])} edges, "
f"schema v{data['meta'].get('schema_version')}"
)
return 0

View File

@@ -0,0 +1,10 @@
"""A sanctioned place to try a dependency against real IRs before adopting it.
Nothing in `ir/`, `extractors/`, `ops/` or `emitters/` may import from here. When
an experiment earns its place it graduates into `ops/` behind an IR->IR
signature, and *then* the dependency is declared.
First candidate: networkx — transitive reduction, cycle detection, dominators.
The question to answer is which part is actually attractive, not whether the
whole library should be adopted on faith.
"""

View File

@@ -0,0 +1,149 @@
"""
EXPERIMENT — a live PostgreSQL schema, as a graphgen-compatible `schema.json`.
**Not a supported path, and deliberately not an extractor.** Reflecting a live
database is `modelgen from-db --url ...`, which already does it across every
SQLAlchemy dialect and writes exactly the file this writes. This exists because
SQLAlchemy is not installed on this machine and the IR still needed testing
against a real schema rather than a fixture — which is what `lab/` is for.
If a psql-based path ever turns out to be worth keeping, it belongs in modelgen
beside the other extractors, not here and not in `extractors/`.
python3 -m docgen.lab.pg_probe --db money26 --port 5433 -o schema.json
## No credentials on the command line
There is no `--password` and no DSN argument, on purpose. `psql` is invoked with
host/port/dbname/user and left to find credentials the way it normally does —
`~/.pgpass`, peer auth, `PGPASSWORD` in the environment. A connection string
passed as an argument lands in shell history and in `ps`, visible to every other
user on the box, and `modelgen/__main__.py:202` additionally prints it to
stdout, so it lands in CI logs too. That is worth not reproducing.
Nothing here reads a row. Only `information_schema` and `pg_catalog`, which is
structure — table names, column names, types, keys. No table data is selected,
so a schema can be drawn without the contents being touched.
"""
import argparse
import json
import subprocess
import sys
# One query, returning JSON, because assembling this from three result sets in
# Python is more code and more ways to get the joins wrong.
QUERY = r"""
SELECT json_build_object(
'tables', (
SELECT COALESCE(json_agg(t), '[]'::json) FROM (
SELECT c.relname AS name,
obj_description(c.oid) AS doc,
(SELECT COALESCE(json_agg(f ORDER BY f->>'ord'), '[]'::json) FROM (
SELECT json_build_object(
'name', a.attname,
'ord', a.attnum,
'type', format_type(a.atttypid, a.atttypmod),
'notnull', a.attnotnull,
'pk', COALESCE((
SELECT true FROM pg_constraint pk
WHERE pk.conrelid = c.oid AND pk.contype = 'p'
AND a.attnum = ANY (pk.conkey)), false),
'fk', (
SELECT ref.relname FROM pg_constraint fk
JOIN pg_class ref ON ref.oid = fk.confrelid
WHERE fk.conrelid = c.oid AND fk.contype = 'f'
AND a.attnum = ANY (fk.conkey)
LIMIT 1)
) AS f
FROM pg_attribute a
WHERE a.attrelid = c.oid AND a.attnum > 0 AND NOT a.attisdropped
) fields) AS fields
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
WHERE n.nspname = 'public' AND c.relkind IN ('r', 'p', 'v', 'm')
ORDER BY c.relname
) t
)
) AS payload;
"""
def probe(db: str, host: str = "localhost", port: int = 5432, user: str | None = None) -> dict:
"""Read structure via psql. Returns the graphgen-compatible schema dict."""
cmd = ["psql", "-h", host, "-p", str(port), "-d", db, "-tAq", "-c", QUERY]
if user:
cmd[1:1] = ["-U", user]
proc = subprocess.run(cmd, capture_output=True, text=True)
if proc.returncode != 0:
# psql puts the reason on stderr; pass it through rather than guessing.
raise RuntimeError(proc.stderr.strip() or f"psql exited {proc.returncode}")
payload = json.loads(proc.stdout.strip())
models = {}
for table in payload["tables"]:
fields = {}
for f in table["fields"]:
if f["fk"]:
type_str = f"FK:{f['fk']}"
else:
type_str = _simplify(f["type"])
entry = {"type": type_str}
if f["pk"]:
entry["pk"] = True
if not f["notnull"]:
entry["nullable"] = True
fields[f["name"]] = entry
model = {"fields": fields}
if table.get("doc"):
model["doc"] = table["doc"]
models[table["name"]] = model
return {"models": models}
# Postgres type names are more precise than a diagram needs; the IR keeps what
# was read, and this only shortens the common ones so a box stays readable.
_SIMPLE = {
"integer": "int", "bigint": "int", "smallint": "int",
"character varying": "str", "text": "str", "character": "str",
"boolean": "bool", "double precision": "float", "real": "float",
"timestamp without time zone": "datetime", "timestamp with time zone": "datetime",
"date": "date", "time without time zone": "time",
"jsonb": "json", "uuid": "uuid",
}
def _simplify(pg_type: str) -> str:
base = pg_type.split("(")[0].strip()
return _SIMPLE.get(base, base)
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.lab.pg_probe")
p.add_argument("--db", required=True)
p.add_argument("--host", default="localhost")
p.add_argument("--port", type=int, default=5432)
p.add_argument("--user")
p.add_argument("--output", "-o")
args = p.parse_args(argv)
try:
schema = probe(args.db, args.host, args.port, args.user)
except (RuntimeError, json.JSONDecodeError) as e:
print(f"Error: {e}", file=sys.stderr)
return 1
text = json.dumps(schema, indent=2) + "\n"
if args.output:
with open(args.output, "w") as fh:
fh.write(text)
tables = len(schema["models"])
cols = sum(len(m["fields"]) for m in schema["models"].values())
print(f"{tables} tables, {cols} columns -> {args.output}")
else:
sys.stdout.write(text)
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,14 @@
"""
The notebook layer: a sequence, a hand-written overlay, and the merge.
from docgen.notebook import from_ir, merge, scaffold
base, _ = merge(from_ir(ir), load("overlay.json"))
`spec.py` holds the format. The `.ipynb` writer is `emitters/notebook.py`, which
renders a merged spec and knows nothing about where it came from.
"""
from .spec import SPEC_VERSION, dump, from_ir, load, merge, scaffold
__all__ = ["from_ir", "merge", "scaffold", "load", "dump", "SPEC_VERSION"]

View File

@@ -0,0 +1,263 @@
"""
The notebook, as a sequence — before it is a `.ipynb`.
A **notebook spec** is a declarative, ordered list of steps: what the document
does, in order, with no Jupyter in it. It is to a notebook what an OpenAPI
document is to an API — the thing you can read, diff, version and generate
*from*, rather than the artifact you end up opening.
IR ──► spec ──(+ overlay)──► spec ──► .ipynb
generated hand-written merged emitted
## Why there are two files and not one
Generation alone gives you a document that is never stale and never says
anything a parser could not work out. Hand-authoring gives you insight and a
document that rots. The split is the only arrangement that gets both:
base regenerated every time, never edited, holds what was extracted
overlay small, hand-written, the only file anyone edits, re-applied on
every build
So a regeneration cannot lose someone's work, and someone's work cannot go
stale silently — if the base moves under an overlay, the overlay's step ids stop
matching and that is reported rather than quietly dropped.
**Extraction must work with the overlay absent**, which is the same rule the
brief sets for LLM annotation. The overlay is an addition, never a dependency.
## What an overlay is for
The honest reason it exists: **a spec says what endpoints are, not how to use
them.** Order of calls, which fields actually matter, a payload that is real
rather than shaped, the GraphQL endpoint sitting beside the REST ones, the
non-RESTful verb that takes a body nobody would guess. None of that is derivable,
and all of it is what makes a walkthrough worth reading.
annotate add prose to a generated step — the common case
replace override the generated code where the generated guess is wrong
insert a step generation could not know about
drop hide a step that is noise here
order pin the sequence, when the order is the point
`replace` is the one that matters most: it is how real usage gets into the
document today, and it keeps working unchanged when a usage extractor exists.
"""
import json
import re
from pathlib import Path
SPEC_VERSION = "1"
# Step kinds. Small and closed, the same discipline the IR's `kind` follows.
KINDS = ("md", "code", "params", "call")
def _step(id: str, kind: str, **fields) -> dict:
step = {"id": id, "kind": kind}
step.update({k: v for k, v in fields.items() if v is not None})
return step
def from_ir(ir: dict, base_url: str = "https://api.example.invalid",
env_var: str = "API_TOKEN") -> dict:
"""The generated half: everything extraction can know, in order."""
meta = ir.get("meta", {})
root = meta.get("root", "api")
fields_of: dict[str, list] = {}
for n in ir["nodes"]:
if n["kind"] == "column" and n.get("parent"):
fields_of.setdefault(n["parent"], []).append(n)
# Observed order wins over alphabetical. A usage recording knows which call
# comes first, and that sequence is most of what a newcomer needs — sorting
# it away would throw out the one thing a spec could not have told us.
def _order(n):
a = n.get("attrs") or {}
if "first_seen" in a:
return (0, a["first_seen"], "")
return (1, 0, f'{a.get("path", "")} {a.get("method", "")}')
endpoints = sorted(
(n for n in ir["nodes"] if n["kind"] in ("endpoint", "operation")),
key=_order,
)
steps = [
_step("intro", "md", title=root,
text=(
f"Generated from `{root}`. The generated steps below are "
"rebuilt on every run and must not be edited here — put "
"changes in the overlay, which is re-applied each time."
)),
_step("setup", "params", title="Parameters",
base_url=base_url, env_var=env_var),
_step("client", "code", title="The client", builtin="client"),
]
for n in endpoints:
a = n.get("attrs") or {}
# A usage recording carries facts a spec cannot: which parameters are
# *always* sent, which status codes really happen, how often it is
# called. Where they exist they are better than anything generated from
# a schema, so they win.
body_fields = None
if a.get("body_fields"):
raw = a["body_fields"]
if raw and isinstance(raw[0], dict) and "always" in raw[0]:
body_fields = [f for f in raw if f.get("always")] or raw
else:
body_fields = raw
elif a.get("request_model"):
body_fields = [
{"name": f.get("label") or f["id"].rsplit(".", 1)[-1],
**(f.get("attrs") or {})}
for f in fields_of.get(a["request_model"], [])
] or None
steps.append(
_step(
n["id"], "call",
title=n["id"] if n["kind"] == "operation"
else f'{a.get("method", "GET")} {a.get("path", "/")}',
summary=a.get("summary"),
method=a.get("method", "POST" if n["kind"] == "operation" else "GET"),
path=a.get("path", "/graphql" if n["kind"] == "operation" else "/"),
status=a.get("status"),
request_model=a.get("request_model"),
response_model=a.get("response_model"),
returns_list=a.get("returns_list"),
# A usage-derived path already carries its placeholders; a
# spec-derived one lists them separately. Either way the cell
# needs a variable, not a literal `{id}` that would 404.
path_params=a.get("path_params")
or re.findall(r"\{(\w+)\}", a.get("path", "")) or None,
body_fields=body_fields,
# usage-only
params_always=a.get("params_always"),
params_sometimes=a.get("params_sometimes"),
statuses=a.get("statuses"),
calls=a.get("calls"),
id_formats=a.get("id_formats"),
observed_paths=a.get("observed_paths"),
graphql=(n["kind"] == "operation") or None,
)
)
shapes = sorted(n["id"] for n in ir["nodes"] if n["kind"] == "table")
if shapes:
steps.append(
_step("shapes", "md", title="Shapes",
text="The models these endpoints carry.",
table=[
{"name": s,
"fields": [f.get("label") or f["id"].rsplit(".", 1)[-1]
for f in fields_of.get(s, [])]}
for s in shapes
])
)
return {
"meta": {
"source": meta.get("source", ""),
"root": root,
"spec_version": SPEC_VERSION,
},
"steps": steps,
}
def scaffold(spec: dict) -> dict:
"""A blank overlay for this spec — every step id, nothing filled in.
Handing someone the list of ids is the difference between an overlay being
written and an overlay being meant to be written.
"""
return {
"note": (
"Hand-written. Re-applied on every generation, so edits here survive "
"and edits to the notebook do not. Keys are step ids from the spec; "
"an id that no longer exists is reported, never silently ignored."
),
"steps": {
s["id"]: {"_kind": s["kind"], "_title": s.get("title", "")}
for s in spec["steps"]
},
"insert": [],
"drop": [],
"order": [],
}
def merge(spec: dict, overlay: dict | None) -> tuple[dict, list[str]]:
"""Apply an overlay. Returns (spec, problems).
Problems are returned rather than raised: an overlay pointing at a step that
no longer exists means the base moved underneath it, which is worth saying
out loud and is not a reason to refuse to build the document.
"""
if not overlay:
return spec, []
problems: list[str] = []
steps = [dict(s) for s in spec["steps"]]
by_id = {s["id"]: s for s in steps}
for sid, patch in (overlay.get("steps") or {}).items():
if sid not in by_id:
problems.append(f"overlay targets step {sid!r}, which the spec no longer has")
continue
step = by_id[sid]
for key, value in patch.items():
if key.startswith("_") or value in (None, "", [], {}):
continue # scaffold hints and unfilled slots are not edits
step[key] = value
for sid in overlay.get("drop") or []:
if sid not in by_id:
problems.append(f"overlay drops step {sid!r}, which the spec no longer has")
continue
steps = [s for s in steps if s["id"] != sid]
by_id.pop(sid, None)
for extra in overlay.get("insert") or []:
step = {k: v for k, v in extra.items() if k != "after"}
step.setdefault("kind", "md")
if "id" not in step:
problems.append("an inserted step has no id; it cannot be re-applied reliably")
continue
after = extra.get("after")
if after is None:
steps.append(step)
elif after in {s["id"] for s in steps}:
at = next(i for i, s in enumerate(steps) if s["id"] == after) + 1
steps.insert(at, step)
else:
problems.append(f"inserted step {step['id']!r} goes after {after!r}, which is gone")
steps.append(step)
by_id[step["id"]] = step
order = overlay.get("order") or []
if order:
missing = [sid for sid in order if sid not in by_id]
problems += [f"overlay orders step {sid!r}, which the spec no longer has"
for sid in missing]
pinned = [by_id[sid] for sid in order if sid in by_id]
rest = [s for s in steps if s["id"] not in set(order)]
steps = pinned + rest
return {"meta": dict(spec["meta"]), "steps": steps}, problems
def load(path) -> dict:
return json.loads(Path(path).read_text())
def dump(spec: dict, path) -> Path:
path = Path(path)
path.parent.mkdir(parents=True, exist_ok=True)
path.write_text(json.dumps(spec, indent=2, sort_keys=True) + "\n")
return path

View File

@@ -0,0 +1,27 @@
"""
Views over the IR. Filtering is not an emitter concern — it belongs here, once,
so the index, the diagram and the diff all narrow the same way.
python3 -m docgen.ops ir.json --drop-stdlib --only class -o smaller.json
"""
from .filter import (
classify,
collapse_to_depth,
drop_builtins,
drop_external,
drop_kinds,
drop_stdlib,
neighbourhood,
only_kinds,
overview,
shape,
split,
subtree,
)
__all__ = [
"overview",
"drop_stdlib", "drop_external", "drop_builtins", "drop_kinds", "only_kinds",
"subtree", "neighbourhood", "collapse_to_depth", "shape", "split", "classify",
]

View File

@@ -0,0 +1,101 @@
""" python3 -m docgen.ops <ir.json> [views...] [-o out.json]
Views compose, left to right, in the order given on the command line."""
import argparse
import json
import sys
from pathlib import Path
from ..ir import check
from . import filter as F
def main(argv=None):
p = argparse.ArgumentParser(prog="python3 -m docgen.ops")
p.add_argument("ir", type=Path)
p.add_argument("--output", "-o", type=Path)
p.add_argument("--overview", action="store_true",
help="The default view for this source type. Usually what you want.")
p.add_argument("--drop-stdlib", action="store_true", help="Remove stdlib externals.")
p.add_argument("--drop-builtins", action="store_true", help="Remove builtin externals.")
p.add_argument("--drop-external", action="store_true", help="Remove every unresolved name.")
p.add_argument("--only", action="append", default=[], help="Keep only this kind. Repeatable.")
p.add_argument("--drop", action="append", default=[], help="Remove this kind. Repeatable.")
p.add_argument("--subtree", help="Just this node id and its contents.")
p.add_argument("--around", help="This node id and its neighbours.")
p.add_argument("--hops", type=int, default=1)
p.add_argument("--depth", type=int, help="Collapse to this containment depth.")
p.add_argument("--split", action="store_true",
help="Write one document per subsystem into OUT/ (a directory).")
p.add_argument("--shape", action="store_true",
help="Report what this will look like, and write nothing.")
args = p.parse_args(argv)
try:
ir = json.loads(args.ir.read_text())
except (OSError, json.JSONDecodeError) as e:
print(f"Error: could not read {args.ir}: {e}", file=sys.stderr)
return 1
before = (len(ir["nodes"]), len(ir["edges"]))
if args.overview:
ir = F.overview(ir)
if args.drop_stdlib:
ir = F.drop_stdlib(ir)
if args.drop_builtins:
ir = F.drop_builtins(ir)
if args.drop_external:
ir = F.drop_external(ir)
if args.drop:
ir = F.drop_kinds(ir, args.drop)
if args.only:
ir = F.only_kinds(ir, args.only)
if args.subtree:
ir = F.subtree(ir, args.subtree)
if args.around:
ir = F.neighbourhood(ir, args.around, hops=args.hops)
if args.depth is not None:
ir = F.collapse_to_depth(ir, args.depth)
if args.shape:
sh = F.shape(ir)
for k, v in sh.items():
print(f" {k:<14} {v}")
return 0
if args.split:
if not args.output:
print("Error: --split needs -o DIRECTORY", file=sys.stderr)
return 1
args.output.mkdir(parents=True, exist_ok=True)
for name, part in F.split(ir).items():
(args.output / f"{name}.json").write_text(json.dumps(part, indent=2) + "\n")
sh = F.shape(part)
print(f" {name:<16} {sh['nodes']:>4} nodes {sh['edges']:>4} edges "
f"-> {args.output / (name + '.json')}")
return 0
problems = check(ir)
if problems:
# A view that produces an invalid document is a bug in the view, and it
# must not be written out for an emitter to trip over later.
print(f"Error: the view produced an invalid IR ({len(problems)}):", file=sys.stderr)
for pr in problems[:5]:
print(f" {pr}", file=sys.stderr)
return 1
text = json.dumps(ir, indent=2) + "\n"
if args.output:
args.output.parent.mkdir(parents=True, exist_ok=True)
args.output.write_text(text)
print(f" {before[0]} nodes, {before[1]} edges -> "
f"{len(ir['nodes'])} nodes, {len(ir['edges'])} edges -> {args.output}")
else:
sys.stdout.write(text)
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,484 @@
"""
Views: IR in, smaller IR out.
A view is a *filter over the graph*, not a rendering option. Putting it here
rather than behind an emitter flag is the difference between one implementation
and one per output — the index, the DOT and the diff all want "just this
subsystem, two hops out, without the stdlib", and none of them should own it.
Everything here is a pure function returning a new IR document. They compose:
ir = drop_stdlib(ir)
ir = neighbourhood(ir, "docgen.ir", hops=2)
## Why this exists at all
The first real diagram produced from this pipeline was a 3000px-wide strip: four
modules of actual content and sixty `sys`/`json`/`typing` boxes, all peers. The
emitter was correct and the picture was useless. That is a missing view, not a
broken renderer, and it is the clearest argument for keeping this layer separate
— the fix belongs to every consumer at once.
## What is not here
Graph algorithms. Transitive reduction, cycle detection, dominators and
shortest paths are `networkx`'s, and reimplementing them is the classic way to
acquire a quiet bug. `lab/` is where that dependency gets tried against real
IRs before anything here depends on it.
"""
import builtins
import sys
# Exact, and needs no hand-maintained list that goes stale one release later.
STDLIB = set(getattr(sys, "stdlib_module_names", ()))
def _rebuild(ir: dict, keep: set[str]) -> dict:
"""A new document holding only `keep`, with dangling references repaired.
A node whose parent was filtered out is re-parented to its nearest surviving
ancestor rather than dropped — losing a class because its module was
filtered would be a surprise, and an orphan fails validation.
"""
by_id = {n["id"]: n for n in ir["nodes"]}
def surviving_parent(nid: str):
parent = by_id.get(nid, {}).get("parent")
while parent is not None and parent not in keep:
parent = by_id.get(parent, {}).get("parent")
return parent
nodes = []
for n in ir["nodes"]:
if n["id"] not in keep:
continue
node = dict(n)
node["parent"] = surviving_parent(n["id"])
nodes.append(node)
def lift(nid: str):
"""The surviving node that should carry an edge touching `nid`."""
cur = nid
while cur is not None and cur not in keep:
cur = by_id.get(cur, {}).get("parent")
return cur
# Edges are **lifted, not dropped**. A class in module A inheriting from a
# class in module B is a dependency of A on B; if you then look at the
# module level, dropping the edge says the two modules are unrelated, which
# is false. Collapsing docgen to its packages lost 69 of 77 edges this way
# and the picture was worse for it — not simpler, wrong.
merged: dict[tuple, dict] = {}
for e in ir["edges"]:
src, dst = lift(e["source"]), lift(e["target"])
if src is None or dst is None or src == dst:
# Both ends inside one surviving node: an internal relationship,
# which is a fact about that node rather than a line between two.
continue
key = (src, dst, e["kind"])
if key in merged:
merged[key]["attrs"]["weight"] = merged[key]["attrs"].get("weight", 1) + 1
continue
attrs = dict(e.get("attrs") or {})
lifted = src != e["source"] or dst != e["target"]
if lifted:
# `weight` is how many underlying relationships this line stands
# for. Semantic, not visual — whether it becomes a thicker stroke is
# the style layer's call, not this one's.
attrs["weight"] = 1
attrs.pop("label", None) # a single field name no longer applies
merged[key] = {"source": src, "target": dst, "kind": e["kind"], "attrs": attrs}
return {"meta": dict(ir["meta"]), "nodes": nodes, "edges": list(merged.values())}
def drop_kinds(ir: dict, kinds) -> dict:
"""Remove every node of these kinds, and their descendants."""
kinds = set(kinds)
doomed = {n["id"] for n in ir["nodes"] if n["kind"] in kinds}
# A surviving child of a removed node would be an orphan with a real parent
# id pointing at nothing, so descendants go too.
changed = True
while changed:
changed = False
for n in ir["nodes"]:
if n["id"] not in doomed and n.get("parent") in doomed:
doomed.add(n["id"])
changed = True
return _rebuild(ir, {n["id"] for n in ir["nodes"]} - doomed)
def only_kinds(ir: dict, kinds) -> dict:
"""Keep only these kinds. Ancestors are kept so containment survives."""
kinds = set(kinds)
by_id = {n["id"]: n for n in ir["nodes"]}
keep = set()
for n in ir["nodes"]:
if n["kind"] not in kinds:
continue
keep.add(n["id"])
parent = n.get("parent")
while parent and parent not in keep:
keep.add(parent)
parent = by_id.get(parent, {}).get("parent")
return _rebuild(ir, keep)
def drop_stdlib(ir: dict) -> dict:
"""Remove external nodes that are the standard library.
The usual first view: `pathlib` and `typing` are true dependencies and
almost never the thing being looked at. Third-party externals stay, because
*those* are the dependency surface worth seeing.
"""
keep = set()
for n in ir["nodes"]:
if n["kind"] == "external" and n["id"].split(".")[0] in STDLIB:
continue
keep.add(n["id"])
return _rebuild(ir, keep)
def drop_external(ir: dict) -> dict:
"""Remove every unresolved name. What is left is this tree talking to itself."""
return drop_kinds(ir, {"external"})
def subtree(ir: dict, root_id: str) -> dict:
"""Just this node and everything inside it."""
children = {}
for n in ir["nodes"]:
children.setdefault(n.get("parent"), []).append(n["id"])
keep, stack = set(), [root_id]
while stack:
nid = stack.pop()
if nid in keep:
continue
keep.add(nid)
stack.extend(children.get(nid, []))
return _rebuild(ir, keep)
def neighbourhood(ir: dict, node_id: str, hops: int = 1, *, undirected: bool = True,
with_contents: bool = False) -> dict:
"""This node and everything within `hops` edges of it.
The view behind "what does this touch, and what touches it". Containment
ancestors are kept so the result still nests.
"""
by_id = {n["id"]: n for n in ir["nodes"]}
out, inc = {}, {}
for e in ir["edges"]:
out.setdefault(e["source"], set()).add(e["target"])
inc.setdefault(e["target"], set()).add(e["source"])
frontier, keep = {node_id}, {node_id}
for _ in range(hops):
nxt = set()
for nid in frontier:
nxt |= out.get(nid, set())
if undirected:
nxt |= inc.get(nid, set())
nxt -= keep
keep |= nxt
frontier = nxt
for nid in list(keep):
parent = by_id.get(nid, {}).get("parent")
while parent and parent not in keep:
keep.add(parent)
parent = by_id.get(parent, {}).get("parent")
if with_contents:
# A table without its columns is not a table. Ancestors come along by
# default because the tree has to stay whole; descendants do not, and
# for anything card-shaped they are the substance.
children: dict[str, list] = {}
for n in ir["nodes"]:
if n.get("parent"):
children.setdefault(n["parent"], []).append(n["id"])
stack = list(keep)
while stack:
for kid in children.get(stack.pop(), ()):
if kid not in keep:
keep.add(kid)
stack.append(kid)
return _rebuild(ir, keep)
def collapse_to_depth(ir: dict, depth: int) -> dict:
"""Keep nodes at or above `depth`, dropping what is inside them.
Different from the emitter's `--max-depth`, and the difference is the point:
that one still draws the deep nodes and merely stops nesting them, while
this removes them from the graph, so an index built afterwards agrees with
the diagram.
"""
by_id = {n["id"]: n for n in ir["nodes"]}
def level(nid: str) -> int:
n, d = by_id.get(nid), 0
while n and n.get("parent"):
d += 1
n = by_id.get(n["parent"])
return d
return _rebuild(ir, {n["id"] for n in ir["nodes"] if level(n["id"]) <= depth})
def drop_builtins(ir: dict) -> dict:
"""Remove externals that are builtins — `ValueError`, `RuntimeError`.
`sys.stdlib_module_names` does not cover these: they are names in
`builtins`, not modules, so `drop_stdlib` leaves them. At overview level
"this exception subclasses ValueError" is true and not what anyone came to
find out.
"""
keep = {
n["id"] for n in ir["nodes"]
if not (n["kind"] == "external" and hasattr(builtins, n["id"].split(".")[-1])
and "." not in n["id"])
}
return _rebuild(ir, keep)
def overview(ir: dict) -> dict:
"""The default view — what you want without having to ask for it.
Dispatches on `meta.source`, because a good default for a codebase and a
good default for a schema are not the same picture:
python modules, no stdlib, no builtin exceptions. Third-party
externals stay: those are the dependency surface.
db tables and their foreign keys. A schema's 254 columns are a
reference, not a diagram.
**Modules, not a depth cut.** Depth looks like the obvious knob and is the
wrong one on a real tree: a directory without `__init__.py` is not a package,
so its modules have no parent and sit at depth 0. soleprint has **173 such
roots**, which left `collapse_to_depth(2)` holding 566 functions and 142
classes — a cut that reduced the count and not the noise. Selecting by
`kind` does not care how the directories happen to be arranged.
Edges are lifted rather than dropped on the way (see `_rebuild`), so
collapsing detail turns relationships into higher-level ones instead of
losing them. That is the whole difference between a simpler picture and a
wrong one.
An unknown source gets `drop_stdlib` and nothing else — a conservative
default beats a confident wrong one.
"""
source = (ir.get("meta") or {}).get("source", "")
if source == "db":
return only_kinds(ir, {"table"})
if source == "python":
# `external` is kept alongside `module`: those nodes are the dependency
# surface, and dropping them is how a diagram comes to look complete
# while saying nothing about what the code reaches outside itself.
return only_kinds(drop_builtins(drop_stdlib(ir)), {"module", "external"})
return drop_stdlib(ir)
def shape(ir: dict) -> dict:
"""What this graph will look like before anything renders it.
Aspect ratio is not a rendering accident, it is a property of the graph:
a layered engine puts everything at the same dependency level side by side,
so the widest level *is* the width. soleprint's overview is 7 levels by 109
nodes, and no engine draws that well — dot, ELK and dagre all layer, and
the sheet is the sheet.
So it is worth knowing *before* writing a 375 KB image, which is why this
computes it from the IR rather than measuring the output.
Measured, one diagram per subsystem:
<= 20 nodes ~1.6:1 readable
70-106 nodes ~7:1 a strip
261 nodes ~14:1 unusable
Roughly twenty nodes is where it stops being a diagram.
"""
ids = {n["id"] for n in ir["nodes"]}
out, inc, adj = {}, {}, {i: set() for i in ids}
for e in ir["edges"]:
out.setdefault(e["source"], set()).add(e["target"])
inc.setdefault(e["target"], set()).add(e["source"])
adj[e["source"]].add(e["target"])
adj[e["target"]].add(e["source"])
seen, components = set(), 0
isolated = 0
for i in ids:
if i in seen:
continue
components += 1
stack, size = [i], 0
while stack:
x = stack.pop()
if x in seen:
continue
seen.add(x)
size += 1
stack.extend(adj[x] - seen)
if size == 1:
isolated += 1
# Longest-path ranking, which is what a layered engine assigns.
rank: dict[str, int] = {}
def rank_of(n, guard=frozenset()):
if n in rank:
return rank[n]
if n in guard:
return 0
parents = inc.get(n, ())
value = 1 + max([rank_of(p, guard | {n}) for p in parents], default=-1)
rank[n] = value
return value
for i in ids:
rank_of(i)
widths: dict[int, int] = {}
for level in rank.values():
widths[level] = widths.get(level, 0) + 1
return {
"nodes": len(ids),
"edges": len(ir["edges"]),
"components": components,
"isolated": isolated,
"levels": len(widths),
"widest_level": max(widths.values(), default=0),
}
def split(ir: dict, depth: int = 0) -> dict[str, dict]:
"""One IR per subsystem — the answer to a graph too big for one picture.
Splits on the `depth`-th segment of the id, so `depth=0` gives one document
per top-level package. Edges that cross between subsystems are kept on both
sides, because "this is what my subsystem reaches" is the useful half of a
crossing edge and dropping it would understate the coupling.
"""
groups: dict[str, set] = {}
for n in ir["nodes"]:
parts = n["id"].split(".")
key = ".".join(parts[: depth + 1])
groups.setdefault(key, set()).add(n["id"])
out: dict[str, dict] = {}
for key, keep in sorted(groups.items()):
if len(keep) < 2:
continue
touching = {
e["source"] for e in ir["edges"] if e["target"] in keep
} | {e["target"] for e in ir["edges"] if e["source"] in keep}
visible = keep | (touching & {n["id"] for n in ir["nodes"]})
out[key] = _rebuild(ir, visible)
return out
def classify(ir: dict) -> dict:
"""What kind of graph is this, and what should draw it.
The lesson that produced this: **a diagram that fights its layout engine is
usually the wrong kind of diagram.** soleprint's module graph rendered 14:1
through `dot`, and no Graphviz flag fixed it because a 7-level, 109-wide
sheet has no good rectangular form. The same database rendered 235:1 through
`dot` and 0.9:1 through `erd` — not because one engine is better, but
because a schema is a set of peer entities with references, and drawing it in
dependency ranks was never the right shape.
So: read the structure, then pick the form. Returns the kind, the emitter
that suits it, and why — the reason matters, because "use the index" is
advice somebody will override unless they know what it is based on.
erd entities with references -> emitters/erd
layered a DAG that is taller than
it is wide -> emitters/dot
tree containment, few cross edges -> emitters/dot
flat many peers, no structure -> emitters/index (a list)
sheet too wide at every level -> emitters/index, or split
"""
sh = shape(ir)
source = (ir.get("meta") or {}).get("source", "")
kinds = {n["kind"] for n in ir["nodes"]}
if source == "db" or "table" in kinds:
return {
**sh,
"kind": "erd",
"emitter": "erd",
"options": {},
"why": "entities with references — cards in columns, not ranks",
}
# Wider than about twenty at any level and it stops being readable, whatever
# draws it. Measured: <=20 nodes renders ~1.6:1, 70-106 about 7:1, 261 14:1.
if sh["widest_level"] > 20:
return {
**sh,
"kind": "sheet",
"emitter": "index",
"options": {},
"why": (
f"{sh['widest_level']} nodes sit at one level; any layered engine "
"draws that as a strip. Split it, scope it, or read it as an index"
),
}
linked = sh["nodes"] - sh["isolated"]
if sh["nodes"] and sh["isolated"] > sh["nodes"] * 0.6:
return {
**sh,
"kind": "flat",
"emitter": "index",
"options": {},
"why": (
f"{sh['isolated']} of {sh['nodes']} have no relationships — "
"that is a list, and a diagram of it says less than the list does"
),
}
# A pipeline is levels-dominant and narrow: a chain with some fan-out, which
# is the shape of an Airflow DAG, a build, an ETL run. Ranks genuinely suit
# it — but read left to right, the way every scheduler's own UI draws it,
# because a pipeline is a sequence and sequences read across.
if source in ("airflow", "dag", "pipeline") or "task" in kinds:
return {
**sh,
"kind": "pipeline",
"emitter": "dot",
"options": {"rankdir": "LR"},
"why": "a sequence with fan-out — ranks suit it, read left to right",
}
if sh["levels"] >= 4 and sh["widest_level"] <= 8 and linked > 3:
return {
**sh,
"kind": "pipeline",
"emitter": "dot",
"options": {"rankdir": "LR"},
"why": (
f"{sh['levels']} levels only {sh['widest_level']} wide — a chain, "
"and a chain reads across rather than down"
),
}
if sh["levels"] <= 2 and linked > 8:
return {
**sh,
"kind": "hub",
"emitter": "dot",
"options": {},
"why": "shallow and wide — one or two things everything points at",
}
return {
**sh,
"kind": "layered" if sh["levels"] > 2 else "tree",
"emitter": "dot",
"options": {},
"why": f"{sh['levels']} levels, widest {sh['widest_level']} — ranks suit this",
}

View File

@@ -0,0 +1,107 @@
"""
Where the rest of spr is, when docgen is not inside it.
docgen is meant to be usable **standalone** — copied out of the repo, pointed at
someone else's codebase, with nothing above it. One capability genuinely needs
more than that: reading an OpenAPI document goes through
`station/tools/modelgen`, which parses the spec and resolves `$ref`, and is
deliberately not reimplemented here.
So there is exactly one seam to the wider repo, and this is it. Everything else
in docgen resolves inside its own directory.
## How it resolves, in order
1. $DOCGEN_REFERENCE an explicit path to the repo root
2. walking up from this file when docgen still sits inside the repo
(1) is the case that matters when docgen is used standalone with the repo kept
alongside as reference. (2) is what happens in place, and needs no configuration.
## Why an env var and not a config file
Because there is one setting and it is a path. A config file for one path is a
file to find, parse, document and validate, and the first question anyone asks
of it is "where does it live" — which is the same question again. The Makefile
passes it through, `make doctor` reports it, and the error message when it is
missing says the variable name.
## Absent is a normal state, not an error
Nothing here raises on a failed lookup. The OpenAPI extractor reports what to
set and every other part of docgen carries on, because for four of the five
extractors this file is irrelevant.
"""
import os
import sys
from pathlib import Path
ENV_VAR = "DOCGEN_REFERENCE"
# What proves a directory is the repo root rather than some other directory:
# the station tools tree, which is the only thing docgen ever reaches for.
MARKER = Path("station") / "tools"
def _candidates():
explicit = os.environ.get(ENV_VAR)
if explicit:
yield Path(explicit).expanduser()
# Walking up covers the in-place case and costs nothing when it fails.
for parent in Path(__file__).resolve().parents:
yield parent
def root() -> Path | None:
"""The repo root holding `station/tools`, or None if it is not reachable."""
for candidate in _candidates():
try:
if (candidate / MARKER).is_dir():
return candidate
except OSError:
continue
return None
def station_tools() -> Path | None:
"""The `station/tools` directory itself, or None."""
found = root()
return found / MARKER if found else None
def describe() -> str:
"""One line for `make doctor`. Says how it resolved, not just whether."""
explicit = os.environ.get(ENV_VAR)
found = root()
if found is None:
return (f"absent — set {ENV_VAR}=/path/to/repo to read OpenAPI "
"(everything else works without it)")
how = f"from ${ENV_VAR}" if explicit and Path(explicit).expanduser() == found \
else "found by walking up"
return f"{found} ({how})"
def on_path() -> Path | None:
"""Put `station/tools` on `sys.path` and return it, or None.
Prepending rather than appending: a `modelgen` earlier on the path is
somebody else's, and silently reading the wrong one is worse than not
finding it at all.
"""
tools = station_tools()
if tools is None:
return None
if str(tools) not in sys.path:
sys.path.insert(0, str(tools))
return tools
def missing(what: str, why: str) -> ImportError:
"""The error to raise when a reference-dependent capability is used."""
return ImportError(
f"{what} is not reachable — {why}\n"
f"docgen is standalone except for this one seam. Point at the repo with:\n"
f" export {ENV_VAR}=/path/to/repo\n"
f"and check it with: make doctor"
)

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,193 @@
"""
Style: what a `kind` looks like. Loaded by emitters only.
A style file names **slots**, never colours. `"border": "atlas"` is a rule; the
theme binds `atlas` to `#43A047` in print and `#15803d` on the docs site. That
indirection is the whole reason this is one colour language rather than three:
`tokens.css`, `docs/graphs/themes/*.gvpr` and this file all use the same slot
names, so a diagram and the page around it match by construction.
`docs/graphs/README.md` states the rule these files obey:
Match the palette to soleprint/common/theme/themes/<name>.css so a diagram
and the page around it are the same visual language.
style = Style.load("lucid")
style.node("class")["border"] -> "#1E88E5" (resolved for the theme)
style.node("nonesuch") -> the `default` entry, never a crash
An unknown `kind` falls back to `default`. That matters more than it looks: it
means a new extractor with a new vocabulary renders plainly and legibly on day
one instead of failing, and nobody is forced to write a style file before they
can see anything.
"""
import json
from pathlib import Path
# Harvesting a theme from real diagrams. Imported lazily by callers that want
# it; `extract` needs lxml and the rest of the package does not.
HERE = Path(__file__).resolve().parent
# Keys whose value is a slot name to be resolved against the theme. Anything
# not listed here is passed through as-is, which is how numbers, shapes and
# booleans survive.
COLOUR_KEYS = ("fill", "border", "text", "color", "header-fill", "bgcolor", "fontcolor")
# Keys whose value names a geometry entry rather than a colour.
GEOMETRY_KEYS = ("font-size", "font")
# Prose, not rules. Stripped before a rule reaches an emitter so it cannot be
# mistaken for an attribute.
NOTE_KEYS = ("note",)
class StyleError(ValueError):
"""A style file that an emitter cannot apply."""
class Style:
"""One style file, resolved against one theme."""
def __init__(self, data: dict, theme: str | None = None, name: str = "<inline>"):
self.name = name
self.data = data
self.theme = theme or data.get("default_theme")
themes = data.get("themes", {})
if self.theme not in themes:
raise StyleError(
f"{name}: no theme {self.theme!r} — have {', '.join(sorted(themes)) or 'none'}"
)
self.slots: dict[str, str] = dict(themes[self.theme].get("slots", {}))
self.geometry: dict = {
k: v for k, v in data.get("geometry", {}).items() if k not in NOTE_KEYS
and not k.endswith("-note")
}
problems = self.validate()
if problems:
raise StyleError(f"{name} [{self.theme}]:\n " + "\n ".join(problems))
# -- loading ----------------------------------------------------------
@classmethod
def load(cls, ref: str | Path, theme: str | None = None) -> "Style":
"""A shipped style by name, or any JSON file by path."""
path = Path(ref)
if not path.suffix and not path.exists():
path = HERE / f"{ref}.json"
if not path.exists():
available = sorted(p.stem for p in HERE.glob("*.json"))
raise StyleError(
f"no style {str(ref)!r} — shipped: {', '.join(available) or 'none'}"
"\n(a path to any JSON file works too)"
)
return cls(json.loads(path.read_text()), theme=theme, name=path.stem)
@classmethod
def available(cls) -> list[str]:
return sorted(p.stem for p in HERE.glob("*.json"))
def themes(self) -> list[str]:
return sorted(self.data.get("themes", {}))
# -- checking ---------------------------------------------------------
def validate(self) -> list[str]:
"""Every slot a rule mentions must exist in this theme.
Run at load, so a half-bound theme fails at the boundary rather than
rendering most of a diagram in the right colours and the rest in
whatever DOT does with an empty string.
"""
problems = []
for section in ("nodes", "groups", "edges"):
for kind, rule in self.data.get(section, {}).items():
if not isinstance(rule, dict):
problems.append(f"{section}.{kind} is not an object")
continue
for key in COLOUR_KEYS:
slot = rule.get(key)
if slot is not None and slot not in self.slots:
problems.append(
f"{section}.{kind}.{key} names slot {slot!r}, "
f"which theme {self.theme!r} does not define"
)
for key in GEOMETRY_KEYS:
ref = rule.get(key)
if ref is not None and ref not in self.geometry:
problems.append(
f"{section}.{kind}.{key} names geometry {ref!r}, which is not defined"
)
graph = self.data.get("graph", {})
for key in COLOUR_KEYS:
slot = graph.get(key)
if slot is not None and slot not in self.slots:
problems.append(f"graph.{key} names slot {slot!r}, undefined in {self.theme!r}")
return problems
# -- reading ----------------------------------------------------------
def _resolve(self, rule: dict) -> dict:
out = {}
for key, value in rule.items():
if key in NOTE_KEYS:
continue
if key in COLOUR_KEYS and isinstance(value, str):
out[key] = self.slots.get(value, value)
elif key in GEOMETRY_KEYS and isinstance(value, str):
out[key] = self.geometry.get(value, value)
else:
out[key] = value
return out
def _lookup(self, section: str, kind: str) -> dict:
rules = self.data.get(section, {})
return self._resolve(rules.get(kind) or rules.get("default") or {})
def node(self, kind: str) -> dict:
return self._lookup("nodes", kind)
def group(self, kind: str) -> dict:
return self._lookup("groups", kind)
def edge(self, kind: str) -> dict:
return self._lookup("edges", kind)
def graph(self) -> dict:
return self._resolve(self.data.get("graph", {}))
def geom(self, key, default=None):
return self.geometry.get(key, default)
def slot(self, name: str, default: str = "") -> str:
return self.slots.get(name, default)
def domain_slot(self, domain: str | None, index: int = 0) -> str:
"""Which slot a group with this domain uses.
The IR says *which spr model* a group is; this says which slot that maps
to. Where there is no domain, the rotation is indexed by the caller's
sorted position, so the assignment is deterministic — two runs of the
same graph colour the same group the same way.
"""
table = self.data.get("domain_slots", {})
if domain and domain in table:
return table[domain]
rotation = table.get("rotation") or ["accent"]
return rotation[index % len(rotation)]
def limits(self) -> dict:
"""Where this style asks for more than the target can express."""
return {k: v for k, v in self.data.get("limits", {}).items() if k not in NOTE_KEYS}
def harvest(folder, out_dir, name: str = "harvested"):
"""A folder of exported diagrams -> tokens.json and a theme to paste in.
Offline, and it never reads text content: style values are visual metadata,
so the semantics of a confidential diagram are not needed and not touched.
"""
from .tokens import from_folder
return from_folder(folder, out_dir, name)

View File

@@ -0,0 +1,221 @@
"""
A folder of exported diagrams -> the style vocabulary they use.
The premise, from `spr/def/prompts/lucid` §2a: style values are *visual*
metadata — hex codes, stroke widths, corner radii, font stacks. None of that
requires reading what the diagram says, and none of it requires sending the file
anywhere. So for confidential diagrams this is the path: run it locally, skip
both the Lucid API and any assistant.
Two rules, and they are not stylistic preferences:
1. **Text content is never read.** `<text>` elements are visited for their style
attributes and nothing else; `.text` and `.tail` are never touched anywhere in
this module. The semantics of a diagram are not needed to derive a palette,
so they are not looked at. `selftest.py` asserts a label from a known fixture
appears nowhere in the output.
2. **Fully offline.** No network import in this file, and nothing here opens a
socket. Confidential source diagrams stay off any network path, and off the
Lucid API path.
## Why lxml and not grep
`prompts/lucid` §2a gives a grep recipe and then says to do this instead, which
is the right call: `fill` appears as a presentation attribute (`fill="#fff"`),
inside an inline style (`style="fill:#fff"`), and inside a `<style>` block that
applies to elements that carry none of it. Grep sees three unrelated strings and
counts a CSS rule once no matter how many shapes it paints. Parsing sees one
vocabulary and counts what is actually drawn.
`lxml` is imported inside the function, so the rest of docgen works without it.
## Frequency is the whole point
The output is sorted by count, because that is what turns a heap of values into a
palette: the top two or three fills *are* the palette, and the modal stroke width
*is* the house line weight. A list of every colour in the file, unsorted, is not
usable — real exports carry dozens of one-off values from shadows, gradients and
whatever someone recoloured once.
"""
import json
import re
import shutil
import subprocess
import tempfile
from collections import Counter
from pathlib import Path
# The vocabulary worth harvesting, from prompt 35.5.
PROPERTIES = ("fill", "stroke", "stroke-width", "font-family", "font-size", "rx")
# Lucid's idiom for "no fill". Mapping it to black is the obvious wrong answer
# and would poison the palette with a colour the diagram does not contain.
TRANSPARENT = "#00000000"
# Values that are the absence of a value. Counted separately rather than dropped,
# because "most shapes have no stroke" is itself a fact about the house style.
NULLISH = {"none", "transparent", "currentColor", "inherit"}
def _values_from(el) -> dict:
"""One element's style vocabulary. Attributes and inline style, never text.
The inline `style="..."` wins over the presentation attribute, which is what
the SVG spec says and what browsers do.
"""
found = {p: el.get(p) for p in PROPERTIES if el.get(p)}
inline = el.get("style")
if inline:
for decl in inline.split(";"):
if ":" not in decl:
continue
name, _, value = decl.partition(":")
name, value = name.strip(), value.strip()
if name in PROPERTIES and value:
found[name] = value
return found
def _normalise(prop: str, value: str) -> str | None:
"""One spelling per value, so `#FFF` and `#ffffff` are not two palette entries."""
value = value.strip()
if not value or value in NULLISH:
return value if value in NULLISH else None
if prop in ("fill", "stroke"):
if value.startswith("url("):
return None # a gradient or pattern reference, not a colour
if value == TRANSPARENT:
return "transparent"
if value.startswith("#"):
hexv = value[1:].lower()
if len(hexv) in (3, 4): # #abc -> #aabbcc
hexv = "".join(c * 2 for c in hexv)
if len(hexv) == 8 and hexv[6:] == "ff":
hexv = hexv[:6] # fully opaque; the alpha says nothing
return "#" + hexv
rgb = re.match(r"rgba?\(([^)]+)\)", value)
if rgb:
parts = [p.strip() for p in rgb.group(1).replace("/", ",").split(",")]
try:
r, g, b = (int(float(p)) for p in parts[:3])
except ValueError:
return value.lower()
return f"#{r:02x}{g:02x}{b:02x}"
return value.lower()
if prop in ("stroke-width", "font-size", "rx"):
# `8pt`, `8px`, `8` — the number is the value; the unit is noise for
# font-size (DOT's fontsize is already points) and for stroke width.
num = re.match(r"(-?[\d.]+)", value)
if not num:
return None
# `11.00` and `11` are the same size and must not be two entries in the
# frequency count. DOT takes either; the tidy one is what lands in a
# profile a person will read.
text = num.group(1)
return text.rstrip("0").rstrip(".") if "." in text else text
if prop == "font-family":
# A font stack's first entry is the one that renders where it exists.
return value.split(",")[0].strip().strip("'\"")
return value
def _svg_files(folder: Path, workdir: Path) -> list[Path]:
"""Every SVG to read, converting PDFs on the way.
`pdftocairo -svg` is the conversion `prompts/lucid` §2a names. Converted
files land in a temp directory — the target folder is read-only here, the
same way `histgen`'s source is.
"""
files = sorted(folder.rglob("*.svg"))
pdfs = sorted(folder.rglob("*.pdf"))
if pdfs:
if shutil.which("pdftocairo") is None:
print(f" {len(pdfs)} PDF(s) skipped — pdftocairo not found "
"(install with: sudo apt install poppler-utils)")
else:
for i, pdf in enumerate(pdfs):
out = workdir / f"pdf-{i:03d}-{pdf.stem}.svg"
proc = subprocess.run(
["pdftocairo", "-svg", str(pdf), str(out)], capture_output=True
)
if proc.returncode == 0 and out.exists():
files.append(out)
else:
print(f" could not convert {pdf.name}: "
f"{proc.stderr.decode('utf-8', 'replace').strip()}")
return files
def harvest(folder: Path | str) -> dict:
"""Read a target folder, return the frequency-sorted token vocabulary."""
try:
from lxml import etree
except ImportError: # pragma: no cover - depends on the host
raise RuntimeError(
"extraction needs lxml — pip install lxml\n"
"(the rest of docgen does not; this is the only place it is used)"
) from None
folder = Path(folder)
if not folder.is_dir():
raise NotADirectoryError(f"not a folder: {folder}")
counters = {p: Counter() for p in PROPERTIES}
read, failed = 0, []
with tempfile.TemporaryDirectory(prefix="docgen-extract-") as tmp:
files = _svg_files(folder, Path(tmp))
for path in files:
try:
tree = etree.parse(str(path))
except Exception as e:
failed.append((path.name, str(e).splitlines()[0]))
continue
read += 1
for el in tree.iter():
# Style attributes only. `el.text` is never referenced — that is
# the "never read text content" rule, and it is one line.
for prop, raw in _values_from(el).items():
value = _normalise(prop, raw)
if value:
counters[prop][value] += 1
return {
"source": str(folder),
"files_read": read,
"files_failed": [{"file": n, "error": e} for n, e in failed],
"tokens": {
prop: [{"value": v, "count": c} for v, c in counters[prop].most_common()]
for prop in PROPERTIES
},
}
def write(folder: Path | str, out: Path | str) -> Path:
"""Harvest and write `tokens.json`. Returns the path."""
data = harvest(folder)
out = Path(out)
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps(data, indent=2) + "\n")
return out
def summarise(data: dict, top: int = 5) -> str:
"""What was found, frequency-sorted — the part a person reads."""
lines = [f"{data['files_read']} file(s) read from {data['source']}"]
for fail in data["files_failed"]:
lines.append(f" could not parse {fail['file']}: {fail['error']}")
for prop in PROPERTIES:
entries = data["tokens"][prop][:top]
if not entries:
continue
lines.append(f"\n {prop}")
for e in entries:
lines.append(f" {e['count']:6} {e['value']}")
return "\n".join(lines)

View File

@@ -0,0 +1,383 @@
{
"name": "lucid",
"note": "Visual language for generated diagrams. Keyed on `kind`, which is the only IR field style may read. Rules name SLOTS, never hexes; a theme binds slot -> colour. That indirection is the point: it is the same vocabulary `soleprint/common/theme/tokens.css` and `docs/graphs/themes/*.gvpr` already use, so a diagram and the page around it are one visual language. docs/graphs/README.md states the rule.",
"default_theme": "dark",
"themes": {
"lucid": {
"note": "Print and light, for documents and anything that gets exported. Extracted from real architecture diagrams, then re-bound onto spr's slots. Ask for it with `--theme lucid`.",
"slots": {
"surface-0": "#FFFFFF",
"surface-1": "#FAFAFA",
"surface-2": "#F5F7FA",
"border": "#BDBDBD",
"border-strong": "#9E9E9E",
"text": "#333333",
"text-muted": "#616E7C",
"text-dim": "#9AA5B1",
"text-on-fill": "#000000",
"accent": "#FFB74D",
"accent-soft": "#FFF3E0",
"artery": "#E53935",
"artery-soft": "#FFEBEE",
"atlas": "#43A047",
"atlas-soft": "#E8F5E9",
"station": "#1E88E5",
"station-soft": "#E3F2FD",
"ok": "#43A047",
"error": "#E53935",
"muted": "#E0E0E0",
"muted-soft": "#FAFAFA",
"alt-1": "#B39DDB",
"alt-1-soft": "#EDE7F6",
"alt-2": "#FFF176",
"alt-2-soft": "#FFFDE7"
}
},
"dark": {
"note": "The default. Every value is a tokens.css variable or a --system-accent, so a diagram embedded in a page matches it without anyone choosing. artery/atlas/station are exactly what artery/index.html:30, atlas/index.html:25 and station/index.html:29 set. `--theme lucid` switches to the print palette.",
"slots": {
"surface-0": "#0a0a0a",
"surface-1": "#141414",
"surface-2": "#1a1a1a",
"border": "#333333",
"border-strong": "#4a4a4a",
"text": "#e5e5e5",
"text-muted": "#a3a3a3",
"text-dim": "#666666",
"text-on-fill": "#0a0a0a",
"accent": "#d4a574",
"accent-soft": "#242424",
"artery": "#b91c1c",
"artery-soft": "#fca5a5",
"atlas": "#15803d",
"atlas-soft": "#86efac",
"station": "#1d4ed8",
"station-soft": "#93c5fd",
"ok": "#3ecf8e",
"error": "#f06565",
"muted": "#555568",
"muted-soft": "#141414",
"alt-1": "#4f9cf9",
"alt-1-soft": "#141414",
"alt-2": "#f5a623",
"alt-2-soft": "#141414"
}
}
},
"geometry": {
"note": "Pinned to tokens.css rather than retyped. The supplied spec and spr's tokens agreed on six of these independently: radius 4-6px, label 10-11px, header 12-14px, padding 8px, hairline 1px, and a Segoe-UI-through-Arial sans stack.",
"radius-sm": 4,
"radius": 6,
"hairline": 1,
"font": "Helvetica",
"font-note": "PostScript-style names, because they are the only spelling Graphviz maps to a real SVG font-weight. `fontname=\"Arial Bold\"` passes through verbatim as font-family=\"Arial Bold\" \u2014 a family no browser has, so it renders neither Arial nor bold. `Helvetica-Bold` emits font-family=\"Helvetica,sans-Serif\" font-weight=\"bold\", which is a real stack with a real weight. It also measures wider (128pt vs 116pt for the same label), so boxes are sized for the text that actually renders instead of overflowing it.",
"font-size-sm": 10,
"font-size-base": 11,
"font-size-header": 13,
"padding": "0.20,0.10",
"font-bold": "Helvetica-Bold"
},
"graph": {
"bgcolor": "surface-0",
"fontcolor": "text",
"rankdir": "TB",
"nodesep": 0.5,
"ranksep": 0.6,
"pad": 0.3
},
"nodes": {
"default": {
"shape": "box",
"rounded": true,
"fill": "surface-0",
"border": "border",
"text": "text",
"note": "Every unknown kind lands here rather than crashing. A new extractor renders plainly and legibly on day one.",
"bold": true
},
"component": {
"shape": "box",
"rounded": true,
"fill": "surface-0",
"border": "border",
"text": "text",
"bold": true
},
"datastore": {
"shape": "cylinder",
"rounded": false,
"fill": "surface-0",
"border": "border",
"text": "text",
"bold": true
},
"bucket": {
"shape": "trapezium",
"rounded": false,
"fill": "surface-0",
"border": "border",
"text": "text",
"note": "An approximation. DOT has no S3-bucket silhouette; trapezium is the nearest native shape.",
"bold": true
},
"module": {
"shape": "box",
"rounded": true,
"fill": "surface-2",
"border": "border-strong",
"text": "text",
"bold": true
},
"class": {
"shape": "box",
"rounded": true,
"fill": "surface-0",
"border": "station",
"text": "text",
"bold": true
},
"function": {
"shape": "box",
"rounded": true,
"fill": "surface-0",
"border": "border",
"text": "text-muted",
"font-size": "font-size-sm",
"bold": true
},
"external": {
"shape": "box",
"rounded": true,
"fill": "muted-soft",
"border": "muted",
"text": "text-muted",
"dashed": true,
"note": "A name that could not be resolved. Drawn as the boundary it is, and never dropped \u2014 a missing dependency that looks like a complete diagram is the worse failure.",
"bold": true
},
"task": {
"shape": "box",
"rounded": true,
"fill": "surface-2",
"border": "station",
"text": "text",
"bold": true,
"note": "A unit of work in a pipeline \u2014 an Airflow task, a build step. Distinct from `module` so a schedule does not render as if it were code structure."
},
"sensor": {
"shape": "hexagon",
"rounded": false,
"fill": "surface-2",
"border": "accent",
"text": "text",
"bold": true,
"note": "A task that waits on something outside the pipeline."
},
"endpoint": {
"shape": "box",
"rounded": true,
"fill": "surface-2",
"border": "station",
"text": "text",
"bold": true,
"note": "An HTTP route. From a spec, or from what was actually called."
},
"operation": {
"shape": "box",
"rounded": true,
"fill": "surface-2",
"border": "accent",
"text": "text",
"bold": true,
"note": "A GraphQL operation \u2014 usually one endpoint carrying many, which is why it is its own kind rather than a path."
},
"interface": {
"shape": "box",
"rounded": true,
"fill": "surface-0",
"border": "accent",
"text": "text",
"dashed": true,
"bold": true,
"note": "A contract rather than an implementation. Dashed, because that is the convention everywhere else too."
}
},
"groups": {
"default": {
"rounded": false,
"dashed": true,
"fill": "surface-2",
"border": "border",
"text": "text-muted",
"bold": true
},
"boundary": {
"rounded": false,
"dashed": false,
"fill": "surface-0",
"border": "border-strong",
"header-fill": "accent",
"text": "text-on-fill",
"bold": true,
"note": "Type A. The header is a filled bar across the top of the container in the source spec; DOT gives a label plus a cluster bgcolor and no separate bar. See limits.header-bar."
},
"zone": {
"rounded": false,
"dashed": true,
"fill": "surface-2",
"border": "border",
"text": "text-muted",
"note": "Type B. Dashed area boundary.",
"bold": true
},
"artery": {
"rounded": false,
"dashed": true,
"fill": "artery-soft",
"border": "artery",
"text": "text-muted",
"bold": true
},
"atlas": {
"rounded": false,
"dashed": true,
"fill": "atlas-soft",
"border": "atlas",
"text": "text-muted",
"bold": true
},
"station": {
"rounded": false,
"dashed": true,
"fill": "station-soft",
"border": "station",
"text": "text-muted",
"bold": true
},
"pipeline": {
"rounded": false,
"dashed": true,
"fill": "surface-2",
"border": "border",
"text": "text-muted",
"bold": true,
"note": "One DAG, when several are drawn together."
}
},
"edges": {
"default": {
"color": "text",
"arrowhead": "normal",
"arrowsize": 0.7,
"text": "text-muted"
},
"imports": {
"color": "border-strong",
"arrowhead": "normal",
"arrowsize": 0.6,
"text": "text-dim"
},
"inherits": {
"color": "station",
"arrowhead": "empty",
"arrowsize": 0.8,
"text": "text-muted"
},
"calls": {
"color": "text",
"arrowhead": "normal",
"arrowsize": 0.7,
"text": "text-muted"
},
"async": {
"color": "text",
"arrowhead": "normal",
"arrowsize": 0.7,
"dashed": true,
"text": "text-muted"
},
"flow": {
"color": "artery",
"arrowhead": "normal",
"arrowsize": 0.7,
"dashed": true,
"text": "artery"
},
"ok": {
"color": "ok",
"arrowhead": "normal",
"arrowsize": 0.7,
"dashed": true,
"text": "ok"
},
"foreign_key": {
"color": "station",
"arrowhead": "normal",
"arrowsize": 0.7,
"text": "text-muted"
},
"references": {
"color": "border-strong",
"arrowhead": "normal",
"arrowsize": 0.6,
"dashed": true,
"text": "text-dim"
},
"depends": {
"color": "border-strong",
"arrowhead": "normal",
"arrowsize": 0.7,
"text": "text-muted"
},
"triggers": {
"color": "accent",
"arrowhead": "normal",
"arrowsize": 0.8,
"text": "accent"
},
"follows": {
"color": "accent",
"arrowhead": "normal",
"arrowsize": 0.7,
"text": "accent",
"note": "Observed order. Consecutive, not caused-by; the weight is the signal."
},
"accepts": {
"color": "border-strong",
"arrowhead": "normal",
"arrowsize": 0.6,
"text": "text-dim"
},
"returns": {
"color": "station",
"arrowhead": "normal",
"arrowsize": 0.6,
"text": "text-dim"
}
},
"domain_slots": {
"note": "How a group picks its colour without the IR ever naming one. The IR says which spr model a group belongs to (attrs.domain \u2014 semantic); this maps that to a slot. Same mechanism as --system-accent. Where a group has no domain, the emitter assigns the rotation below by sorted id, so the choice is deterministic and two runs give the same bytes.",
"artery": "artery",
"atlas": "atlas",
"station": "station",
"soleprint": "accent",
"rotation": [
"accent",
"alt-1",
"alt-2",
"station",
"atlas",
"artery"
]
},
"limits": {
"note": "Where DOT stops. Recorded rather than worked around; the full spec is kept here so a richer emitter needs no re-authoring. See the plan's 'Use DOT until it hits its limits'.",
"header-bar": "Type A's filled 100%-width header rectangle. DOT clusters have a label and a bgcolor, not a header bar. Needs an HTML-like label table.",
"dasharray": "stroke-dasharray is not parameterised \u2014 style=dashed is one pattern, so 4,4 and 5,5 collapse. Async and thematic edges must differ by colour, not by dash.",
"corner-radius": "style=rounded is binary, so radius-sm (4) and radius (6) render identically.",
"icon-above-label": "Icon centered above a centered label needs an HTML-like label or image+labelloc; text only here.",
"sequence-badge": "A numbered circle on an edge. xlabel carries the number; the circle, its fill and its border are not expressible.",
"bucket-shape": "trapezium approximates the S3 bucket silhouette.",
"ranged-widths": "The spec gives 1-2px and 1-1.5px; penwidth is scalar, so these pin to geometry.hairline."
}
}

View File

@@ -0,0 +1,210 @@
"""
A harvested token vocabulary -> a **theme**.
`extract.py` says which values a folder of real diagrams uses and how often.
This decides which of them are the house palette. The rule is the plain one:
the top two or three fills *are* the palette
the modal stroke width *is* the house line weight
Nothing cleverer. Frequency is a good enough signal because a real diagram set
repeats its own vocabulary constantly and its one-offs stay one-offs.
## It produces a theme, not a style file
A style file says *what a `kind` looks like* — "a class is filled with
`surface-0` and outlined in `station`". That is a design decision and no amount
of counting hexes recovers it. What harvesting recovers is the **binding**:
which colour `station` should be. So the output is a `themes` entry — a slot to
hex map — that drops into an existing style file beside `dark` and `lucid`.
That split is the point of the whole colour language. One set of rules, several
bindings, and a new binding costs a paste rather than a rewrite.
## The judgement calls, named
Three, all of the kind that should be visible rather than buried:
- **Canvas and ink are the two lightness extremes, not the two most common
values.** Frequency is the wrong signal: in a Graphviz SVG every label carries
a `fill`, so the ink outnumbers the canvas 87 to 43, and "most common is the
background" produces a theme whose text is invisible against it.
- **Polarity is assumed light**, because a folder of exports is usually for
print. `dark=True` flips it. Get it backwards and the contrast check says so
rather than letting an unreadable theme through quietly.
- **Slot assignments are a guess.** Which accent means `artery` is a decision,
not a measurement. The note says so, and they are meant to be checked.
- **Widths and sizes take the mode, not the mean.** A mean of 1 and 4 is 2.5,
which is a width the diagrams never use.
"""
import json
from pathlib import Path
# The slots a theme has to bind for a style file to load. Anything a harvest
# cannot find gets a neutral, so a partial harvest is still usable.
REQUIRED_SLOTS = (
"surface-0", "surface-1", "surface-2",
"border", "border-strong",
"text", "text-muted", "text-dim", "text-on-fill",
"accent", "accent-soft",
"artery", "artery-soft", "atlas", "atlas-soft", "station", "station-soft",
"ok", "error", "muted", "muted-soft",
"alt-1", "alt-1-soft", "alt-2", "alt-2-soft",
)
def _top(tokens: dict, prop: str, n: int = 10) -> list[str]:
return [e["value"] for e in tokens.get(prop, [])[:n] if e["value"].startswith("#")]
def _mode(tokens: dict, prop: str, fallback: str) -> str:
entries = [e for e in tokens.get(prop, []) if e["value"] not in ("none", "transparent")]
return entries[0]["value"] if entries else fallback
def _luminance(hexv: str) -> float:
"""Rough perceptual lightness, 0..1. Enough to sort pale from dark."""
if not hexv.startswith("#") or len(hexv) != 7:
return 0.5
r, g, b = (int(hexv[i: i + 2], 16) / 255 for i in (1, 3, 5))
return 0.2126 * r + 0.7152 * g + 0.0722 * b
def derive(data: dict, name: str = "harvested", *, dark: bool = False) -> dict:
"""Token JSON -> a `themes` entry: {name: {note, slots}}.
Paste the result into a style file's `themes` and select it with
`--theme <name>`. Every slot is bound, so the file loads; the ones a harvest
could not distinguish fall back to a neutral rather than being absent, which
would fail validation at load.
"""
tokens = data.get("tokens", {})
fills = _top(tokens, "fill")
strokes = _top(tokens, "stroke")
# The canvas and the ink are the two **lightness extremes**, not the two
# most common values. Frequency is the wrong signal here: in a Graphviz SVG
# every label carries a `fill`, so the ink outnumbers the canvas 87 to 43
# and "most common is the background" puts the text colour on the canvas —
# which renders a theme whose text is invisible against its own background.
if fills:
lightest = max(fills, key=_luminance)
darkest = min(fills, key=_luminance)
else:
lightest, darkest = "#ffffff", "#333333"
# Polarity is a guess, and it is the one the caller most needs to check:
# a light export and a dark one use the same two extremes the other way
# round. Light is assumed, because a folder of exports is usually for print.
background, ink = (lightest, darkest) if not dark else (darkest, lightest)
mid = [f for f in fills if f not in (lightest, darkest)]
surface = (min(mid, key=lambda f: abs(_luminance(f) - _luminance(background)))
if mid else background)
border = strokes[0] if strokes else "#9aa5b1"
strong = strokes[1] if len(strokes) > 1 else border
# Accents are the frequent colours that are neither the canvas nor the ink —
# the ones a diagram uses to mean something.
neutral = {background, surface, ink, border, strong}
accents = [f for f in fills + strokes if f not in neutral]
seen, ordered = set(), []
for a in accents:
if a not in seen:
seen.add(a)
ordered.append(a)
def accent(i: int, fallback: str) -> str:
return ordered[i] if i < len(ordered) else fallback
slots = {
"surface-0": background,
"surface-1": surface,
"surface-2": surface,
"border": border,
"border-strong": strong,
"text": ink,
"text-muted": strong,
"text-dim": border,
"text-on-fill": background,
"accent": accent(0, strong),
"accent-soft": surface,
"artery": accent(1, strong),
"artery-soft": surface,
"atlas": accent(2, strong),
"atlas-soft": surface,
"station": accent(3, strong),
"station-soft": surface,
"ok": accent(2, strong),
"error": accent(1, strong),
"muted": border,
"muted-soft": surface,
"alt-1": accent(4, strong),
"alt-1-soft": surface,
"alt-2": accent(5, strong),
"alt-2-soft": surface,
}
for slot in REQUIRED_SLOTS:
slots.setdefault(slot, border)
contrast = abs(_luminance(slots["text"]) - _luminance(slots["surface-0"]))
if contrast < 0.25:
# Not fatal — the harvest is still the best available guess — but a
# theme nobody can read is worth saying out loud rather than shipping.
slots["_warning"] = (
f"text and surface-0 differ by {contrast:.2f} in lightness; this theme "
"is close to unreadable. The polarity is probably backwards."
)
return {
name: {
"note": (
f"Harvested from {data.get('files_read', 0)} diagram(s) in "
f"{data.get('source', 'a target folder')}. Frequency-sorted: the top "
"fills are the palette, the modal stroke width the house line weight. "
"Slot *assignments* are a guess — which accent means `artery` is a "
"decision, not a measurement, so check them. Polarity was assumed "
f"{'dark' if dark else 'light'} — pass dark=True if that is backwards. "
"No text content was read. "
f"Also seen, unassigned: {', '.join(ordered[6:12]) or 'none'}."
),
"slots": slots,
}
}
def geometry(data: dict) -> dict:
"""The non-colour half: line weight, type size, corner rounding."""
tokens = data.get("tokens", {})
size = _mode(tokens, "font-size", "11")
return {
"hairline": _mode(tokens, "stroke-width", "1"),
"font-size-base": size,
"font-size-sm": str(max(int(float(size)) - 1, 6)),
"font-size-header": str(int(float(size)) + 2),
"font": _mode(tokens, "font-family", "Helvetica"),
# Lucid's `rounding` is a scalar and DOT's `rounded` is binary, so the
# radius is dropped. Its presence is the only part that survives.
"rounded": bool(tokens.get("rx")),
}
def write(data: dict, out, name: str = "harvested", *, dark: bool = False) -> Path:
"""Write the theme entry, ready to paste into a style file's `themes`."""
out = Path(out)
out.parent.mkdir(parents=True, exist_ok=True)
out.write_text(json.dumps({"themes": derive(data, name, dark=dark),
"geometry": geometry(data)}, indent=2) + "\n")
return out
def from_folder(folder, out_dir, name: str = "harvested", *, dark: bool = False) -> dict:
"""The whole path: a folder of exports -> tokens.json + a theme."""
from . import extract
out_dir = Path(out_dir)
tokens_path = extract.write(folder, out_dir / "tokens.json")
data = json.loads(tokens_path.read_text())
theme_path = write(data, out_dir / f"{name}.theme.json", name, dark=dark)
return {"tokens": tokens_path, "theme": theme_path, "data": data}

View File

@@ -0,0 +1,59 @@
// Drive the generated viewer's logic under a stub DOM and assert the toggle.
const fs = require('fs');
const html = fs.readFileSync(process.argv[2] || '/tmp/site/viewer.html', 'utf8');
const script = html.split('<script>')[1].split('</script>')[0];
const handlers = {};
const listen = (t, f) => { (handlers[t] = handlers[t] || []).push(f); };
function node(extra) {
return Object.assign({
style: {}, classList: { add() {}, remove() {} },
addEventListener: listen,
getBoundingClientRect: () => ({ left: 0, top: 0, width: 1000, height: 800 }),
}, extra || {});
}
const imgStub = node({ naturalWidth: 2000, naturalHeight: 1000, src: '' });
const pctStub = node({ textContent: '' });
const modeStub = node({ textContent: '' });
const ids = { img: imgStub, container: node(), pct: pctStub, mode: modeStub };
global.document = { getElementById: (i) => ids[i], title: '' };
global.window = { innerWidth: 1000, innerHeight: 800, addEventListener: listen };
global.location = { search: '?src=graph.svg', href: '' };
new Function(script)();
imgStub.onload();
const fire = (t, e) => (handlers[t] || []).forEach((f) => f(e));
let ok = true;
const check = (name, cond) => {
console.log(` ${cond ? 'ok ' : 'FAIL'} ${name}`);
if (!cond) ok = false;
};
const click = (x, y) => {
fire('mousedown', { button: 0, clientX: x, clientY: y, preventDefault() {} });
fire('mouseup', { button: 0, clientX: x, clientY: y });
};
check('fits on load, below 100%', modeStub.textContent === 'fit' && parseInt(pctStub.textContent) < 100);
const fitPct = pctStub.textContent;
click(500, 400);
check('a click toggles to 1:1', modeStub.textContent === '1:1' && pctStub.textContent === '100%');
click(500, 400);
check('clicking again returns to fit', modeStub.textContent === 'fit' && pctStub.textContent === fitPct);
fire('mousedown', { button: 0, clientX: 100, clientY: 100, preventDefault() {} });
fire('mousemove', { clientX: 260, clientY: 180 });
fire('mouseup', { button: 0, clientX: 260, clientY: 180 });
check('a drag pans and does NOT toggle', modeStub.textContent === 'fit');
fire('wheel', { deltaY: -1, clientX: 500, clientY: 400, preventDefault() {} });
check('the wheel zooms in past fit', parseInt(pctStub.textContent) > parseInt(fitPct));
fire('dblclick', {});
check('double-click resets to fit', modeStub.textContent === 'fit');
process.exit(ok ? 0 : 1);

View File

@@ -0,0 +1,107 @@
digraph system_overview {
bgcolor="#0a0e17"
rankdir=TB
splines="spline"
nodesep="0.45"
ranksep="0.6"
pad="0.3"
fontname="Helvetica"
label="Soleprint — System Overview"
labelloc=t
fontsize="14"
fontcolor="#e8eaf0"
compound=true
node [shape="box" style="filled,rounded" fillcolor="#131a2a" color="#1e2a4a" penwidth="1" fontname="Helvetica" fontsize="10" fontcolor="#e8eaf0" margin="0.22,0.12" height="0.45"]
edge [color="#4a5568" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9" fontcolor="#8892a8"]
subgraph cluster_core {
label="Soleprint Hub"
style="rounded,dashed"
color="#0066ff"
bgcolor="#0d1320"
fontcolor="#e8eaf0"
fontname="Helvetica"
hub [label="soleprint
core coordinator
port 12000" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#0066ff" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
}
subgraph cluster_artery {
label="Artery — Todo lo vital"
style="rounded,dashed"
color="#e05c4a"
bgcolor="#0d1320"
fontcolor="#e8eaf0"
fontname="Helvetica"
veins [label="Veins
stateless connectors" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#e05c4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
shunts [label="Shunts
mock connectors" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#e05c4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
pulses [label="Pulses
composed flows" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#e05c4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
}
subgraph cluster_atlas {
label="Atlas — Documentación accionable"
style="rounded,dashed"
color="#2fbf6b"
bgcolor="#0d1320"
fontcolor="#e8eaf0"
fontname="Helvetica"
books [label="Books
documentation" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#2fbf6b" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
templates [label="Templates
patterns" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#2fbf6b" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
}
subgraph cluster_station {
label="Station — Centro de control"
style="rounded,dashed"
color="#5b8cff"
bgcolor="#0d1320"
fontcolor="#e8eaf0"
fontname="Helvetica"
tools [label="Tools
tester · datagen · modelgen" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#5b8cff" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
monitors [label="Monitors
databrowse" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#5b8cff" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
}
subgraph cluster_external {
label="External APIs"
style="rounded,dashed"
color="#1e2a4a"
bgcolor="#0d1320"
fontcolor="#8892a8"
fontname="Helvetica"
jira [label="Jira" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#1e2a4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
google [label="Google" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#1e2a4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
slack [label="Slack" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#1e2a4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
}
subgraph cluster_managed {
label="Managed App"
style="rounded,dashed"
color="#1e2a4a"
bgcolor="#0d1320"
fontcolor="#8892a8"
fontname="Helvetica"
app_fe [label="Frontend" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#1e2a4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
app_be [label="Backend" shape="box" style="filled,rounded" fillcolor="#131a2a" color="#1e2a4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
app_db [label="Database" shape="cylinder" style="filled,rounded" fillcolor="#131a2a" color="#1e2a4a" fontcolor="#e8eaf0" penwidth="1" fontname="Helvetica" fontsize="10" margin="0.22,0.12" height="0.45"]
}
hub -> veins [label="routes" color="#e05c4a" fontcolor="#e8eaf0" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9"]
hub -> books [label="routes" color="#2fbf6b" fontcolor="#e8eaf0" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9"]
hub -> tools [label="routes" color="#5b8cff" fontcolor="#e8eaf0" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9"]
veins -> jira [label="API" color="#4a5568" fontcolor="#8892a8" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9"]
veins -> google [label="OAuth" color="#4a5568" fontcolor="#8892a8" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9"]
veins -> slack [label="API" color="#4a5568" fontcolor="#8892a8" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9"]
veins -> pulses [label="compose" color="#4a5568" fontcolor="#8892a8" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9"]
tools -> app_be [label="test" color="#4a5568" fontcolor="#8892a8" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9" style="dashed"]
monitors -> app_db [label="browse" color="#4a5568" fontcolor="#8892a8" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9" style="dashed"]
hub -> app_fe [label="sidebar
injection" color="#0066ff" fontcolor="#e8eaf0" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Helvetica" fontsize="9" style="dashed"]
}

View File

@@ -0,0 +1,209 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 14.1.2 (0)
-->
<!-- Title: system_overview Pages: 1 -->
<svg width="1033pt" height="415pt"
viewBox="0.00 0.00 1033.00 415.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(21.6 393.45)">
<title>system_overview</title>
<polygon fill="#0a0e17" stroke="none" points="-21.6,21.6 -21.6,-393.45 1011.6,-393.45 1011.6,21.6 -21.6,21.6"/>
<text xml:space="preserve" text-anchor="middle" x="495" y="-354.55" font-family="Helvetica,sans-Serif" font-size="14.00" fill="#e8eaf0">Soleprint — System Overview</text>
<g id="clust1" class="cluster">
<title>cluster_core</title>
<path fill="#0d1320" stroke="#0066ff" stroke-dasharray="5,2" d="M449,-241.82C449,-241.82 553,-241.82 553,-241.82 559,-241.82 565,-247.82 565,-253.82 565,-253.82 565,-326.6 565,-326.6 565,-332.6 559,-338.6 553,-338.6 553,-338.6 449,-338.6 449,-338.6 443,-338.6 437,-332.6 437,-326.6 437,-326.6 437,-253.82 437,-253.82 437,-247.82 443,-241.82 449,-241.82"/>
<text xml:space="preserve" text-anchor="middle" x="501" y="-321.3" font-family="Helvetica,sans-Serif" font-size="14.00" fill="#e8eaf0">Soleprint Hub</text>
</g>
<g id="clust2" class="cluster">
<title>cluster_artery</title>
<path fill="#0d1320" stroke="#e05c4a" stroke-dasharray="5,2" d="M20,-8C20,-8 291,-8 291,-8 297,-8 303,-14 303,-20 303,-20 303,-196.57 303,-196.57 303,-202.57 297,-208.57 291,-208.57 291,-208.57 20,-208.57 20,-208.57 14,-208.57 8,-202.57 8,-196.57 8,-196.57 8,-20 8,-20 8,-14 14,-8 20,-8"/>
<text xml:space="preserve" text-anchor="middle" x="155.5" y="-191.27" font-family="Helvetica,sans-Serif" font-size="14.00" fill="#e8eaf0">Artery — Todo lo vital</text>
</g>
<g id="clust3" class="cluster">
<title>cluster_atlas</title>
<path fill="#0d1320" stroke="#2fbf6b" stroke-dasharray="5,2" d="M323,-124.54C323,-124.54 558,-124.54 558,-124.54 564,-124.54 570,-130.54 570,-136.54 570,-136.54 570,-196.57 570,-196.57 570,-202.57 564,-208.57 558,-208.57 558,-208.57 323,-208.57 323,-208.57 317,-208.57 311,-202.57 311,-196.57 311,-196.57 311,-136.54 311,-136.54 311,-130.54 317,-124.54 323,-124.54"/>
<text xml:space="preserve" text-anchor="middle" x="440.5" y="-191.27" font-family="Helvetica,sans-Serif" font-size="14.00" fill="#e8eaf0">Atlas — Documentación accionable</text>
</g>
<g id="clust4" class="cluster">
<title>cluster_station</title>
<path fill="#0d1320" stroke="#5b8cff" stroke-dasharray="5,2" d="M590,-124.54C590,-124.54 869,-124.54 869,-124.54 875,-124.54 881,-130.54 881,-136.54 881,-136.54 881,-196.57 881,-196.57 881,-202.57 875,-208.57 869,-208.57 869,-208.57 590,-208.57 590,-208.57 584,-208.57 578,-202.57 578,-196.57 578,-196.57 578,-136.54 578,-136.54 578,-130.54 584,-124.54 590,-124.54"/>
<text xml:space="preserve" text-anchor="middle" x="729.5" y="-191.27" font-family="Helvetica,sans-Serif" font-size="14.00" fill="#e8eaf0">Station — Centro de control</text>
</g>
<g id="clust5" class="cluster">
<title>cluster_external</title>
<path fill="#0d1320" stroke="#1e2a4a" stroke-dasharray="5,2" d="M323,-13.19C323,-13.19 557,-13.19 557,-13.19 563,-13.19 569,-19.19 569,-25.19 569,-25.19 569,-74.84 569,-74.84 569,-80.84 563,-86.84 557,-86.84 557,-86.84 323,-86.84 323,-86.84 317,-86.84 311,-80.84 311,-74.84 311,-74.84 311,-25.19 311,-25.19 311,-19.19 317,-13.19 323,-13.19"/>
<text xml:space="preserve" text-anchor="middle" x="440" y="-69.54" font-family="Helvetica,sans-Serif" font-size="14.00" fill="#8892a8">External APIs</text>
</g>
<g id="clust6" class="cluster">
<title>cluster_managed</title>
<path fill="#0d1320" stroke="#1e2a4a" stroke-dasharray="5,2" d="M689,-8.74C689,-8.74 970,-8.74 970,-8.74 976,-8.74 982,-14.74 982,-20.74 982,-20.74 982,-79.29 982,-79.29 982,-85.29 976,-91.29 970,-91.29 970,-91.29 689,-91.29 689,-91.29 683,-91.29 677,-85.29 677,-79.29 677,-79.29 677,-20.74 677,-20.74 677,-14.74 683,-8.74 689,-8.74"/>
<text xml:space="preserve" text-anchor="middle" x="829.5" y="-73.99" font-family="Helvetica,sans-Serif" font-size="14.00" fill="#8892a8">Managed App</text>
</g>
<!-- hub -->
<g id="node1" class="node">
<title>hub</title>
<path fill="#131a2a" stroke="#0066ff" d="M544.59,-305.35C544.59,-305.35 457.41,-305.35 457.41,-305.35 451.41,-305.35 445.41,-299.35 445.41,-293.35 445.41,-293.35 445.41,-261.82 445.41,-261.82 445.41,-255.82 451.41,-249.82 457.41,-249.82 457.41,-249.82 544.59,-249.82 544.59,-249.82 550.59,-249.82 556.59,-255.82 556.59,-261.82 556.59,-261.82 556.59,-293.35 556.59,-293.35 556.59,-299.35 550.59,-305.35 544.59,-305.35"/>
<text xml:space="preserve" text-anchor="middle" x="501" y="-287.21" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">soleprint</text>
<text xml:space="preserve" text-anchor="middle" x="501" y="-274.46" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">core coordinator</text>
<text xml:space="preserve" text-anchor="middle" x="501" y="-261.71" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">port 12000</text>
</g>
<!-- veins -->
<g id="node2" class="node">
<title>veins</title>
<path fill="#131a2a" stroke="#e05c4a" d="M136.09,-175.32C136.09,-175.32 27.91,-175.32 27.91,-175.32 21.91,-175.32 15.91,-169.32 15.91,-163.32 15.91,-163.32 15.91,-144.54 15.91,-144.54 15.91,-138.54 21.91,-132.54 27.91,-132.54 27.91,-132.54 136.09,-132.54 136.09,-132.54 142.09,-132.54 148.09,-138.54 148.09,-144.54 148.09,-144.54 148.09,-163.32 148.09,-163.32 148.09,-169.32 142.09,-175.32 136.09,-175.32"/>
<text xml:space="preserve" text-anchor="middle" x="82" y="-157.18" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Veins</text>
<text xml:space="preserve" text-anchor="middle" x="82" y="-144.43" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">stateless connectors</text>
</g>
<!-- hub&#45;&gt;veins -->
<g id="edge1" class="edge">
<title>hub&#45;&gt;veins</title>
<path fill="none" stroke="#e05c4a" d="M445.05,-271.7C376.82,-264.3 258.98,-246.84 165,-208.57 147.66,-201.5 130.02,-190.71 115.48,-180.66"/>
<polygon fill="#e05c4a" stroke="#e05c4a" points="116.98,-178.71 109.85,-176.67 114.15,-182.72 116.98,-178.71"/>
<text xml:space="preserve" text-anchor="middle" x="231.94" y="-219.27" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#e8eaf0">routes</text>
</g>
<!-- books -->
<g id="node5" class="node">
<title>books</title>
<path fill="#131a2a" stroke="#2fbf6b" d="M427.59,-175.32C427.59,-175.32 346.41,-175.32 346.41,-175.32 340.41,-175.32 334.41,-169.32 334.41,-163.32 334.41,-163.32 334.41,-144.54 334.41,-144.54 334.41,-138.54 340.41,-132.54 346.41,-132.54 346.41,-132.54 427.59,-132.54 427.59,-132.54 433.59,-132.54 439.59,-138.54 439.59,-144.54 439.59,-144.54 439.59,-163.32 439.59,-163.32 439.59,-169.32 433.59,-175.32 427.59,-175.32"/>
<text xml:space="preserve" text-anchor="middle" x="387" y="-157.18" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Books</text>
<text xml:space="preserve" text-anchor="middle" x="387" y="-144.43" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">documentation</text>
</g>
<!-- hub&#45;&gt;books -->
<g id="edge2" class="edge">
<title>hub&#45;&gt;books</title>
<path fill="none" stroke="#2fbf6b" d="M475.7,-249.58C456.8,-229.41 431.07,-201.96 412.19,-181.81"/>
<polygon fill="#2fbf6b" stroke="#2fbf6b" points="414.18,-180.35 407.6,-176.91 410.6,-183.7 414.18,-180.35"/>
<text xml:space="preserve" text-anchor="middle" x="468.46" y="-219.27" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#e8eaf0">routes</text>
</g>
<!-- tools -->
<g id="node7" class="node">
<title>tools</title>
<path fill="#131a2a" stroke="#5b8cff" d="M740.34,-175.32C740.34,-175.32 597.66,-175.32 597.66,-175.32 591.66,-175.32 585.66,-169.32 585.66,-163.32 585.66,-163.32 585.66,-144.54 585.66,-144.54 585.66,-138.54 591.66,-132.54 597.66,-132.54 597.66,-132.54 740.34,-132.54 740.34,-132.54 746.34,-132.54 752.34,-138.54 752.34,-144.54 752.34,-144.54 752.34,-163.32 752.34,-163.32 752.34,-169.32 746.34,-175.32 740.34,-175.32"/>
<text xml:space="preserve" text-anchor="middle" x="669" y="-157.18" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Tools</text>
<text xml:space="preserve" text-anchor="middle" x="669" y="-144.43" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">tester · datagen · modelgen</text>
</g>
<!-- hub&#45;&gt;tools -->
<g id="edge3" class="edge">
<title>hub&#45;&gt;tools</title>
<path fill="none" stroke="#5b8cff" d="M538.28,-249.58C566.62,-229.06 605.36,-201.01 633.3,-180.77"/>
<polygon fill="#5b8cff" stroke="#5b8cff" points="634.7,-182.79 638.94,-176.7 631.83,-178.82 634.7,-182.79"/>
<text xml:space="preserve" text-anchor="middle" x="594.21" y="-219.27" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#e8eaf0">routes</text>
</g>
<!-- app_fe -->
<g id="node12" class="node">
<title>app_fe</title>
<path fill="#131a2a" stroke="#1e2a4a" d="M963.42,-53.59C963.42,-53.59 910.58,-53.59 910.58,-53.59 905.18,-53.59 899.78,-48.19 899.78,-42.79 899.78,-42.79 899.78,-31.99 899.78,-31.99 899.78,-26.59 905.18,-21.19 910.58,-21.19 910.58,-21.19 963.42,-21.19 963.42,-21.19 968.82,-21.19 974.22,-26.59 974.22,-31.99 974.22,-31.99 974.22,-42.79 974.22,-42.79 974.22,-48.19 968.82,-53.59 963.42,-53.59"/>
<text xml:space="preserve" text-anchor="middle" x="937" y="-34.27" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Frontend</text>
</g>
<!-- hub&#45;&gt;app_fe -->
<g id="edge10" class="edge">
<title>hub&#45;&gt;app_fe</title>
<path fill="none" stroke="#0066ff" stroke-dasharray="5,2" d="M556.96,-273.95C652.42,-268.32 841.05,-251.89 889,-208.57 930.96,-170.65 937.39,-99.18 937.69,-62.08"/>
<polygon fill="#0066ff" stroke="#0066ff" points="940.14,-62.36 937.67,-55.37 935.24,-62.37 940.14,-62.36"/>
<text xml:space="preserve" text-anchor="middle" x="950.21" y="-156.63" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#e8eaf0">sidebar</text>
<text xml:space="preserve" text-anchor="middle" x="950.21" y="-145.38" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#e8eaf0">injection</text>
</g>
<!-- pulses -->
<g id="node4" class="node">
<title>pulses</title>
<path fill="#131a2a" stroke="#e05c4a" d="M125.22,-58.78C125.22,-58.78 38.78,-58.78 38.78,-58.78 32.78,-58.78 26.78,-52.78 26.78,-46.78 26.78,-46.78 26.78,-28 26.78,-28 26.78,-22 32.78,-16 38.78,-16 38.78,-16 125.22,-16 125.22,-16 131.22,-16 137.22,-22 137.22,-28 137.22,-28 137.22,-46.78 137.22,-46.78 137.22,-52.78 131.22,-58.78 125.22,-58.78"/>
<text xml:space="preserve" text-anchor="middle" x="82" y="-40.64" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Pulses</text>
<text xml:space="preserve" text-anchor="middle" x="82" y="-27.89" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">composed flows</text>
</g>
<!-- veins&#45;&gt;pulses -->
<g id="edge7" class="edge">
<title>veins&#45;&gt;pulses</title>
<path fill="none" stroke="#4a5568" d="M82,-132.18C82,-114.1 82,-87.68 82,-67.49"/>
<polygon fill="#4a5568" stroke="#4a5568" points="84.45,-67.49 82,-60.49 79.55,-67.49 84.45,-67.49"/>
<text xml:space="preserve" text-anchor="middle" x="102.25" y="-101.99" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">compose</text>
</g>
<!-- jira -->
<g id="node9" class="node">
<title>jira</title>
<path fill="#131a2a" stroke="#1e2a4a" d="M362.2,-53.59C362.2,-53.59 329.8,-53.59 329.8,-53.59 324.4,-53.59 319,-48.19 319,-42.79 319,-42.79 319,-31.99 319,-31.99 319,-26.59 324.4,-21.19 329.8,-21.19 329.8,-21.19 362.2,-21.19 362.2,-21.19 367.6,-21.19 373,-26.59 373,-31.99 373,-31.99 373,-42.79 373,-42.79 373,-48.19 367.6,-53.59 362.2,-53.59"/>
<text xml:space="preserve" text-anchor="middle" x="346" y="-34.27" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Jira</text>
</g>
<!-- veins&#45;&gt;jira -->
<g id="edge4" class="edge">
<title>veins&#45;&gt;jira</title>
<path fill="none" stroke="#4a5568" d="M140.25,-132.14C148.52,-129.44 156.94,-126.82 165,-124.54 229.02,-106.35 256.27,-129.14 311,-91.29 321.79,-83.83 330.11,-71.86 335.93,-61.19"/>
<polygon fill="#4a5568" stroke="#4a5568" points="338.05,-62.44 339.04,-55.09 333.68,-60.22 338.05,-62.44"/>
<text xml:space="preserve" text-anchor="middle" x="302.47" y="-101.99" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">API</text>
</g>
<!-- google -->
<g id="node10" class="node">
<title>google</title>
<path fill="#131a2a" stroke="#1e2a4a" d="M460.29,-53.59C460.29,-53.59 415.71,-53.59 415.71,-53.59 410.31,-53.59 404.91,-48.19 404.91,-42.79 404.91,-42.79 404.91,-31.99 404.91,-31.99 404.91,-26.59 410.31,-21.19 415.71,-21.19 415.71,-21.19 460.29,-21.19 460.29,-21.19 465.69,-21.19 471.09,-26.59 471.09,-31.99 471.09,-31.99 471.09,-42.79 471.09,-42.79 471.09,-48.19 465.69,-53.59 460.29,-53.59"/>
<text xml:space="preserve" text-anchor="middle" x="438" y="-34.27" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Google</text>
</g>
<!-- veins&#45;&gt;google -->
<g id="edge5" class="edge">
<title>veins&#45;&gt;google</title>
<path fill="none" stroke="#4a5568" d="M136.84,-132.12C146.14,-129.19 155.77,-126.51 165,-124.54 263.43,-103.51 302.89,-143.39 389,-91.29 401.95,-83.45 413.6,-71.22 422.28,-60.49"/>
<polygon fill="#4a5568" stroke="#4a5568" points="424,-62.28 426.36,-55.25 420.13,-59.27 424,-62.28"/>
<text xml:space="preserve" text-anchor="middle" x="388.23" y="-101.99" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">OAuth</text>
</g>
<!-- slack -->
<g id="node11" class="node">
<title>slack</title>
<path fill="#131a2a" stroke="#1e2a4a" d="M550.17,-53.59C550.17,-53.59 513.84,-53.59 513.84,-53.59 508.44,-53.59 503.04,-48.19 503.04,-42.79 503.04,-42.79 503.04,-31.99 503.04,-31.99 503.04,-26.59 508.44,-21.19 513.84,-21.19 513.84,-21.19 550.17,-21.19 550.17,-21.19 555.57,-21.19 560.97,-26.59 560.97,-31.99 560.97,-31.99 560.97,-42.79 560.97,-42.79 560.97,-48.19 555.57,-53.59 550.17,-53.59"/>
<text xml:space="preserve" text-anchor="middle" x="532" y="-34.27" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Slack</text>
</g>
<!-- veins&#45;&gt;slack -->
<g id="edge6" class="edge">
<title>veins&#45;&gt;slack</title>
<path fill="none" stroke="#4a5568" d="M135.89,-132.13C145.46,-129.12 155.43,-126.41 165,-124.54 235.6,-110.71 425.88,-129.22 487,-91.29 499.44,-83.57 510.2,-71.27 518.08,-60.48"/>
<polygon fill="#4a5568" stroke="#4a5568" points="519.97,-62.05 521.96,-54.91 515.95,-59.25 519.97,-62.05"/>
<text xml:space="preserve" text-anchor="middle" x="477.1" y="-101.99" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">API</text>
</g>
<!-- shunts -->
<g id="node3" class="node">
<title>shunts</title>
<path fill="#131a2a" stroke="#e05c4a" d="M283.47,-175.32C283.47,-175.32 192.53,-175.32 192.53,-175.32 186.53,-175.32 180.53,-169.32 180.53,-163.32 180.53,-163.32 180.53,-144.54 180.53,-144.54 180.53,-138.54 186.53,-132.54 192.53,-132.54 192.53,-132.54 283.47,-132.54 283.47,-132.54 289.47,-132.54 295.47,-138.54 295.47,-144.54 295.47,-144.54 295.47,-163.32 295.47,-163.32 295.47,-169.32 289.47,-175.32 283.47,-175.32"/>
<text xml:space="preserve" text-anchor="middle" x="238" y="-157.18" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Shunts</text>
<text xml:space="preserve" text-anchor="middle" x="238" y="-144.43" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">mock connectors</text>
</g>
<!-- templates -->
<g id="node6" class="node">
<title>templates</title>
<path fill="#131a2a" stroke="#2fbf6b" d="M541.97,-175.32C541.97,-175.32 484.03,-175.32 484.03,-175.32 478.03,-175.32 472.03,-169.32 472.03,-163.32 472.03,-163.32 472.03,-144.54 472.03,-144.54 472.03,-138.54 478.03,-132.54 484.03,-132.54 484.03,-132.54 541.97,-132.54 541.97,-132.54 547.97,-132.54 553.97,-138.54 553.97,-144.54 553.97,-144.54 553.97,-163.32 553.97,-163.32 553.97,-169.32 547.97,-175.32 541.97,-175.32"/>
<text xml:space="preserve" text-anchor="middle" x="513" y="-157.18" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Templates</text>
<text xml:space="preserve" text-anchor="middle" x="513" y="-144.43" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">patterns</text>
</g>
<!-- app_be -->
<g id="node13" class="node">
<title>app_be</title>
<path fill="#131a2a" stroke="#1e2a4a" d="M746.67,-53.59C746.67,-53.59 695.33,-53.59 695.33,-53.59 689.93,-53.59 684.53,-48.19 684.53,-42.79 684.53,-42.79 684.53,-31.99 684.53,-31.99 684.53,-26.59 689.93,-21.19 695.33,-21.19 695.33,-21.19 746.67,-21.19 746.67,-21.19 752.07,-21.19 757.47,-26.59 757.47,-31.99 757.47,-31.99 757.47,-42.79 757.47,-42.79 757.47,-48.19 752.07,-53.59 746.67,-53.59"/>
<text xml:space="preserve" text-anchor="middle" x="721" y="-34.27" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Backend</text>
</g>
<!-- tools&#45;&gt;app_be -->
<g id="edge8" class="edge">
<title>tools&#45;&gt;app_be</title>
<path fill="none" stroke="#4a5568" stroke-dasharray="5,2" d="M678.42,-132.18C687.46,-112.27 701.1,-82.23 710.49,-61.54"/>
<polygon fill="#4a5568" stroke="#4a5568" points="712.7,-62.61 713.36,-55.22 708.23,-60.58 712.7,-62.61"/>
<text xml:space="preserve" text-anchor="middle" x="700.51" y="-101.99" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">test</text>
</g>
<!-- monitors -->
<g id="node8" class="node">
<title>monitors</title>
<path fill="#131a2a" stroke="#5b8cff" d="M861.34,-175.32C861.34,-175.32 796.66,-175.32 796.66,-175.32 790.66,-175.32 784.66,-169.32 784.66,-163.32 784.66,-163.32 784.66,-144.54 784.66,-144.54 784.66,-138.54 790.66,-132.54 796.66,-132.54 796.66,-132.54 861.34,-132.54 861.34,-132.54 867.34,-132.54 873.34,-138.54 873.34,-144.54 873.34,-144.54 873.34,-163.32 873.34,-163.32 873.34,-169.32 867.34,-175.32 861.34,-175.32"/>
<text xml:space="preserve" text-anchor="middle" x="829" y="-157.18" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Monitors</text>
<text xml:space="preserve" text-anchor="middle" x="829" y="-144.43" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">databrowse</text>
</g>
<!-- app_db -->
<g id="node14" class="node">
<title>app_db</title>
<path fill="#131a2a" stroke="#1e2a4a" d="M868.09,-54.28C868.09,-56.35 850.57,-58.04 829,-58.04 807.43,-58.04 789.91,-56.35 789.91,-54.28 789.91,-54.28 789.91,-20.5 789.91,-20.5 789.91,-18.43 807.43,-16.74 829,-16.74 850.57,-16.74 868.09,-18.43 868.09,-20.5 868.09,-20.5 868.09,-54.28 868.09,-54.28"/>
<path fill="none" stroke="#1e2a4a" d="M868.09,-54.28C868.09,-52.21 850.57,-50.53 829,-50.53 807.43,-50.53 789.91,-52.21 789.91,-54.28"/>
<text xml:space="preserve" text-anchor="middle" x="829" y="-34.27" font-family="Helvetica,sans-Serif" font-size="10.00" fill="#e8eaf0">Database</text>
</g>
<!-- monitors&#45;&gt;app_db -->
<g id="edge9" class="edge">
<title>monitors&#45;&gt;app_db</title>
<path fill="none" stroke="#4a5568" stroke-dasharray="5,2" d="M829,-132.18C829,-113.8 829,-86.78 829,-66.46"/>
<polygon fill="#4a5568" stroke="#4a5568" points="831.45,-66.72 829,-59.72 826.55,-66.72 831.45,-66.72"/>
<text xml:space="preserve" text-anchor="middle" x="845.12" y="-101.99" font-family="Helvetica,sans-Serif" font-size="9.00" fill="#8892a8">browse</text>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 19 KiB

View File

@@ -0,0 +1,107 @@
digraph system_overview {
bgcolor="#ffffff"
rankdir=TB
splines="ortho"
nodesep="0.55"
ranksep="0.7"
pad="0.3"
fontname="Arial"
label="Soleprint — System Overview"
labelloc=t
fontsize="14"
fontcolor="#1f2933"
compound=true
node [shape="box" style="filled,rounded" fillcolor="#ffffff" color="#9aa5b1" penwidth="1" fontname="Arial" fontsize="10" fontcolor="#1f2933" margin="0.25,0.14" height="0.5"]
edge [color="#9aa5b1" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9" fontcolor="#616e7c"]
subgraph cluster_core {
label="Soleprint Hub"
style="rounded,dashed"
color="#3a7dff"
bgcolor="#d6e4ff"
fontcolor="#616e7c"
fontname="Arial"
hub [label="soleprint
core coordinator
port 12000" shape="box" style="filled,rounded" fillcolor="#d6e4ff" color="#3a7dff" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
}
subgraph cluster_artery {
label="Artery — Todo lo vital"
style="rounded,dashed"
color="#c0392b"
bgcolor="#fdeaea"
fontcolor="#616e7c"
fontname="Arial"
veins [label="Veins
stateless connectors" shape="box" style="filled,rounded" fillcolor="#fdeaea" color="#c0392b" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
shunts [label="Shunts
mock connectors" shape="box" style="filled,rounded" fillcolor="#fdeaea" color="#c0392b" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
pulses [label="Pulses
composed flows" shape="box" style="filled,rounded" fillcolor="#fdeaea" color="#c0392b" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
}
subgraph cluster_atlas {
label="Atlas — Documentación accionable"
style="rounded,dashed"
color="#1a7f45"
bgcolor="#e6f5ec"
fontcolor="#616e7c"
fontname="Arial"
books [label="Books
documentation" shape="box" style="filled,rounded" fillcolor="#e6f5ec" color="#1a7f45" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
templates [label="Templates
patterns" shape="box" style="filled,rounded" fillcolor="#e6f5ec" color="#1a7f45" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
}
subgraph cluster_station {
label="Station — Centro de control"
style="rounded,dashed"
color="#2b5fd9"
bgcolor="#e8effd"
fontcolor="#616e7c"
fontname="Arial"
tools [label="Tools
tester · datagen · modelgen" shape="box" style="filled,rounded" fillcolor="#e8effd" color="#2b5fd9" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
monitors [label="Monitors
databrowse" shape="box" style="filled,rounded" fillcolor="#e8effd" color="#2b5fd9" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
}
subgraph cluster_external {
label="External APIs"
style="rounded,dashed"
color="#cbd2d9"
bgcolor="#f5f7fa"
fontcolor="#616e7c"
fontname="Arial"
jira [label="Jira" shape="box" style="filled,rounded" fillcolor="#ffffff" color="#9aa5b1" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
google [label="Google" shape="box" style="filled,rounded" fillcolor="#ffffff" color="#9aa5b1" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
slack [label="Slack" shape="box" style="filled,rounded" fillcolor="#ffffff" color="#9aa5b1" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
}
subgraph cluster_managed {
label="Managed App"
style="rounded,dashed"
color="#cbd2d9"
bgcolor="#f5f7fa"
fontcolor="#616e7c"
fontname="Arial"
app_fe [label="Frontend" shape="box" style="filled,rounded" fillcolor="#ffffff" color="#9aa5b1" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
app_be [label="Backend" shape="box" style="filled,rounded" fillcolor="#ffffff" color="#9aa5b1" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
app_db [label="Database" shape="cylinder" style="filled,rounded" fillcolor="#ffffff" color="#9aa5b1" fontcolor="#1f2933" penwidth="1" fontname="Arial" fontsize="10" margin="0.25,0.14" height="0.5"]
}
hub -> veins [xlabel="routes" color="#c0392b" fontcolor="#c0392b" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9"]
hub -> books [xlabel="routes" color="#1a7f45" fontcolor="#1a7f45" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9"]
hub -> tools [xlabel="routes" color="#2b5fd9" fontcolor="#2b5fd9" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9"]
veins -> jira [xlabel="API" color="#9aa5b1" fontcolor="#616e7c" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9"]
veins -> google [xlabel="OAuth" color="#9aa5b1" fontcolor="#616e7c" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9"]
veins -> slack [xlabel="API" color="#9aa5b1" fontcolor="#616e7c" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9"]
veins -> pulses [xlabel="compose" color="#9aa5b1" fontcolor="#616e7c" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9"]
tools -> app_be [xlabel="test" color="#9aa5b1" fontcolor="#616e7c" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9" style="dashed"]
monitors -> app_db [xlabel="browse" color="#9aa5b1" fontcolor="#616e7c" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9" style="dashed"]
hub -> app_fe [xlabel="sidebar
injection" color="#3a7dff" fontcolor="#3a7dff" penwidth="1" arrowhead="normal" arrowsize="0.7" fontname="Arial" fontsize="9" style="dashed"]
}

View File

@@ -0,0 +1,209 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 14.1.2 (0)
-->
<!-- Title: system_overview Pages: 1 -->
<svg width="1032pt" height="367pt"
viewBox="0.00 0.00 1032.00 367.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(21.6 345.64)">
<title>system_overview</title>
<polygon fill="#ffffff" stroke="none" points="-21.6,21.6 -21.6,-345.64 1010.6,-345.64 1010.6,21.6 -21.6,21.6"/>
<text xml:space="preserve" text-anchor="middle" x="494.5" y="-306.74" font-family="Arial" font-size="14.00" fill="#1f2933">Soleprint — System Overview</text>
<g id="clust1" class="cluster">
<title>cluster_core</title>
<path fill="#d6e4ff" stroke="#3a7dff" stroke-dasharray="5,2" d="M478,-196.38C478,-196.38 576,-196.38 576,-196.38 582,-196.38 588,-202.38 588,-208.38 588,-208.38 588,-280.29 588,-280.29 588,-286.29 582,-292.29 576,-292.29 576,-292.29 478,-292.29 478,-292.29 472,-292.29 466,-286.29 466,-280.29 466,-280.29 466,-208.38 466,-208.38 466,-202.38 472,-196.38 478,-196.38"/>
<text xml:space="preserve" text-anchor="middle" x="527" y="-274.99" font-family="Arial" font-size="14.00" fill="#616e7c">Soleprint Hub</text>
</g>
<g id="clust2" class="cluster">
<title>cluster_artery</title>
<path fill="#fdeaea" stroke="#c0392b" stroke-dasharray="5,2" d="M20,-8.03C20,-8.03 290,-8.03 290,-8.03 296,-8.03 302,-14.03 302,-20.03 302,-20.03 302,-174.13 302,-174.13 302,-180.13 296,-186.13 290,-186.13 290,-186.13 20,-186.13 20,-186.13 14,-186.13 8,-180.13 8,-174.13 8,-174.13 8,-20.03 8,-20.03 8,-14.03 14,-8.03 20,-8.03"/>
<text xml:space="preserve" text-anchor="middle" x="155" y="-168.83" font-family="Arial" font-size="14.00" fill="#616e7c">Artery — Todo lo vital</text>
</g>
<g id="clust3" class="cluster">
<title>cluster_atlas</title>
<path fill="#e6f5ec" stroke="#1a7f45" stroke-dasharray="5,2" d="M337,-102.22C337,-102.22 550,-102.22 550,-102.22 556,-102.22 562,-108.22 562,-114.22 562,-114.22 562,-174.13 562,-174.13 562,-180.13 556,-186.13 550,-186.13 550,-186.13 337,-186.13 337,-186.13 331,-186.13 325,-180.13 325,-174.13 325,-174.13 325,-114.22 325,-114.22 325,-108.22 331,-102.22 337,-102.22"/>
<text xml:space="preserve" text-anchor="middle" x="443.5" y="-168.83" font-family="Arial" font-size="14.00" fill="#616e7c">Atlas — Documentación accionable</text>
</g>
<g id="clust4" class="cluster">
<title>cluster_station</title>
<path fill="#e8effd" stroke="#2b5fd9" stroke-dasharray="5,2" d="M597,-102.22C597,-102.22 871,-102.22 871,-102.22 877,-102.22 883,-108.22 883,-114.22 883,-114.22 883,-174.13 883,-174.13 883,-180.13 877,-186.13 871,-186.13 871,-186.13 597,-186.13 597,-186.13 591,-186.13 585,-180.13 585,-174.13 585,-174.13 585,-114.22 585,-114.22 585,-108.22 591,-102.22 597,-102.22"/>
<text xml:space="preserve" text-anchor="middle" x="734" y="-168.83" font-family="Arial" font-size="14.00" fill="#616e7c">Station — Centro de control</text>
</g>
<g id="clust5" class="cluster">
<title>cluster_external</title>
<path fill="#f5f7fa" stroke="#cbd2d9" stroke-dasharray="5,2" d="M322,-12.11C322,-12.11 575,-12.11 575,-12.11 581,-12.11 587,-18.11 587,-24.11 587,-24.11 587,-75.86 587,-75.86 587,-81.86 581,-87.86 575,-87.86 575,-87.86 322,-87.86 322,-87.86 316,-87.86 310,-81.86 310,-75.86 310,-75.86 310,-24.11 310,-24.11 310,-18.11 316,-12.11 322,-12.11"/>
<text xml:space="preserve" text-anchor="middle" x="448.5" y="-70.56" font-family="Arial" font-size="14.00" fill="#616e7c">External APIs</text>
</g>
<g id="clust6" class="cluster">
<title>cluster_managed</title>
<path fill="#f5f7fa" stroke="#cbd2d9" stroke-dasharray="5,2" d="M671,-8C671,-8 969,-8 969,-8 975,-8 981,-14 981,-20 981,-20 981,-79.97 981,-79.97 981,-85.97 975,-91.97 969,-91.97 969,-91.97 671,-91.97 671,-91.97 665,-91.97 659,-85.97 659,-79.97 659,-79.97 659,-20 659,-20 659,-14 665,-8 671,-8"/>
<text xml:space="preserve" text-anchor="middle" x="820" y="-74.67" font-family="Arial" font-size="14.00" fill="#616e7c">Managed App</text>
</g>
<!-- hub -->
<g id="node1" class="node">
<title>hub</title>
<path fill="#d6e4ff" stroke="#3a7dff" d="M567.88,-260.54C567.88,-260.54 486.12,-260.54 486.12,-260.54 480.12,-260.54 474.12,-254.54 474.12,-248.54 474.12,-248.54 474.12,-216.38 474.12,-216.38 474.12,-210.38 480.12,-204.38 486.12,-204.38 486.12,-204.38 567.88,-204.38 567.88,-204.38 573.88,-204.38 579.88,-210.38 579.88,-216.38 579.88,-216.38 579.88,-248.54 579.88,-248.54 579.88,-254.54 573.88,-260.54 567.88,-260.54"/>
<text xml:space="preserve" text-anchor="middle" x="527" y="-240.96" font-family="Arial" font-size="10.00" fill="#1f2933">soleprint</text>
<text xml:space="preserve" text-anchor="middle" x="527" y="-228.96" font-family="Arial" font-size="10.00" fill="#1f2933">core coordinator</text>
<text xml:space="preserve" text-anchor="middle" x="527" y="-216.96" font-family="Arial" font-size="10.00" fill="#1f2933">port 12000</text>
</g>
<!-- veins -->
<g id="node2" class="node">
<title>veins</title>
<path fill="#fdeaea" stroke="#c0392b" d="M130.38,-154.38C130.38,-154.38 27.62,-154.38 27.62,-154.38 21.62,-154.38 15.62,-148.38 15.62,-142.38 15.62,-142.38 15.62,-122.22 15.62,-122.22 15.62,-116.22 21.62,-110.22 27.62,-110.22 27.62,-110.22 130.38,-110.22 130.38,-110.22 136.38,-110.22 142.38,-116.22 142.38,-122.22 142.38,-122.22 142.38,-142.38 142.38,-142.38 142.38,-148.38 136.38,-154.38 130.38,-154.38"/>
<text xml:space="preserve" text-anchor="middle" x="79" y="-134.8" font-family="Arial" font-size="10.00" fill="#1f2933">Veins</text>
<text xml:space="preserve" text-anchor="middle" x="79" y="-122.8" font-family="Arial" font-size="10.00" fill="#1f2933">stateless connectors</text>
</g>
<!-- hub&#45;&gt;veins -->
<g id="edge1" class="edge">
<title>hub&#45;&gt;veins</title>
<path fill="none" stroke="#c0392b" d="M473.85,-242C355.18,-242 79,-242 79,-242 79,-242 79,-163.28 79,-163.28"/>
<polygon fill="#c0392b" stroke="#c0392b" points="81.45,-163.28 79,-156.28 76.55,-163.28 81.45,-163.28"/>
<text xml:space="preserve" text-anchor="middle" x="224.31" y="-243.95" font-family="Arial" font-size="9.00" fill="#c0392b">routes</text>
</g>
<!-- books -->
<g id="node5" class="node">
<title>books</title>
<path fill="#e6f5ec" stroke="#1a7f45" d="M420.88,-154.38C420.88,-154.38 345.12,-154.38 345.12,-154.38 339.12,-154.38 333.12,-148.38 333.12,-142.38 333.12,-142.38 333.12,-122.22 333.12,-122.22 333.12,-116.22 339.12,-110.22 345.12,-110.22 345.12,-110.22 420.88,-110.22 420.88,-110.22 426.88,-110.22 432.88,-116.22 432.88,-122.22 432.88,-122.22 432.88,-142.38 432.88,-142.38 432.88,-148.38 426.88,-154.38 420.88,-154.38"/>
<text xml:space="preserve" text-anchor="middle" x="383" y="-134.8" font-family="Arial" font-size="10.00" fill="#1f2933">Books</text>
<text xml:space="preserve" text-anchor="middle" x="383" y="-122.8" font-family="Arial" font-size="10.00" fill="#1f2933">documentation</text>
</g>
<!-- hub&#45;&gt;books -->
<g id="edge2" class="edge">
<title>hub&#45;&gt;books</title>
<path fill="none" stroke="#1a7f45" d="M473.79,-223C432.77,-223 383,-223 383,-223 383,-223 383,-163.23 383,-163.23"/>
<polygon fill="#1a7f45" stroke="#1a7f45" points="385.45,-163.23 383,-156.23 380.55,-163.23 385.45,-163.23"/>
<text xml:space="preserve" text-anchor="middle" x="385.76" y="-224.95" font-family="Arial" font-size="9.00" fill="#1a7f45">routes</text>
</g>
<!-- tools -->
<g id="node7" class="node">
<title>tools</title>
<path fill="#e8effd" stroke="#2b5fd9" d="M736.62,-154.38C736.62,-154.38 605.38,-154.38 605.38,-154.38 599.38,-154.38 593.38,-148.38 593.38,-142.38 593.38,-142.38 593.38,-122.22 593.38,-122.22 593.38,-116.22 599.38,-110.22 605.38,-110.22 605.38,-110.22 736.62,-110.22 736.62,-110.22 742.62,-110.22 748.62,-116.22 748.62,-122.22 748.62,-122.22 748.62,-142.38 748.62,-142.38 748.62,-148.38 742.62,-154.38 736.62,-154.38"/>
<text xml:space="preserve" text-anchor="middle" x="671" y="-134.8" font-family="Arial" font-size="10.00" fill="#1f2933">Tools</text>
<text xml:space="preserve" text-anchor="middle" x="671" y="-122.8" font-family="Arial" font-size="10.00" fill="#1f2933">tester · datagen · modelgen</text>
</g>
<!-- hub&#45;&gt;tools -->
<g id="edge3" class="edge">
<title>hub&#45;&gt;tools</title>
<path fill="none" stroke="#2b5fd9" d="M566.44,-203.94C566.44,-174.24 566.44,-132 566.44,-132 566.44,-132 584.58,-132 584.58,-132"/>
<polygon fill="#2b5fd9" stroke="#2b5fd9" points="584.58,-134.45 591.58,-132 584.58,-129.55 584.58,-134.45"/>
<text xml:space="preserve" text-anchor="middle" x="579.19" y="-160.85" font-family="Arial" font-size="9.00" fill="#2b5fd9">routes</text>
</g>
<!-- app_fe -->
<g id="node12" class="node">
<title>app_fe</title>
<path fill="#ffffff" stroke="#9aa5b1" d="M961.12,-56.11C961.12,-56.11 910.88,-56.11 910.88,-56.11 904.88,-56.11 898.88,-50.11 898.88,-44.11 898.88,-44.11 898.88,-32.11 898.88,-32.11 898.88,-26.11 904.88,-20.11 910.88,-20.11 910.88,-20.11 961.12,-20.11 961.12,-20.11 967.12,-20.11 973.12,-26.11 973.12,-32.11 973.12,-32.11 973.12,-44.11 973.12,-44.11 973.12,-50.11 967.12,-56.11 961.12,-56.11"/>
<text xml:space="preserve" text-anchor="middle" x="936" y="-34.61" font-family="Arial" font-size="10.00" fill="#1f2933">Frontend</text>
</g>
<!-- hub&#45;&gt;app_fe -->
<g id="edge10" class="edge">
<title>hub&#45;&gt;app_fe</title>
<path fill="none" stroke="#3a7dff" stroke-dasharray="5,2" d="M580.05,-232C690.72,-232 936,-232 936,-232 936,-232 936,-64.86 936,-64.86"/>
<polygon fill="#3a7dff" stroke="#3a7dff" points="938.45,-64.86 936,-57.86 933.55,-64.86 938.45,-64.86"/>
<text xml:space="preserve" text-anchor="middle" x="824.34" y="-244.45" font-family="Arial" font-size="9.00" fill="#3a7dff">sidebar</text>
<text xml:space="preserve" text-anchor="middle" x="824.34" y="-233.95" font-family="Arial" font-size="9.00" fill="#3a7dff">injection</text>
</g>
<!-- pulses -->
<g id="node4" class="node">
<title>pulses</title>
<path fill="#fdeaea" stroke="#c0392b" d="M120.62,-60.19C120.62,-60.19 37.38,-60.19 37.38,-60.19 31.38,-60.19 25.38,-54.19 25.38,-48.19 25.38,-48.19 25.38,-28.03 25.38,-28.03 25.38,-22.03 31.38,-16.03 37.38,-16.03 37.38,-16.03 120.62,-16.03 120.62,-16.03 126.62,-16.03 132.62,-22.03 132.62,-28.03 132.62,-28.03 132.62,-48.19 132.62,-48.19 132.62,-54.19 126.62,-60.19 120.62,-60.19"/>
<text xml:space="preserve" text-anchor="middle" x="79" y="-40.61" font-family="Arial" font-size="10.00" fill="#1f2933">Pulses</text>
<text xml:space="preserve" text-anchor="middle" x="79" y="-28.61" font-family="Arial" font-size="10.00" fill="#1f2933">composed flows</text>
</g>
<!-- veins&#45;&gt;pulses -->
<g id="edge7" class="edge">
<title>veins&#45;&gt;pulses</title>
<path fill="none" stroke="#9aa5b1" d="M79,-109.98C79,-109.98 79,-69.11 79,-69.11"/>
<polygon fill="#9aa5b1" stroke="#9aa5b1" points="81.45,-69.11 79,-62.11 76.55,-69.11 81.45,-69.11"/>
<text xml:space="preserve" text-anchor="middle" x="60.25" y="-91.49" font-family="Arial" font-size="9.00" fill="#616e7c">compose</text>
</g>
<!-- jira -->
<g id="node9" class="node">
<title>jira</title>
<path fill="#ffffff" stroke="#9aa5b1" d="M360,-56.11C360,-56.11 330,-56.11 330,-56.11 324,-56.11 318,-50.11 318,-44.11 318,-44.11 318,-32.11 318,-32.11 318,-26.11 324,-20.11 330,-20.11 330,-20.11 360,-20.11 360,-20.11 366,-20.11 372,-26.11 372,-32.11 372,-32.11 372,-44.11 372,-44.11 372,-50.11 366,-56.11 360,-56.11"/>
<text xml:space="preserve" text-anchor="middle" x="345" y="-34.61" font-family="Arial" font-size="10.00" fill="#1f2933">Jira</text>
</g>
<!-- veins&#45;&gt;jira -->
<g id="edge4" class="edge">
<title>veins&#45;&gt;jira</title>
<path fill="none" stroke="#9aa5b1" d="M139.94,-109.95C139.94,-82.07 139.94,-38 139.94,-38 139.94,-38 309.26,-38 309.26,-38"/>
<polygon fill="#9aa5b1" stroke="#9aa5b1" points="309.26,-40.45 316.26,-38 309.26,-35.55 309.26,-40.45"/>
<text xml:space="preserve" text-anchor="middle" x="181.5" y="-39.95" font-family="Arial" font-size="9.00" fill="#616e7c">API</text>
</g>
<!-- google -->
<g id="node10" class="node">
<title>google</title>
<path fill="#ffffff" stroke="#9aa5b1" d="M466.38,-56.11C466.38,-56.11 423.62,-56.11 423.62,-56.11 417.62,-56.11 411.62,-50.11 411.62,-44.11 411.62,-44.11 411.62,-32.11 411.62,-32.11 411.62,-26.11 417.62,-20.11 423.62,-20.11 423.62,-20.11 466.38,-20.11 466.38,-20.11 472.38,-20.11 478.38,-26.11 478.38,-32.11 478.38,-32.11 478.38,-44.11 478.38,-44.11 478.38,-50.11 472.38,-56.11 466.38,-56.11"/>
<text xml:space="preserve" text-anchor="middle" x="445" y="-34.61" font-family="Arial" font-size="10.00" fill="#1f2933">Google</text>
</g>
<!-- veins&#45;&gt;google -->
<g id="edge5" class="edge">
<title>veins&#45;&gt;google</title>
<path fill="none" stroke="#9aa5b1" d="M135.06,-109.81C135.06,-94.5 135.06,-77 135.06,-77 135.06,-77 422.25,-77 422.25,-77 422.25,-77 422.25,-64.7 422.25,-64.7"/>
<polygon fill="#9aa5b1" stroke="#9aa5b1" points="424.7,-64.7 422.25,-57.7 419.8,-64.7 424.7,-64.7"/>
<text xml:space="preserve" text-anchor="middle" x="255.65" y="-78.95" font-family="Arial" font-size="9.00" fill="#616e7c">OAuth</text>
</g>
<!-- slack -->
<g id="node11" class="node">
<title>slack</title>
<path fill="#ffffff" stroke="#9aa5b1" d="M567.38,-56.11C567.38,-56.11 530.62,-56.11 530.62,-56.11 524.62,-56.11 518.62,-50.11 518.62,-44.11 518.62,-44.11 518.62,-32.11 518.62,-32.11 518.62,-26.11 524.62,-20.11 530.62,-20.11 530.62,-20.11 567.38,-20.11 567.38,-20.11 573.38,-20.11 579.38,-26.11 579.38,-32.11 579.38,-32.11 579.38,-44.11 579.38,-44.11 579.38,-50.11 573.38,-56.11 567.38,-56.11"/>
<text xml:space="preserve" text-anchor="middle" x="549" y="-34.61" font-family="Arial" font-size="10.00" fill="#1f2933">Slack</text>
</g>
<!-- veins&#45;&gt;slack -->
<g id="edge6" class="edge">
<title>veins&#45;&gt;slack</title>
<path fill="none" stroke="#9aa5b1" d="M137.5,-110.02C137.5,-101.6 137.5,-94 137.5,-94 137.5,-94 536.06,-94 536.06,-94 536.06,-94 536.06,-64.92 536.06,-64.92"/>
<polygon fill="#9aa5b1" stroke="#9aa5b1" points="538.51,-64.92 536.06,-57.92 533.61,-64.92 538.51,-64.92"/>
<text xml:space="preserve" text-anchor="middle" x="336.18" y="-95.95" font-family="Arial" font-size="9.00" fill="#616e7c">API</text>
</g>
<!-- shunts -->
<g id="node3" class="node">
<title>shunts</title>
<path fill="#fdeaea" stroke="#c0392b" d="M281.5,-154.38C281.5,-154.38 194.5,-154.38 194.5,-154.38 188.5,-154.38 182.5,-148.38 182.5,-142.38 182.5,-142.38 182.5,-122.22 182.5,-122.22 182.5,-116.22 188.5,-110.22 194.5,-110.22 194.5,-110.22 281.5,-110.22 281.5,-110.22 287.5,-110.22 293.5,-116.22 293.5,-122.22 293.5,-122.22 293.5,-142.38 293.5,-142.38 293.5,-148.38 287.5,-154.38 281.5,-154.38"/>
<text xml:space="preserve" text-anchor="middle" x="238" y="-134.8" font-family="Arial" font-size="10.00" fill="#1f2933">Shunts</text>
<text xml:space="preserve" text-anchor="middle" x="238" y="-122.8" font-family="Arial" font-size="10.00" fill="#1f2933">mock connectors</text>
</g>
<!-- templates -->
<g id="node6" class="node">
<title>templates</title>
<path fill="#e6f5ec" stroke="#1a7f45" d="M541.5,-154.38C541.5,-154.38 484.5,-154.38 484.5,-154.38 478.5,-154.38 472.5,-148.38 472.5,-142.38 472.5,-142.38 472.5,-122.22 472.5,-122.22 472.5,-116.22 478.5,-110.22 484.5,-110.22 484.5,-110.22 541.5,-110.22 541.5,-110.22 547.5,-110.22 553.5,-116.22 553.5,-122.22 553.5,-122.22 553.5,-142.38 553.5,-142.38 553.5,-148.38 547.5,-154.38 541.5,-154.38"/>
<text xml:space="preserve" text-anchor="middle" x="513" y="-134.8" font-family="Arial" font-size="10.00" fill="#1f2933">Templates</text>
<text xml:space="preserve" text-anchor="middle" x="513" y="-122.8" font-family="Arial" font-size="10.00" fill="#1f2933">patterns</text>
</g>
<!-- app_be -->
<g id="node13" class="node">
<title>app_be</title>
<path fill="#ffffff" stroke="#9aa5b1" d="M729.12,-56.11C729.12,-56.11 678.88,-56.11 678.88,-56.11 672.88,-56.11 666.88,-50.11 666.88,-44.11 666.88,-44.11 666.88,-32.11 666.88,-32.11 666.88,-26.11 672.88,-20.11 678.88,-20.11 678.88,-20.11 729.12,-20.11 729.12,-20.11 735.12,-20.11 741.12,-26.11 741.12,-32.11 741.12,-32.11 741.12,-44.11 741.12,-44.11 741.12,-50.11 735.12,-56.11 729.12,-56.11"/>
<text xml:space="preserve" text-anchor="middle" x="704" y="-34.61" font-family="Arial" font-size="10.00" fill="#1f2933">Backend</text>
</g>
<!-- tools&#45;&gt;app_be -->
<g id="edge8" class="edge">
<title>tools&#45;&gt;app_be</title>
<path fill="none" stroke="#9aa5b1" stroke-dasharray="5,2" d="M704,-109.98C704,-109.98 704,-64.99 704,-64.99"/>
<polygon fill="#9aa5b1" stroke="#9aa5b1" points="706.45,-64.99 704,-57.99 701.55,-64.99 706.45,-64.99"/>
<text xml:space="preserve" text-anchor="middle" x="696.88" y="-89.43" font-family="Arial" font-size="9.00" fill="#616e7c">test</text>
</g>
<!-- monitors -->
<g id="node8" class="node">
<title>monitors</title>
<path fill="#e8effd" stroke="#2b5fd9" d="M863.12,-154.38C863.12,-154.38 800.88,-154.38 800.88,-154.38 794.88,-154.38 788.88,-148.38 788.88,-142.38 788.88,-142.38 788.88,-122.22 788.88,-122.22 788.88,-116.22 794.88,-110.22 800.88,-110.22 800.88,-110.22 863.12,-110.22 863.12,-110.22 869.12,-110.22 875.12,-116.22 875.12,-122.22 875.12,-122.22 875.12,-142.38 875.12,-142.38 875.12,-148.38 869.12,-154.38 863.12,-154.38"/>
<text xml:space="preserve" text-anchor="middle" x="832" y="-134.8" font-family="Arial" font-size="10.00" fill="#1f2933">Monitors</text>
<text xml:space="preserve" text-anchor="middle" x="832" y="-122.8" font-family="Arial" font-size="10.00" fill="#1f2933">databrowse</text>
</g>
<!-- app_db -->
<g id="node14" class="node">
<title>app_db</title>
<path fill="#ffffff" stroke="#9aa5b1" d="M859,-56.2C859,-58.42 841.52,-60.22 820,-60.22 798.48,-60.22 781,-58.42 781,-56.2 781,-56.2 781,-20.02 781,-20.02 781,-17.8 798.48,-16 820,-16 841.52,-16 859,-17.8 859,-20.02 859,-20.02 859,-56.2 859,-56.2"/>
<path fill="none" stroke="#9aa5b1" d="M859,-56.2C859,-53.98 841.52,-52.18 820,-52.18 798.48,-52.18 781,-53.98 781,-56.2"/>
<text xml:space="preserve" text-anchor="middle" x="820" y="-34.61" font-family="Arial" font-size="10.00" fill="#1f2933">Database</text>
</g>
<!-- monitors&#45;&gt;app_db -->
<g id="edge9" class="edge">
<title>monitors&#45;&gt;app_db</title>
<path fill="none" stroke="#9aa5b1" stroke-dasharray="5,2" d="M823.94,-109.98C823.94,-109.98 823.94,-69.11 823.94,-69.11"/>
<polygon fill="#9aa5b1" stroke="#9aa5b1" points="826.39,-69.11 823.94,-62.11 821.49,-69.11 826.39,-69.11"/>
<text xml:space="preserve" text-anchor="middle" x="808.94" y="-91.49" font-family="Arial" font-size="9.00" fill="#616e7c">browse</text>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 18 KiB

View File

@@ -0,0 +1,38 @@
# graphgen
Interactive database-schema explorer. Supabase-style: a card per model, its
columns listed, foreign keys drawn between them, laid out in columns you can
drag.
```
/station/tools/graphgen/ the viewer
/station/tools/graphgen/api/schema what it draws
```
`schema.py` finds a schema and normalises it to one shape:
| source | |
|---|---|
| `schema.json` | in a `cfg/<room>/soleprint/station/tools/graphgen/` directory |
| a modelgen `schema/` folder | Python dataclasses, read through `modelgen.loader` |
## The published contract
`{models, relationships, source}` is **not an internal shape**. `modelgen` emits
it (`generator/jsonschema.py`), `datagen` exposes it (`base.py::schema`),
`shuntgen` generates it and `cfg/amar` reads it off disk;
`modelgen/tests/test_extractors.py:240,394` assert it. It does not change to
suit a consumer.
## Static diagrams are docgen's
This draws a schema *in the browser*, for exploring. For a rendered SVG — an ER
diagram in a document, a minimap, a site — the schema goes to `atlas2/docgen`,
which reads this same contract and emits from it:
```bash
python3 -m docgen.extractors db --schema schema.json -o ir.json
python3 -m docgen.emitters erd ir.json -o schema.svg
```
Two tools, one contract, and neither has to know about the other.

View File

@@ -1 +1,43 @@
"""Graphgen — interactive DB schema visualization."""
"""
Graphgen — what a graph is, and where graphs come from.
Two halves, and they are independent:
graph.py the model. Nodes, edges, groups, and the meaning carried on
them. No colour, no font, no layout engine.
schema.py the first source. A `schema.json` or a modelgen `schema/`
folder becomes `{models, relationships, source}`, which
`api.py` serves at /station/tools/graphgen/api/schema and the
browser viewer draws.
`{models, relationships, source}` is a **published contract**, not an internal
shape: modelgen emits it (`generator/jsonschema.py`), datagen exposes it
(`base.py::schema`), shuntgen generates it, and `cfg/amar` reads it off disk.
Two modelgen tests assert it. It does not change to suit anything here.
The senses of "graph" parked for later — video pipeline processing graphs, local
computer-vision graphs, Supabase-style schema diagrams — are more *sources*.
They belong beside `schema.py`, feeding the one model, rather than each growing
its own drawing code.
## Drawing one is not this tool's job
`docgen` takes a graph and produces DOT, SVG, and documents, with the styling
supplied as a data profile. The dependency runs one way: docgen reads this
model structurally and imports nothing from here, so either folder works with
the other absent. `docgen/shape.py` is that contract, written down.
from graphgen import Graph
from docgen import Profile, emit, render # the other half, if present
## Importing this is cheap
`api.py` is deliberately not imported here. Pulling in the model must not pull
in FastAPI — `run.py` imports `station.tools.graphgen.api` explicitly when it
mounts the router, and nothing else should have to pay for that.
"""
from .graph import CLASSES, Edge, Graph, Group, Node
__all__ = ["Graph", "Node", "Edge", "Group", "CLASSES"]