diff --git a/Makefile b/Makefile index 566fa29..472780a 100644 --- a/Makefile +++ b/Makefile @@ -42,7 +42,7 @@ $(eval $(ARGS):;@:) endif .DEFAULT_GOAL := help -.PHONY: help build start stop dist theme docs cluster deploy component +.PHONY: help build start stop dist theme ui docs cluster deploy component help: ## list targets @grep -hE '^[a-z]+:.*?##' $(MAKEFILE_LIST) | sed 's/:.*##/\t/' | expand -t16 @@ -66,6 +66,11 @@ dist: ## compile the plexus UIs to single files [ The target project receives the built files and commits them directly. No npm dependency on spr. +To give a project only the parts it uses — readable source, not the whole +library — slice it instead: + +```bash +make ui scan # what it uses, and how its copy compares with spr +./ctrl/ui.sh slice /ui.toml # write the slice its run file describes +``` + +See [Export / Compile](#export) for run files, the two deliveries and drift. + ## soleprint-modelgen Model generator. Reads schema definitions and produces model files. diff --git a/docs/data/en/export.md b/docs/data/en/export.md index 305d3d9..39f8817 100644 --- a/docs/data/en/export.md +++ b/docs/data/en/export.md @@ -112,6 +112,38 @@ build it first: cd soleprint/common/ui && pnpm build Each publish leaves a `.spr` stamp in the destination recording name, version, type, source and mode, so a copy can say where it came from. +### Slices: only what a consumer uses + +`publish` hands over the whole component. A **slice** hands over the part one +consumer actually uses, as readable source, computed rather than picked: + +```bash +make ui # list: every export and part, and what it pulls in +make ui scan ../mpr/ui/detection-app # what an app uses; its framework copy against spr +./ctrl/ui.sh slice path/to/ui.toml # write the slices a run file lists +./ctrl/ui.sh check path/to/ui.toml # the same, writing nothing — exit 1 if stale +``` + +A run file names the consumer, where the slice goes, and whether it is read from +the consumer's own imports (`scan`) or named (`use`). `target = "vue"` takes files +from `common/ui/src` — `Panel` alone is one file and `vue`; `TimeSeriesRenderer` +brings uplot and nothing else does. `target = "plain"` takes parts from +`common/theme/parts` as one `ui.css` and one `ui.js`. See +`soleprint/common/slice/ui.example.toml`. + +Two deliveries: + +| `delivery` | for | what changes | +| --- | --- | --- | +| `dependency` | your own projects | nothing: spr's names stay, and a `.spr` stamp says where the copy came from | +| `vendored` | a repository that is not spr's | names go through a mapping profile, headers are stripped, no stamp is written, a scrub fails on any spr word, and `notice` must be stated | + +Every run writes a manifest beside its run file recording the hash of each file +it wrote. That is what makes drift visible both ways: when spr moves on, `check` +says the slice is stale; when someone edits the copy, the next run names the file +and refuses to overwrite it without `--force` — so an improvement made in a +consumer can be brought back instead of lost. + ## Deploying ```bash diff --git a/soleprint/common/slice/.gitignore b/soleprint/common/slice/.gitignore new file mode 100644 index 0000000..0af916d --- /dev/null +++ b/soleprint/common/slice/.gitignore @@ -0,0 +1,7 @@ +# Everything a run writes, and a real run file: it names a consumer's paths and +# belongs beside that consumer. ui.example.toml is the committed one. +out/ +ui.toml +*.slice.json +names/names.local.json +__pycache__/ diff --git a/soleprint/common/slice/__init__.py b/soleprint/common/slice/__init__.py new file mode 100644 index 0000000..c340364 --- /dev/null +++ b/soleprint/common/slice/__init__.py @@ -0,0 +1,40 @@ +""" +Slices: the part of spr's UI one consumer uses, generated as readable source. + + python3 -m common.slice (from soleprint/; or ./ctrl/ui.sh, make ui) + +A **slice** is the closure of what a consumer actually uses, cut out of spr and +written into that consumer: + + target = "vue" from common/ui/src: the .vue/.ts files the used exports + need, a barrel that exports only those, and tokens.css + with only the variables they (and the consumer) read. + target = "plain" from common/theme/parts: ui.css + ui.js carrying only the + parts the consumer's markup uses, with resolved tokens. + +It is never a bundle. The first attempt at reuse was one artifact for everyone, +and it failed the way bundles do: every consumer carried every part. A slice is +the per-part owned copy that common/theme/bake.py already makes for pages, +scaled up to a whole consumer. The selection is computed from the import graph +and from the consumer's own imports and markup, not typed by hand, and a check +reports drift in both directions: spr moved on, or the copy was edited. + +## Two deliveries, in the words of the shipping model + + delivery = "dependency" own projects. spr's names stay; a `.spr` stamp in + the ctrl/spr.py format says where the copy came from. + delivery = "vendored" a slice renamed to corp terms through a mapping + profile (names/corp.json), headers stripped, no + stamp, and a scrub that fails on any word that + would identify spr. A `notice` must be stated — + this tool never guesses the licensing. + +Both write a manifest beside the run file: selection, source commit and the hash +of every file written. For vendored slices that file stays with the run file in +a private place, and only the neutral files land in the consumer's repo. + +## What lives where + +Library modules here hold no command-line code; `cli/` holds all of it (the +same split docgen uses). The generator itself never ships — only its output. +""" diff --git a/soleprint/common/slice/__main__.py b/soleprint/common/slice/__main__.py new file mode 100644 index 0000000..f0bc986 --- /dev/null +++ b/soleprint/common/slice/__main__.py @@ -0,0 +1,8 @@ +""" python3 -m common.slice [args] — see cli/__init__.py for the commands.""" + +import sys + +from .cli import main + +if __name__ == "__main__": + sys.exit(main()) diff --git a/soleprint/common/slice/cli/__init__.py b/soleprint/common/slice/cli/__init__.py new file mode 100644 index 0000000..6d5c035 --- /dev/null +++ b/soleprint/common/slice/cli/__init__.py @@ -0,0 +1,58 @@ +""" +The command line — every command slices have, and nothing else. + + python3 -m common.slice [args] (from soleprint/) + ./ctrl/ui.sh [args] (from anywhere) + make ui + + list what can be sliced: every export and part, with what it pulls in + scan what a consumer uses, and how its framework copy compares with spr + slice write every slice a run file lists (only what is stale) + check the same, writing nothing: exit 1 if anything is stale + selftest this package's own suite + +Library modules beside this package hold no command-line code; selftest asserts +that nothing outside cli/ imports argparse, the same rule docgen keeps. +""" + +import sys + +# command -> (module in this package, one-line description) +COMMANDS = { + "list": ("list", "what can be sliced, and what each piece pulls in"), + "scan": ("scan", "what a consumer uses, and its copy against spr"), + "slice": ("slice", "write every slice a run file lists"), + "check": ("slice", "the same, writing nothing — exit 1 if stale"), + "selftest": ("selftest", "this package's own suite"), +} + + +def usage() -> str: + from .common import PROG + + lines = [f"usage: {PROG} [args]", ""] + lines += [f" {name:<9} {desc}" for name, (_, desc) in COMMANDS.items()] + lines += ["", f" {PROG} --help for a command's options"] + return "\n".join(lines) + + +def main(argv=None) -> int: + from importlib import import_module + + from .common import Abort + + argv = sys.argv[1:] if argv is None else list(argv) + if not argv or argv[0] in ("-h", "--help", "help"): + print(usage()) + return 0 if argv else 2 + name, rest = argv[0], argv[1:] + if name not in COMMANDS: + print(f"Error: no command {name!r}\n\n{usage()}", file=sys.stderr) + return 2 + module = import_module(f".{COMMANDS[name][0]}", package=__name__) + try: + return module.main(rest, command=name) + except Abort as e: + # The only exit path for an expected failure: one line, stderr, exit 1. + print(f"Error: {e}", file=sys.stderr) + return 1 diff --git a/soleprint/common/slice/cli/common.py b/soleprint/common/slice/cli/common.py new file mode 100644 index 0000000..cbc02c8 --- /dev/null +++ b/soleprint/common/slice/cli/common.py @@ -0,0 +1,25 @@ +"""What every command does the same way — written once.""" + +from pathlib import Path + +ROOT = __package__.rsplit(".", 1)[0] +PROG = f"python3 -m {ROOT}" + + +class Abort(Exception): + """An expected failure. Printed as one line, exit 1 — never a traceback.""" + + +def kb(n: int) -> str: + return f"{n / 1024:.1f} KB" + + +def load_runfile(path: Path): + from .. import runfile, tokens + + if not path.exists(): + raise Abort(f"no run file at {path} — see soleprint/common/slice/ui.example.toml") + try: + return runfile.load(path, tokens.available_themes()) + except runfile.ConfigError as e: + raise Abort(str(e)) from None diff --git a/soleprint/common/slice/cli/list.py b/soleprint/common/slice/cli/list.py new file mode 100644 index 0000000..6de5eda --- /dev/null +++ b/soleprint/common/slice/cli/list.py @@ -0,0 +1,81 @@ +""" + python3 -m common.slice list every export and part, and what it pulls in + python3 -m common.slice list --json the same, as data (the derived half of a manifest) + +The point is the right-hand columns: what taking one piece costs. `Panel` is one +file and `vue`; `TimeSeriesRenderer` brings uplot; `maximize` brings `panel`. +""" + +import argparse +import json + +from ...theme import bake +from .. import graph +from ..slices import _needs +from .common import PROG, kb + + +def exports_table() -> list[dict]: + nodes = graph.load() + rows = [] + for e in graph.barrel(nodes=nodes): + closure, _ = graph.select([e.name], [], graph.barrel(nodes=nodes), nodes) + rows.append({ + "name": e.name, + "kind": "type" if e.is_type else "value", + "file": e.file, + "depends_on": sorted(closure - {e.file}), + "packages": graph.dependencies(closure, nodes), + "tokens": sorted(graph.tokens_read(closure, nodes)), + "bytes": sum(nodes[f].size for f in closure), + }) + return rows + + +def parts_table() -> list[dict]: + parts = bake.load_parts() + rows = [] + for name, (css, classes, detects, always) in sorted(parts.items()): + js = (bake.PARTS / f"{name}.js") + js_text = js.read_text() if js.exists() else "" + rows.append({ + "name": name, + "always": always, + "depends_on": _needs(name, parts), + "exposes": {"classes": sorted(classes), "detect": sorted(detects)}, + "tokens": sorted(set(bake.re.findall(r"var\(\s*(--[\w-]+)", css + js_text))), + "bytes": len(css.encode()) + len(js_text.encode()), + }) + return rows + + +def main(argv, command=None) -> int: + p = argparse.ArgumentParser(prog=f"{PROG} list") + p.add_argument("--json", action="store_true", help="Print the tables as JSON.") + args = p.parse_args(argv) + + exports, parts = exports_table(), parts_table() + if args.json: + print(json.dumps({"vue": exports, "plain": parts}, indent=2)) + return 0 + + print(f"target = \"vue\" — common/ui/src ({len(exports)} exports)\n") + print(f" {'export':<22} {'file':<34} {'pulls in':<30} {'packages':<22} size") + for r in exports: + if r["kind"] == "type": + continue + pulls = ", ".join(d.rsplit("/", 1)[-1] for d in r["depends_on"]) or "-" + pkgs = ", ".join(r["packages"]) or "-" + print(f" {r['name']:<22} {r['file']:<34} {pulls:<30} {pkgs:<22} {kb(r['bytes'])}") + types = [r["name"] for r in exports if r["kind"] == "type"] + print(f"\n types ride with their file: {', '.join(types)}") + + print(f"\ntarget = \"plain\" — common/theme/parts ({len(parts)} parts)\n") + print(f" {'part':<10} {'needs':<10} {'found by':<40} size") + for r in parts: + found = "always, with any other part" if r["always"] else \ + ", ".join(["." + c for c in sorted(r["exposes"]["classes"], + key=lambda c: (not c.startswith(r["name"]), c))[:3]] + + r["exposes"]["detect"]) + print(f" {r['name']:<10} {', '.join(r['depends_on']) or '-':<10} {found[:40]:<40} {kb(r['bytes'])}") + return 0 diff --git a/soleprint/common/slice/cli/scan.py b/soleprint/common/slice/cli/scan.py new file mode 100644 index 0000000..e363d4b --- /dev/null +++ b/soleprint/common/slice/cli/scan.py @@ -0,0 +1,103 @@ +""" + python3 -m common.slice scan [ …] + python3 -m common.slice scan --copy + python3 -m common.slice scan --json + +What an app uses of the framework, what a slice of it would hold, and — when the +app keeps a copy of the framework — how that copy compares with spr. Reads only; +nothing is written anywhere. +""" + +import argparse +import json +from pathlib import Path + +from .. import graph +from .. import scan as scanning +from .common import PROG, Abort + + +def report(app: Path, aliases, copy: Path | None) -> dict: + nodes = graph.load() + copy = copy or scanning.find_copy(app) + usage = scanning.usage([app], aliases, exclude=[copy] if copy else [], nodes=nodes) + exports = graph.barrel(nodes=nodes) + files = sorted(usage.files - graph.GENERATED - {"base.css"}) + closure, unknown = graph.select(sorted(usage.names), files, exports, nodes) + out = { + "app": str(app), + "imports": {"barrel": usage.barrel, "deep": usage.deep}, + "names": sorted(usage.names), + "files": sorted(usage.files), + "unknown": sorted(set(unknown) | usage.unknown), + "closure": sorted(closure), + "packages": graph.dependencies(closure, nodes), + "own_tokens": len(usage.tokens), + "copy": None, + } + if copy: + state = scanning.compare(copy) + out["copy"] = { + "path": str(copy), + "stamp": state.stamp, + "stamp_commit_exists": state.stamp_known, + "rows": [{"file": r, "status": s, "detail": d} for r, s, d in state.rows], + } + return out + + +def show(r: dict) -> None: + print(r["app"]) + print(f" imports {r['imports']['deep']} by deep path, {r['imports']['barrel']} through the barrel") + if r["names"]: + print(f" names {', '.join(r['names'])}") + if r["files"]: + print(f" files {', '.join(r['files'])}") + if r["unknown"]: + print(f" UNKNOWN {', '.join(r['unknown'])} (spr has no such export or file)") + pkgs = ", ".join(f"{k} {v}" for k, v in r["packages"].items()) or "none" + print(f" a slice {len(r['closure'])} file(s): {', '.join(r['closure']) or '-'}") + print(f" packages: {pkgs}; {r['own_tokens']} custom properties read by the app itself") + c = r["copy"] + if not c: + print(" copy none found (no `link:` in package.json, no @framework alias)") + return + stamp = c["stamp"] + if stamp: + known = "exists in spr" if c["stamp_commit_exists"] else \ + "NOT in spr's history — compared with current spr instead" + print(f" copy {c['path']} (.spr: {stamp.get('mode', '?')} at {stamp.get('sha', '?')}, {known})") + else: + print(f" copy {c['path']} (no .spr stamp)") + rows = [x for x in c["rows"] if x["status"] != "same"] + same = len(c["rows"]) - len(rows) + for x in rows: + print(f" {x['status']:<10} {x['file']:<32} {x['detail']}") + print(f" {same} file(s) identical to spr") + + +def main(argv, command=None) -> int: + p = argparse.ArgumentParser(prog=f"{PROG} scan") + p.add_argument("apps", type=Path, nargs="+", help="App directories to read.") + p.add_argument("--alias", action="append", default=None, + help="An import specifier that means the framework " + "(default: soleprint-ui, @framework). Repeatable.") + p.add_argument("--copy", type=Path, default=None, + help="The app's framework copy, when it cannot be found automatically.") + p.add_argument("--json", action="store_true", help="Print the report as JSON.") + args = p.parse_args(argv) + + aliases = tuple(args.alias) if args.alias else scanning.DEFAULT_ALIASES + reports = [] + for app in args.apps: + if not app.is_dir(): + raise Abort(f"{app} is not a directory") + reports.append(report(app.resolve(), aliases, args.copy.resolve() if args.copy else None)) + if args.json: + print(json.dumps(reports if len(reports) > 1 else reports[0], indent=2)) + return 0 + for i, r in enumerate(reports): + if i: + print() + show(r) + return 0 diff --git a/soleprint/common/slice/cli/selftest.py b/soleprint/common/slice/cli/selftest.py new file mode 100644 index 0000000..4aee670 --- /dev/null +++ b/soleprint/common/slice/cli/selftest.py @@ -0,0 +1,11 @@ +""" python3 -m common.slice selftest — runs selftest.py in a fresh interpreter.""" + +import subprocess +import sys +from pathlib import Path + +SOLEPRINT = Path(__file__).resolve().parents[3] + + +def main(argv, command=None) -> int: + return subprocess.call([sys.executable, "-m", "common.slice.selftest", *argv], cwd=SOLEPRINT) diff --git a/soleprint/common/slice/cli/slice.py b/soleprint/common/slice/cli/slice.py new file mode 100644 index 0000000..27ea1af --- /dev/null +++ b/soleprint/common/slice/cli/slice.py @@ -0,0 +1,106 @@ +""" + python3 -m common.slice slice [--only NAME] [--list] [--force] + python3 -m common.slice check [--only NAME] + +`slice` writes what is stale and leaves the rest alone; `check` says what +`slice` would do and exits 1 if that is anything at all. `--list` shows every +slice fully resolved — what it selects, what that pulls in, where it goes — +and writes nothing. +""" + +import argparse +from pathlib import Path + +from .. import slices +from .common import PROG, Abort, kb, load_runfile + + +def describe(built) -> list[str]: + e = built.entry + lines = [f" {e.line()}"] + if built.selected: + lines.append(f" selected {', '.join(built.selected)}") + lines.append(f" closure {', '.join(built.closure) or '(nothing)'}") + if e.target == "vue": + pkgs = ", ".join(f"{k} {v}" for k, v in built.deps.items()) or "none" + lines.append(f" packages {pkgs}") + source = "library" if e.theme in (None, "library") else f"theme {e.theme}" + lines.append(f" tokens {len(built.carried)} carried, from {source}" + + (f"; fell back to the library for {', '.join(built.fell_back)}" + if built.fell_back else "")) + if built.missing: + lines.append(f" WARNING read but defined nowhere: {', '.join(built.missing)}") + size = sum(len(t.encode()) for t in built.files.values()) + lines.append(f" files {len(built.files)}, {kb(size)}") + for problem in built.blocked: + lines.append(f" BLOCKED {problem}") + for rel, line, term, text in built.hits[:20]: + lines.append(f" {rel}:{line} {term!r} {text[:80]}") + if len(built.hits) > 20: + lines.append(f" … and {len(built.hits) - 20} more") + return lines + + +def main(argv, command="slice") -> int: + p = argparse.ArgumentParser(prog=f"{PROG} {command}") + p.add_argument("runfile", type=Path, help="The run file (see ui.example.toml).") + p.add_argument("--only", action="append", default=[], metavar="NAME", + help="Just this slice. Repeatable.") + if command == "slice": + p.add_argument("--list", action="store_true", help="Resolve and show; write nothing.") + p.add_argument("--check", action="store_true", help="Same as the `check` command.") + p.add_argument("--force", action="store_true", + help="Overwrite files edited by hand in dest. Never skips the scrub.") + args = p.parse_args(argv) + checking = command == "check" or getattr(args, "check", False) + listing = getattr(args, "list", False) + + rf = load_runfile(args.runfile) + try: + chosen = rf.select(args.only) + except ValueError as e: + raise Abort(str(e)) from None + + mode = " (check: nothing written)" if checking else " (list: nothing written)" if listing else "" + print(f"{rf.path} — {len(chosen)} slice(s){mode}") + failed = 0 + for entry in chosen: + built = slices.build(entry) + steps = slices.plan(entry, built) + print() + print("\n".join(describe(built))) + counts = {k: len(steps.of(k)) for k in ("create", "update", "same", "delete", "conflict")} + print(" plan " + ", ".join(f"{k} {v}" for k, v in counts.items())) + for rel, action, detail in steps.actions: + if action != "same" and (listing or checking or action == "conflict"): + print(f" {action:<9} {rel}" + (f" — {detail}" if detail else "")) + if built.blocked: + failed += 1 + continue + if listing: + continue + if checking: + if steps.stale: + failed += 1 + print(" STALE run `slice` to bring it up to date") + else: + print(" ok up to date") + continue + try: + changed = slices.apply(entry, built, steps, args.force) + except ValueError as e: + failed += 1 + print(f" REFUSED {e}") + continue + if changed: + for line in changed: + print(f" {line}") + else: + print(" ok up to date, nothing written") + + print() + if failed: + print(f"{failed} of {len(chosen)} slice(s) did not hold") + return 1 + print(f"{len(chosen)} slice(s) " + ("current" if checking else "resolved" if listing else "done")) + return 0 diff --git a/soleprint/common/slice/graph.py b/soleprint/common/slice/graph.py new file mode 100644 index 0000000..ab5958d --- /dev/null +++ b/soleprint/common/slice/graph.py @@ -0,0 +1,201 @@ +""" +The import graph of common/ui/src, and the closure of what a selection needs. + +Read out of the source with patterns, not a TypeScript parser: the library is +thirty-odd small files, every import is on a line of its own, and a parser would +be a node dependency for a question grep answers. The selftest pins the edges it +must find, so a pattern that stops matching fails loudly. + +Three kinds of edge matter, and all three are followed: + + import { x } from './a' code, including `import type` — a type-only + import still has to resolve for vue-tsc + import './theme.css' side effects: stylesheets + export { default as X } from './b' the barrel's own re-exports + +A bare specifier (`vue`, `uplot/dist/uPlot.min.css`) is a package, not a file. +It becomes a dependency of the slice, with the version common/ui/package.json +pins for it. +""" + +import json +import re +from dataclasses import dataclass, field +from pathlib import Path + +HERE = Path(__file__).resolve().parent +COMMON = HERE.parent +SRC = COMMON / "ui" / "src" +PACKAGE_JSON = COMMON / "ui" / "package.json" + +# The files a slice regenerates rather than copies. The barrel is rebuilt to +# export only the selection; the token file is cut down to what is read. +GENERATED = {"index.ts", "theme.css", "tokens.css"} + +# Source that never belongs in a delivery. +SKIP = {"__tests__"} + +_STRING = r"""['"]([^'"]+)['"]""" +IMPORT_FROM = re.compile(r"(?:^|\n)\s*import\s+(?:type\s+)?[^'\";]*?\s*from\s*" + _STRING, re.S) +IMPORT_BARE = re.compile(r"(?:^|\n)\s*import\s*" + _STRING) +EXPORT_FROM = re.compile( + r"(?:^|\n)\s*export\s+(type\s+)?\{([^}]*)\}\s*from\s*" + _STRING, re.S) +DYNAMIC = re.compile(r"import\(\s*" + _STRING + r"\s*\)") +CSS_IMPORT = re.compile(r"@import\s+(?:url\()?\s*" + _STRING) +VAR = re.compile(r"var\(\s*(--[\w-]+)") + +SCRIPT = re.compile(r"]*>(.*?)", re.S) +STYLE = re.compile(r"]*>(.*?)", re.S) + + +@dataclass +class Node: + """One file of the library.""" + + path: str # relative to SRC, posix + imports: set = field(default_factory=set) # other nodes, relative to SRC + packages: set = field(default_factory=set) # bare specifiers, as written + tokens: set = field(default_factory=set) # custom properties it reads + size: int = 0 + + +@dataclass +class Export: + """One name the barrel exports.""" + + name: str # what a consumer imports + local: str # what the file exports it as (`default` for SFCs) + file: str # relative to SRC + is_type: bool + + +def code_of(path: Path, text: str) -> tuple[str, str]: + """(script, style) text of a file. A .vue file is split into its blocks.""" + if path.suffix == ".vue": + return ("\n".join(SCRIPT.findall(text)), "\n".join(STYLE.findall(text))) + if path.suffix == ".css": + return "", text + return text, "" + + +def package_of(specifier: str) -> str: + """`@vue-flow/core/dist/style.css` -> `@vue-flow/core`; `uplot` -> `uplot`.""" + parts = specifier.split("/") + return "/".join(parts[:2]) if specifier.startswith("@") else parts[0] + + +def _resolve(importer: str, specifier: str, known: set) -> str | None: + """A relative specifier -> the SRC-relative file it names, or None.""" + base = (Path(importer).parent / specifier).as_posix() + base = str(Path(base)) # normalise ./ and ../ lexically + parts = [] + for part in base.split("/"): + if part == "..": + if parts: + parts.pop() + elif part not in ("", "."): + parts.append(part) + base = "/".join(parts) + for candidate in (base, base + ".ts", base + ".vue", base + "/index.ts"): + if candidate in known: + return candidate + return None + + +def load(src: Path = SRC) -> dict[str, Node]: + """Every file under src, with its edges.""" + files = { + p.relative_to(src).as_posix(): p + for p in sorted(src.rglob("*")) + if p.is_file() and p.suffix in (".ts", ".vue", ".css") + and not SKIP & set(p.relative_to(src).parts) + } + known = set(files) + nodes = {} + for rel, path in files.items(): + text = path.read_text() + script, style = code_of(path, text) + node = Node(path=rel, size=len(text.encode())) + specifiers = (IMPORT_FROM.findall(script) + IMPORT_BARE.findall(script) + + [m[2] for m in EXPORT_FROM.findall(script)] + + DYNAMIC.findall(script) + CSS_IMPORT.findall(style)) + for spec in specifiers: + if spec.startswith("."): + target = _resolve(rel, spec, known) + if target: + node.imports.add(target) + else: + node.packages.add(spec) + if path.suffix == ".vue": + # An SFC compiles to imports from `vue` even when its script names + # none (Panel.vue uses only compiler macros), so it always needs vue. + node.packages.add("vue") + node.tokens = set(VAR.findall(style if path.suffix == ".vue" else text)) + nodes[rel] = node + return nodes + + +def barrel(src: Path = SRC, nodes: dict | None = None) -> list[Export]: + """What index.ts exports, name by name, in its own order.""" + nodes = nodes if nodes is not None else load(src) + text = (src / "index.ts").read_text() + out = [] + for block_type, names, spec in EXPORT_FROM.findall(text): + target = _resolve("index.ts", spec, set(nodes)) + if target is None: + continue + for item in names.split(","): + item = item.strip() + if not item: + continue + item_type = item.startswith("type ") + item = item[5:].strip() if item_type else item + local, _, alias = item.partition(" as ") + local, alias = local.strip(), alias.strip() + out.append(Export(name=alias or local, local=local, file=target, + is_type=bool(block_type) or item_type)) + return out + + +def closure(start: set, nodes: dict[str, Node]) -> set: + """Every file reachable from `start` through relative imports.""" + seen, stack = set(), list(start) + while stack: + rel = stack.pop() + if rel in seen or rel not in nodes: + continue + seen.add(rel) + stack.extend(nodes[rel].imports - seen) + return seen + + +def select(names: list[str], files: list[str], exports: list[Export], + nodes: dict[str, Node]) -> tuple[set, list[str]]: + """Export names and/or SRC-relative files -> (closure, unknown names).""" + by_name = {e.name: e for e in exports} + start, unknown = set(files), [] + for name in names: + if name in by_name: + start.add(by_name[name].file) + elif name in nodes: + start.add(name) + else: + unknown.append(name) + unknown += [f for f in files if f not in nodes] + # The library's own theme files are regenerated, never selected into a copy. + return closure(start - GENERATED, nodes) - GENERATED, unknown + + +def dependencies(files: set, nodes: dict[str, Node], + package_json: Path = PACKAGE_JSON) -> dict[str, str]: + """The packages a set of files imports, with the version common/ui pins.""" + pins = {} + if package_json.exists(): + data = json.loads(package_json.read_text()) + pins = {**data.get("devDependencies", {}), **data.get("dependencies", {})} + wanted = sorted({package_of(s) for f in files for s in nodes[f].packages}) + return {name: pins.get(name, "*") for name in wanted} + + +def tokens_read(files: set, nodes: dict[str, Node]) -> set: + return {t for f in files for t in nodes[f].tokens} diff --git a/soleprint/common/slice/manifest.py b/soleprint/common/slice/manifest.py new file mode 100644 index 0000000..0b05362 --- /dev/null +++ b/soleprint/common/slice/manifest.py @@ -0,0 +1,97 @@ +""" +The manifest a slice leaves beside its run file, and the `.spr` stamp. + +The manifest is what makes a copy checkable. It records, for every file written, +the hash of what was written — so the next run can tell the two kinds of drift +apart: + + source moved on the file in dest still matches the manifest, but spr + would now write something else: stale, rewrite it. + edited by hand the file in dest no longer matches the manifest: someone + changed the copy. Refused unless --force, and named, so a + real improvement can be brought back to spr instead of + being overwritten. + +A first run with files already in dest has no manifest to trust, so any file +that differs from what would be written counts as edited. That is the case of +adopting an existing copy — mts's framework copy was ahead of spr, and a tool +that overwrote it would have destroyed the only copy of that work. + +It carries no timestamp. The same inputs write the same bytes, so a check can +compare files rather than dates. The `.spr` stamp is the exception on purpose: +it keeps ctrl/spr.py's format (with `updated=`), and it is written only when +the slice actually changed. +""" + +import hashlib +import json +import subprocess +from datetime import datetime, timezone +from pathlib import Path + +from .graph import COMMON + +REPO = COMMON.parent.parent # the spr checkout +STAMP = ".spr" + + +def sha256(text: str) -> str: + return hashlib.sha256(text.encode()).hexdigest() + + +def file_sha(path: Path) -> str | None: + try: + return hashlib.sha256(path.read_bytes()).hexdigest() + except OSError: + return None + + +def git(*args) -> str: + try: + out = subprocess.run(["git", "-C", str(REPO), *args], capture_output=True, text=True) + except FileNotFoundError: + return "" + return out.stdout.strip() if out.returncode == 0 else "" + + +def source_state(paths) -> dict: + """The spr commit a slice was cut from, and whether its inputs were dirty.""" + rel = [str(Path(p).resolve().relative_to(REPO)) for p in paths if Path(p).exists()] + dirty = bool(git("status", "--porcelain", "--", *rel)) if rel else False + return {"commit": git("rev-parse", "HEAD") or "unknown", "dirty": dirty} + + +def commit_exists(sha: str) -> bool: + return bool(sha) and bool(git("cat-file", "-t", f"{sha}^{{commit}}")) + + +def read(path: Path) -> dict | None: + try: + return json.loads(path.read_text()) + except (OSError, json.JSONDecodeError): + return None + + +def write(path: Path, data: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(json.dumps(data, indent=2) + "\n") + + +def stamp(name: str, kind: str, version: str, source: Path) -> str: + """ctrl/spr.py's stamp, so a dependency slice says where it came from.""" + sha = git("rev-parse", "--short", "HEAD") or "unknown" + return (f"name={name}\nversion={version}\ntype={kind}\nsha={sha}\nmode=slice\n" + f"updated={datetime.now(timezone.utc).strftime('%Y-%m-%dT%H:%M:%SZ')}\n" + f"source={source}\n") + + +def read_stamp(directory: Path) -> dict: + path = directory / STAMP + if not path.exists(): + return {} + out = {} + for line in path.read_text().splitlines(): + key, sep, value = line.partition("=") + if sep: + out[key.strip()] = value.strip() + return out diff --git a/soleprint/common/slice/names.py b/soleprint/common/slice/names.py new file mode 100644 index 0000000..e1fd9ef --- /dev/null +++ b/soleprint/common/slice/names.py @@ -0,0 +1,127 @@ +""" +The mapping profile a vendored slice is written through, and the scrub after it. + +Three passes, in this order, and the order matters: + +1. **Identifiers**, everywhere — code and comments. `SprFeed` becomes whatever + the profile says in the page that calls it and in the file that defines it, + or the rename breaks the code. This goes first so that a longer identifier + (`soleprint-ui`) is replaced whole before a prose term inside it is seen. +2. **Prose terms**, only inside comments, case kept (vein -> connector, Vein -> + Connector). Never in code: a variable that happens to be called `vein` is + an identifier, and renaming it silently is how a working slice breaks. +3. **Scrub**: every `deny` term, case-insensitive, whole words only — so + `sprint` passes and `spr` fails — plus any camelCase identifier that still + starts with `spr`. A hit fails the slice; nothing is written. + +The profile is names/corp.json, generic by design, extended by a gitignored +names.local.json beside it and by a run file's own `rename`. +""" + +import json +import re +from dataclasses import dataclass, field +from pathlib import Path + +HERE = Path(__file__).resolve().parent +PROFILE = HERE / "names" / "corp.json" +LOCAL = HERE / "names" / "names.local.json" + +WORD = r"[A-Za-z0-9_$]" +# A camelCase identifier that still carries spr's prefix: sprThing, SprThing. +# Lower case after `spr` is an English word (sprint, spread), not a leak. +SPR_PREFIX = re.compile(r"(?" # HTML and .vue template comments + r"|(? dict: + """Renamed -> original, to read a vendored page back in spr's words.""" + return {v: k for k, v in self.identifiers.items()} + + +def load(renames: dict | None = None) -> Profile: + """corp.json, then names.local.json, then the run file's own renames.""" + data = json.loads(PROFILE.read_text()) + profile = Profile(dict(data.get("identifiers", {})), dict(data.get("prose", {})), + list(data.get("deny", []))) + if LOCAL.exists(): + local = json.loads(LOCAL.read_text()) + profile.identifiers.update(local.get("identifiers", {})) + profile.prose.update(local.get("prose", {})) + profile.deny += [d for d in local.get("deny", []) if d not in profile.deny] + profile.identifiers.update(renames or {}) + return profile + + +def _bounded(term: str) -> re.Pattern: + edge = r"[A-Za-z0-9_$-]" if "-" in term else WORD + return re.compile(rf"(? str: + if match.isupper() and len(match) > 1: + return replacement.upper() + if match[:1].isupper(): + return replacement[:1].upper() + replacement[1:] + return replacement + + +def rename(text: str, profile: Profile) -> str: + # Longest first, so `soleprint-ui` goes before any `soleprint` could. + for old in sorted(profile.identifiers, key=len, reverse=True): + text = _bounded(old).sub(profile.identifiers[old], text) + + prose = sorted(profile.prose, key=len, reverse=True) + patterns = [(re.compile(rf"(? str: + chunk = m.group(0) + for pattern, replacement in patterns: + chunk = pattern.sub(lambda w: _cased(w.group(0), replacement), chunk) + return chunk + + return COMMENTS.sub(in_comment, text) + + +def scrub(text: str, profile: Profile) -> list[tuple[int, str, str]]: + """Every surviving term: (line number, term, the line).""" + hits = [] + lines = text.splitlines() + patterns = [(t, re.compile(rf"(? str: + """The file without its leading comment — the provenance a vendored copy drops. + + Only the first comment, and only when it is the first thing in the file: + documentation further down is the code's, and stays (through `rename`). + """ + if suffix in (".css", ".ts", ".js"): + stripped = re.sub(r"\A\s*/\*.*?\*/[ \t]*\n?", "", text, count=1, flags=re.S) + if stripped == text: + stripped = re.sub(r"\A(?:\s*//[^\n]*\n)+", "", text, count=1) + return stripped.lstrip("\n") + if suffix in (".vue", ".html"): + return re.sub(r"\A\s*[ \t]*\n?", "", text, count=1, flags=re.S) + return text diff --git a/soleprint/common/slice/names/corp.json b/soleprint/common/slice/names/corp.json new file mode 100644 index 0000000..05d9427 --- /dev/null +++ b/soleprint/common/slice/names/corp.json @@ -0,0 +1,25 @@ +{ + "_comment": "The mapping profile a vendored slice is written through. Generic on purpose: nothing here names a client. Identifiers are renamed everywhere (code and comments), prose terms only inside comments, and after both the scrub must find none of `deny`. Extend or override per machine in names.local.json beside this file (gitignored), and per slice with the run file's `rename`. When spr's naming index lands, it becomes the source of these tables.", + "identifiers": { + "SprFeed": "LiveFeed", + "sprParams": "renderParams", + "soleprint-ui": "ui-kit" + }, + "prose": { + "veins": "connectors", + "vein": "connector", + "shunts": "simulators", + "shunt": "simulator", + "cabinets": "backing services", + "cabinet": "backing service", + "plexuses": "single-file builds", + "plexus": "single-file build", + "soleprint": "the toolkit" + }, + "deny": [ + "soleprint", "spr", "vein", "veins", "shunt", "shunts", "plexus", "plexuses", + "larder", "artery", "mcrn", "mpr", "mts", "meetus", "doocus", "unt", "nvi", "amar", + "common/theme", "common/ui", "common/slice", "bake.py", "theme:here", "theme:parts", + "theme:baked-defaults" + ] +} diff --git a/soleprint/common/slice/runfile.py b/soleprint/common/slice/runfile.py new file mode 100644 index 0000000..723c0ef --- /dev/null +++ b/soleprint/common/slice/runfile.py @@ -0,0 +1,287 @@ +""" +A run file: every slice (and plexus) one consumer takes from spr, and how. + + python3 -m common.slice slice ui.toml # write what is stale + python3 -m common.slice slice ui.toml --list # resolved; writes nothing + python3 -m common.slice check ui.toml # exit 1 if anything is stale + +This module only loads and validates. Same three rules as docgen's and theme's +run files, so spr has one convention for "rebuild these, with these settings": + +- **Paths are relative to the run file**, not to where the command runs. +- **An entry's value replaces the default**, for every key. +- **Unknown keys are refused**, and every problem is reported in one pass. + +## The shape + + [defaults] + delivery = "dependency" + + [[slice]] + name = "app-ui" + target = "vue" # or "plain" + dest = "../framework" # where the files land + scan = ["src"] # read the consumer's own imports / markup + use = ["Panel"] # and/or name what to take + delivery = "vendored" # needs `notice` + notice = "none" # or a path: copied into dest as-is + +## Where it lives + +Beside the consumer, never in spr: it names the consumer's paths. For a vendored +slice keep it — and the manifest it writes — outside the consumer's repository, +with `dest` pointing in. `ui.example.toml` beside this file is the committed one. +""" + +import os +import re +from dataclasses import dataclass, field +from pathlib import Path + +try: + import tomllib +except ModuleNotFoundError: # pragma: no cover - Python < 3.11 + tomllib = None + +TOP_KEYS = {"defaults", "slice", "plexus"} +SLICE_KEYS = {"name", "target", "delivery", "layout", "dest", "use", "scan", "alias", "theme", + "tokens", "package", "notice", "header", "manifest", "rename"} +DEFAULT_KEYS = SLICE_KEYS - {"name", "dest", "manifest"} +TARGETS = ("vue", "plain") +DELIVERIES = ("dependency", "vendored") +LAYOUTS = ("package", "flat") +VUE_ONLY = {"layout", "alias", "package"} + +NAME = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]*$") +TOKEN = re.compile(r"^--[A-Za-z0-9_-]+$") + + +class ConfigError(ValueError): + """A run file that cannot be run. Carries every problem, not the first.""" + + def __init__(self, path, problems: list[str]): + self.path, self.problems = path, problems + super().__init__(f"{path}: {len(problems)} problem(s)\n " + "\n ".join(problems)) + + +@dataclass +class Slice: + """One slice, fully resolved — nothing relative, nothing defaulted later.""" + + name: str + target: str + delivery: str + dest: Path + manifest: Path + layout: str = "package" + use: tuple = () + scan: tuple = () + alias: tuple = ("soleprint-ui", "@framework") + theme: str | None = None + tokens: dict = field(default_factory=dict) + package: bool | str = False + notice: str | Path | None = None + header: str | None = None + rename: dict = field(default_factory=dict) + + @property + def vendored(self) -> bool: + return self.delivery == "vendored" + + def line(self) -> str: + return f"{self.name:<16} {self.target:<5} {self.delivery:<10} -> {self.dest}" + + +@dataclass +class RunFile: + path: Path + slices: list = field(default_factory=list) + plexuses: list = field(default_factory=list) # raw tables; plexus.py resolves them + + def select(self, names) -> list[Slice]: + if not names: + return list(self.slices) + known = {s.name for s in self.slices} + unknown = [n for n in names if n not in known] + if unknown: + raise ConfigError(self.path, [f"no slice named {n!r} — have: " + f"{', '.join(sorted(known))}" for n in unknown]) + return [s for s in self.slices if s.name in set(names)] + + +def _path(value, base: Path) -> Path: + p = Path(str(value)).expanduser() + return Path(os.path.normpath(p if p.is_absolute() else base / p)) + + +def _strings(value) -> bool: + return isinstance(value, list) and all(isinstance(x, str) for x in value) + + +def _table_of_strings(value) -> bool: + return isinstance(value, dict) and all(isinstance(k, str) and isinstance(v, str) + for k, v in value.items()) + + +def load(path, themes: list[str]) -> RunFile: + """Read and resolve a run file. Raises ConfigError listing every problem. + + `themes` is passed in so this stays a reader of files with nothing to know + about where themes live. + """ + path = Path(path) + if tomllib is None: + raise ConfigError(path, ["run files need Python 3.11+ (tomllib)"]) + try: + data = tomllib.loads(path.read_text()) + except OSError as e: + raise ConfigError(path, [f"cannot read: {e}"]) from None + except tomllib.TOMLDecodeError as e: + raise ConfigError(path, [f"not valid TOML: {e}"]) from None + + base = path.resolve().parent + problems: list[str] = [] + for key in sorted(set(data) - TOP_KEYS): + problems.append(f"unknown top-level key {key!r} — have: {', '.join(sorted(TOP_KEYS))}") + + defaults = data.get("defaults") or {} + if not isinstance(defaults, dict): + problems.append("[defaults] must be a table") + defaults = {} + for key in sorted(set(defaults) - DEFAULT_KEYS): + problems.append(f"[defaults] has unknown key {key!r} — have: " + f"{', '.join(sorted(DEFAULT_KEYS))}") + + raw_slices = data.get("slice") or [] + plexuses = data.get("plexus") or [] + if not isinstance(raw_slices, list) or not isinstance(plexuses, list): + problems.append("[[slice]] and [[plexus]] must be arrays of tables") + raw_slices, plexuses = [], [] + if not raw_slices and not plexuses: + problems.append("nothing to do — add a [[slice]] (or a [[plexus]]) table") + + slices: list[Slice] = [] + for i, raw in enumerate(raw_slices): + where = f"slice[{i}]" + if not isinstance(raw, dict): + problems.append(f"{where} must be a table") + continue + name = raw.get("name") + if isinstance(name, str): + where = f"slice {name!r}" + if not isinstance(name, str) or not NAME.match(name): + problems.append(f"{where} needs a name of letters, digits, '.', '_' or '-'") + continue + for key in sorted(set(raw) - SLICE_KEYS): + problems.append(f"{where} has unknown key {key!r} — have: {', '.join(sorted(SLICE_KEYS))}") + + def pick(key, default=None): + return raw[key] if key in raw else defaults.get(key, default) + + target = pick("target") + if target not in TARGETS: + problems.append(f"{where}: target must be one of {', '.join(TARGETS)}") + continue + delivery = pick("delivery", "dependency") + if delivery not in DELIVERIES: + problems.append(f"{where}: delivery must be one of {', '.join(DELIVERIES)}") + if target == "plain": + for key in sorted(VUE_ONLY & set(raw)): + problems.append(f"{where}: {key!r} applies to target = \"vue\" only") + + if not isinstance(raw.get("dest"), str): + problems.append(f"{where} needs `dest`, the directory the slice is written to") + continue + dest = _path(raw["dest"], base) + + layout = pick("layout", "flat" if delivery == "vendored" else "package") + if layout not in LAYOUTS: + problems.append(f"{where}: layout must be one of {', '.join(LAYOUTS)}") + + use = pick("use", []) + if not _strings(use): + problems.append(f"{where}: use must be a list of names") + use = [] + scan = pick("scan", []) + if not _strings(scan): + problems.append(f"{where}: scan must be a list of paths") + scan = [] + scan_paths = [] + for s in scan: + sp = _path(s, base) + if not sp.exists(): + problems.append(f"{where}: scan path {sp} does not exist") + scan_paths.append(sp) + if not use and not scan: + problems.append(f"{where}: nothing selected — give `use`, `scan`, or both") + + alias = pick("alias", ["soleprint-ui", "@framework"]) + if not _strings(alias) or not alias: + problems.append(f"{where}: alias must be a non-empty list of import specifiers") + alias = ["soleprint-ui", "@framework"] + + theme = pick("theme") + if theme is not None and (not isinstance(theme, str) or + (theme != "library" and theme not in themes)): + problems.append(f"{where}: theme must be \"library\" or one of: {', '.join(themes)}") + theme = None + if target == "plain" and theme in (None, "library"): + # Parts are written against the theme contract, never the library's file. + theme = "soleprint" + + tokens = pick("tokens", {}) + if not _table_of_strings(tokens) or not all(TOKEN.match(k) for k in tokens): + problems.append(f"{where}: tokens must map --names to values") + tokens = {} + + package = pick("package", False) + if not isinstance(package, (bool, str)): + problems.append(f"{where}: package must be true, false, or a package name") + package = False + + notice = pick("notice") + if delivery == "vendored": + if notice is None: + problems.append(f"{where}: a vendored slice needs `notice` — \"none\", or a file to " + f"copy in. The tool never picks the licensing for you.") + elif notice != "none": + np = _path(notice, base) + if not np.is_file(): + problems.append(f"{where}: notice {np} is not a file") + notice = np + elif notice is not None and notice != "none": + notice = _path(notice, base) + + header = pick("header") + if header is not None and not isinstance(header, str): + problems.append(f"{where}: header must be a string") + header = None + + rename = pick("rename", {}) + if not _table_of_strings(rename): + problems.append(f"{where}: rename must map identifiers to identifiers") + rename = {} + + manifest = _path(raw["manifest"], base) if isinstance(raw.get("manifest"), str) \ + else base / f"{name}.slice.json" + + slices.append(Slice(name=name, target=target, delivery=delivery, dest=dest, + manifest=manifest, layout=layout, use=tuple(use), + scan=tuple(scan_paths), alias=tuple(alias), theme=theme, + tokens=dict(tokens), package=package, notice=notice, + header=header, rename=dict(rename))) + + seen_names, seen_dests, seen_manifests = set(), {}, {} + for s in slices: + if s.name in seen_names: + problems.append(f"slice {s.name!r} is listed twice") + seen_names.add(s.name) + for key, seen, what in ((s.dest, seen_dests, "dest"), (s.manifest, seen_manifests, "manifest")): + k = key.resolve() + if k in seen: + problems.append(f"slices {seen[k]!r} and {s.name!r} share one {what}: {key}") + seen[k] = s.name + + if problems: + raise ConfigError(path, problems) + return RunFile(path=path, slices=slices, plexuses=list(plexuses)) diff --git a/soleprint/common/slice/scan.py b/soleprint/common/slice/scan.py new file mode 100644 index 0000000..40f3a72 --- /dev/null +++ b/soleprint/common/slice/scan.py @@ -0,0 +1,190 @@ +""" +What a consumer uses of the framework, read from the consumer itself. + +This is the check that used to be done by hand — grep an app for its imports, +diff its framework copy against spr — made repeatable, and made the input of a +slice. Two questions, answered separately: + +**Usage.** Which exports, files and custom properties the consumer's own code +reads. Both import styles in use are understood: + + import { Panel, type LogEntry } from 'soleprint-ui' barrel (mpr, nvi, unt) + import Panel from '@framework/components/Panel.vue' deep path (mts) + import '@framework/tokens.css' a stylesheet + +Its own `var(--…)` reads count too: every consumer imports tokens.css directly +and styles its own components with it, so a slice that carried only what the +framework reads would leave the app's own CSS unstyled. + +**Copy state.** When the consumer holds a copy of the framework (found through +its package.json `link:` or its Vite alias, or named with --copy), every file is +compared with current spr: same, differs, missing from the copy, or only in the +copy. A copy that is *ahead* shows up here first — which is how mts's maximize +was found before anything could overwrite it. +""" + +import difflib +import json +import re +from dataclasses import dataclass, field +from datetime import datetime +from pathlib import Path + +from . import graph +from .manifest import commit_exists, git, read_stamp + +DEFAULT_ALIASES = ("soleprint-ui", "@framework") +SKIP_DIRS = {"node_modules", "dist", ".git", "__pycache__", ".venv", "out", "gen"} +CODE = {".vue", ".ts", ".tsx", ".js", ".jsx", ".mjs"} +STYLE = {".css"} +MARKUP = {".html", ".htm", ".jinja", ".j2"} + +VAR = re.compile(r"var\(\s*(--[\w-]+)") +LINK = re.compile(r"^(?:link|file):(.+)$") +VITE_ALIAS = re.compile(r"""['"]([^'"]+)['"]\s*:\s*[^,\n]*?['"](\.{1,2}/[^'"]*)['"]""") + + +@dataclass +class Usage: + names: set = field(default_factory=set) # barrel names imported + files: set = field(default_factory=set) # SRC-relative files deep-imported + tokens: set = field(default_factory=set) # custom properties the consumer reads + texts: list = field(default_factory=list) # (path, text) of every file read + barrel: int = 0 # imports through the barrel + deep: int = 0 # imports by deep path + unknown: set = field(default_factory=set) # deep paths spr does not have + + def markup(self) -> list: + return [(p, t) for p, t in self.texts if p.suffix in MARKUP] + + +def _files(paths, exclude=()): + excluded = [Path(e).resolve() for e in exclude] + for base in paths: + base = Path(base) + candidates = [base] if base.is_file() else sorted(base.rglob("*")) + for p in candidates: + if not p.is_file() or SKIP_DIRS & set(p.parts): + continue + rp = p.resolve() + if any(rp == e or e in rp.parents for e in excluded): + continue + if p.suffix in CODE | STYLE | MARKUP: + yield p + + +def _deep_target(rest: str, nodes: dict) -> str | None: + rest = rest.lstrip("/") + if rest.startswith("src/"): + rest = rest[4:] + for candidate in (rest, rest + ".ts", rest + ".vue", rest + "/index.ts"): + if candidate in nodes or candidate in graph.GENERATED | {"base.css"}: + return candidate + return None + + +def usage(paths, aliases=DEFAULT_ALIASES, exclude=(), nodes=None) -> Usage: + """Read every file under `paths` for its use of the framework.""" + nodes = nodes if nodes is not None else graph.load() + alias_group = "|".join(re.escape(a) for a in sorted(aliases, key=len, reverse=True)) + named = re.compile( + r"import\s+(type\s+)?\{([^}]*)\}\s*from\s*['\"](" + alias_group + r")(/[^'\"]*)?['\"]", re.S) + any_import = re.compile( + r"(?:import|export)\s+(?:[^'\";]*?\s*from\s*)?['\"](" + alias_group + r")(/[^'\"]*)?['\"]", + re.S) + out = Usage() + for path in _files(paths, exclude): + text = path.read_text(errors="replace") + out.texts.append((path, text)) + out.tokens |= set(VAR.findall(text)) + if path.suffix not in CODE: + continue + for _, names, _, rest in named.findall(text): + if rest: + continue # `{ x } from '@framework/…'` — counted as deep below + for item in names.split(","): + item = item.strip() + if item.startswith("type "): + item = item[5:].strip() + if item: + out.names.add(item.split(" as ")[0].strip()) + for _, rest in any_import.findall(text): + if not rest: + out.barrel += 1 + continue + out.deep += 1 + target = _deep_target(rest, nodes) + if target: + out.files.add(target) + else: + out.unknown.add(rest.lstrip("/")) + return out + + +def find_copy(app: Path) -> Path | None: + """The framework copy an app resolves to: package.json `link:`, else a Vite alias.""" + pkg = app / "package.json" + if pkg.exists(): + try: + data = json.loads(pkg.read_text()) + except json.JSONDecodeError: + data = {} + deps = {**data.get("devDependencies", {}), **data.get("dependencies", {})} + for name in DEFAULT_ALIASES: + m = LINK.match(str(deps.get(name, ""))) + if m: + return (app / m.group(1)).resolve() + for config in sorted(app.glob("vite.config.*")): + for alias, target in VITE_ALIAS.findall(config.read_text()): + if alias in DEFAULT_ALIASES: + found = (app / target).resolve() + return found.parent if found.name == "src" else found + return None + + +@dataclass +class CopyState: + root: Path + rows: list = field(default_factory=list) # (rel, status, detail) + stamp: dict = field(default_factory=dict) + stamp_known: bool = False + + +def compare(copy_root: Path, src: Path = graph.SRC) -> CopyState: + """Every file of spr's src against the copy's src, both ways.""" + copy_src = copy_root / "src" if (copy_root / "src").is_dir() else copy_root + state = CopyState(root=copy_root, stamp=read_stamp(copy_root)) + state.stamp_known = commit_exists(state.stamp.get("sha", "")) + + def listing(root): + return {p.relative_to(root).as_posix() for p in root.rglob("*") + if p.is_file() and not graph.SKIP & set(p.relative_to(root).parts) + and p.suffix in (".ts", ".vue", ".css")} + + ours, theirs = listing(src), listing(copy_src) + for rel in sorted(ours | theirs): + if rel not in theirs: + state.rows.append((rel, "missing", "in spr, not in the copy")) + elif rel not in ours: + state.rows.append((rel, "copy-only", "in the copy, not in spr")) + else: + a = (src / rel).read_text().splitlines() + b = (copy_src / rel).read_text().splitlines() + if a == b: + state.rows.append((rel, "same", "")) + continue + added = removed = 0 + for line in difflib.unified_diff(a, b, lineterm="", n=0): + if line.startswith("+") and not line.startswith("+++"): + added += 1 + elif line.startswith("-") and not line.startswith("---"): + removed += 1 + # Dates, not a verdict. A worktree's mtimes are checkout times, so + # spr's side is dated by its last commit; the copy has no history, + # so its mtime is all there is. + ours_date = git("log", "-1", "--format=%cs", "--", str((src / rel).resolve())) + theirs_date = datetime.fromtimestamp((copy_src / rel).stat().st_mtime).strftime("%Y-%m-%d") + state.rows.append((rel, "differs", + f"+{added} -{removed} in the copy · copy {theirs_date}, " + f"spr last commit {ours_date or '?'}")) + return state diff --git a/soleprint/common/slice/selftest.py b/soleprint/common/slice/selftest.py new file mode 100644 index 0000000..2642fd2 --- /dev/null +++ b/soleprint/common/slice/selftest.py @@ -0,0 +1,255 @@ +""" +Prove slices on spr's own tree, writing only into a temporary directory. + + python3 -m common.slice.selftest (from soleprint/; or make ui selftest) + +Nothing in spr is written, and nothing is installed. The negative cases carry the +same weight as the positive ones; the checks that are the *design* rather than a +regression are marked (design), and they are the ones to keep if anything is cut: + + a slice holds the closure, not the library it is never a bundle + a vendored slice fails on any spr word no --force past the scrub + a hand edit in dest is refused, not overwritten drift is reported both ways + a vendored slice needs an explicit notice the tool never picks licensing +""" + +import ast +import re +import sys +import tempfile +from pathlib import Path + +from . import graph, manifest, names, runfile, scan, slices, tokens + +HERE = Path(__file__).resolve().parent +PASS, FAIL = [], [] + + +def check(name, condition, detail=""): + (PASS if condition else FAIL).append(name) + print(f" {'ok ' if condition else 'FAIL'} {name}" + ("" if condition or not detail + else f"\n {detail}")) + + +def section(title): + print(f"\n{title}") + + +def run_file(tmp: Path, body: str, name="ui.toml") -> Path: + path = tmp / name + path.write_text(body) + return path + + +def load(path: Path): + return runfile.load(path, tokens.available_themes()) + + +def main() -> int: + nodes = graph.load() + exports = graph.barrel(nodes=nodes) + + section("the graph") + check("Panel.vue imports no other library file", not nodes["components/Panel.vue"].imports) + check("an SFC always depends on vue, even with no import", + "vue" in nodes["components/Panel.vue"].packages) + check("useDataSource reaches DataSource", + "datasources/DataSource.ts" in nodes["composables/useDataSource.ts"].imports) + check("theme.css imports tokens.css and base.css", + nodes["theme.css"].imports == {"tokens.css", "base.css"}) + check("a type-only import is an edge (plugins -> InteractionPlugin)", + "plugins/InteractionPlugin.ts" in nodes["plugins/BBoxDrawPlugin.ts"].imports) + check("the barrel's exports are all found", len(exports) >= 34, f"found {len(exports)}") + check("type exports are marked as types", + {e.name for e in exports if e.is_type} >= {"LogEntry", "ConfigField", "DataSourceStatus"}) + + section("closures") + c, unknown = graph.select(["Panel", "SplitPane"], [], exports, nodes) + check("(design) Panel + SplitPane is two files, not the library", + c == {"components/Panel.vue", "components/SplitPane.vue"}, sorted(c)) + check("... and depends on vue alone", graph.dependencies(c, nodes) == {"vue": "^3.5"}, + graph.dependencies(c, nodes)) + c, _ = graph.select(["TimeSeriesRenderer"], [], exports, nodes) + check("TimeSeriesRenderer brings uplot, and only it", set(graph.dependencies(c, nodes)) == {"uplot", "vue"}) + c, _ = graph.select(["GraphRenderer"], [], exports, nodes) + check("GraphRenderer brings @vue-flow/core", "@vue-flow/core" in graph.dependencies(c, nodes)) + c, unknown = graph.select(["Panel", "NoSuchThing"], ["components/Nope.vue"], exports, nodes) + check("an unknown name or file is reported, not ignored", + set(unknown) == {"NoSuchThing", "components/Nope.vue"}, unknown) + c, _ = graph.select([], ["tokens.css", "components/Panel.vue"], exports, nodes) + check("the library's own token file is regenerated, never copied", "tokens.css" not in c) + + section("tokens") + lib, theme = tokens.source(None), tokens.source("soleprint") + differ = {n for n in lib.table if tokens.resolved(lib, n) != tokens.resolved(theme, n)} + check("library and soleprint theme differ exactly where documented " + "(why Vue defaults to the library)", differ == {"--font-mono", "--font-ui", "--surface-3"}, + sorted(differ)) + block, missing = lib.block({"--panel-border"}) + check("a chain travels whole: --panel-border brings --border", + "--border:" in block and "var(--border)" in block and not missing, block) + with_override = tokens.source(None, {"--border": "#ff0000"}) + check("an override lands before aliases are read", + tokens.resolved(with_override, "--panel-border") == "1px solid #ff0000") + _, missing = lib.block({"--does-not-exist"}) + check("a name nobody defines is reported", missing == ["--does-not-exist"]) + check("the .waiting rule is found in the library file", ".waiting" in tokens.library_rules()) + + section("names and the scrub") + profile = names.load() + check("sprint is an English word, not a leak", not names.scrub("sprint planning", profile)) + check("spr on its own is a leak", bool(names.scrub("built by spr", profile))) + check("a camelCase spr identifier is a leak", bool(names.scrub("sprHelper()", profile))) + renamed = names.rename("global.SprFeed = SprFeed // a vein page", profile) + check("identifiers rename in code, prose only in comments", + "LiveFeed" in renamed and "connector page" in renamed and "SprFeed" not in renamed, renamed) + renamed = names.rename("var vein = 1", profile) + check("a code identifier named like a prose term is left alone (and the scrub catches it)", + renamed == "var vein = 1" and names.scrub(renamed, profile)) + check("a hyphenated identifier is replaced whole", + names.rename("'soleprint-ui'", profile) == "'ui-kit'") + check("the leading header is stripped, later comments kept", + names.strip_header("/* head */\na{}\n/* keep */\n", ".css") == "a{}\n/* keep */\n") + + section("the run file") + with tempfile.TemporaryDirectory() as t: + tmp = Path(t) + try: + load(run_file(tmp, '[[slice]]\nname="x"\ntarget="vue"\ndelivery="vendored"\n' + 'dest="d"\nuse=["Panel"]\n')) + check("(design) a vendored slice without notice is refused", False) + except runfile.ConfigError as e: + check("(design) a vendored slice without notice is refused", + any("notice" in p for p in e.problems), e.problems) + try: + load(run_file(tmp, '[[slice]]\nname="x"\ntarget="vue"\ndset="d"\nuse=["Panel"]\n' + 'colour="red"\n[[slice]]\nname="y"\ntarget="plain"\ndest="e"\n' + 'layout="flat"\n')) + check("every problem is reported in one pass", False) + except runfile.ConfigError as e: + check("every problem is reported in one pass", len(e.problems) >= 4, e.problems) + rf = load(run_file(tmp, '[[slice]]\nname="x"\ntarget="plain"\ndest="sub/out"\nuse=["panel"]\n')) + check("paths are relative to the run file", rf.slices[0].dest == tmp / "sub" / "out") + check("plain resolves through the theme contract by default", rf.slices[0].theme == "soleprint") + check("the manifest defaults beside the run file", rf.slices[0].manifest == tmp / "x.slice.json") + + section("writing, and drift both ways") + with tempfile.TemporaryDirectory() as t: + tmp = Path(t) + rf = load(run_file(tmp, '[[slice]]\nname="app"\ntarget="vue"\ndest="fw"\n' + 'use=["Panel","SplitPane"]\n')) + entry = rf.slices[0] + built = slices.build(entry) + steps = slices.plan(entry, built) + check("a first run creates every file", {a for _, a, _ in steps.actions} == {"create"}) + slices.apply(entry, built, steps, force=False) + check("a dependency slice carries a .spr stamp in spr.py's format", + manifest.read_stamp(entry.dest).get("mode") == "slice") + check("the barrel exports only what the slice holds", + "LogRenderer" not in (entry.dest / "src/index.ts").read_text() + and "SplitPane" in (entry.dest / "src/index.ts").read_text()) + check("a second run changes nothing", not slices.plan(entry, slices.build(entry)).stale) + + panel = entry.dest / "src/components/Panel.vue" + panel.write_text(panel.read_text() + "\n\n") + steps = slices.plan(entry, slices.build(entry)) + check("(design) a hand edit is a conflict", + [a for r, a, _ in steps.actions if r.endswith("Panel.vue")] == ["conflict"]) + try: + slices.apply(entry, slices.build(entry), steps, force=False) + check("(design) ... and nothing is written without --force", False) + except ValueError: + check("(design) ... and nothing is written without --force", + "local change" in panel.read_text()) + + # Spr moving on: dest still holds what the manifest recorded, but the + # slice would now write something else. Simulated by recording the + # edited file as ours. + m = manifest.read(entry.manifest) + m["files"]["src/components/Panel.vue"]["sha256"] = manifest.file_sha(panel) + manifest.write(entry.manifest, m) + steps = slices.plan(entry, slices.build(entry)) + check("a file the manifest trusts is an update, not a conflict", + [a for r, a, _ in steps.actions if r.endswith("Panel.vue")] == ["update"]) + slices.apply(entry, slices.build(entry), steps, force=False) + + rf = load(run_file(tmp, '[[slice]]\nname="app"\ntarget="vue"\ndest="fw"\nuse=["Panel"]\n')) + steps = slices.plan(rf.slices[0], slices.build(rf.slices[0])) + check("a file the slice no longer needs is deleted", + ("src/components/SplitPane.vue", "delete", "no longer part of the slice") in steps.actions) + + section("vendored") + with tempfile.TemporaryDirectory() as t: + tmp = Path(t) + (tmp / "NOTICE").write_text("Notice text chosen by the owner.\n") + rf = load(run_file(tmp, '[[slice]]\nname="v"\ntarget="vue"\ndelivery="vendored"\n' + 'notice="NOTICE"\ndest="repo/ui"\nmanifest="private/v.json"\n' + 'use=["Panel","SplitPane","SSEDataSource"]\n')) + entry = rf.slices[0] + built = slices.build(entry) + check("(design) a vendored slice of real pieces scrubs clean", not built.hits, built.hits[:3]) + slices.apply(entry, built, slices.plan(entry, built), force=False) + written = sorted(p.relative_to(entry.dest).as_posix() for p in entry.dest.rglob("*") if p.is_file()) + check("no stamp and no manifest land in the consumer's tree", + ".spr" not in written and not any(w.endswith(".json") for w in written), written) + check("the manifest is written where the run file says", (tmp / "private/v.json").exists()) + check("the notice is copied as given", (entry.dest / "NOTICE").read_text().startswith("Notice text")) + check("flat layout for a vendored slice", (entry.dest / "index.ts").exists()) + leaks = [w for w in written for _ in names.scrub((entry.dest / w).read_text(), names.load())] + check("an independent scrub of the written files finds nothing", not leaks, leaks) + + page = tmp / "page.html" + page.write_text('
' + 'x
\n' + '\n') + rf = load(run_file(tmp, '[[slice]]\nname="p"\ntarget="plain"\ndelivery="vendored"\n' + 'notice="none"\ndest="plain"\nscan=["page.html"]\n', "plain.toml")) + built = slices.build(rf.slices[0]) + check("maximize brings panel; base comes with any part", + {"base", "panel", "maximize"} <= set(built.closure), built.closure) + check("a vendored page calling the renamed global still selects feed", "feed" in built.closure, + built.closure) + check("the renamed global is what ships", "LiveFeed" in built.files.get("ui.js", "") + and "SprFeed" not in built.files.get("ui.js", "")) + check("the plain slice scrubs clean", not built.hits, built.hits[:3]) + js = built.files.get("ui.js", "") + chunks = [c.strip() for c in re.split(r"(?m)^/\* [\w-]+ \*/$", js) if c.strip()] + check("parts joined into one ui.js are each terminated (an unterminated IIFE " + "turns the next one into a call)", + chunks and all(c.endswith(";") for c in chunks), [c[-30:] for c in chunks]) + + section("consumers (read-only, skipped when absent)") + meetus = Path.home() / "wdir/semester/mts/ui/meetus-app" + if meetus.exists(): + u = scan.usage([meetus / "src"], nodes=nodes) + check("meetus is read through deep paths", + u.files >= {"components/Panel.vue", "components/SplitPane.vue", "tokens.css"} and u.deep > 0) + copy = scan.find_copy(meetus) + check("its framework copy is found through the Vite alias", copy and copy.name == "framework") + else: + print(" skip meetus snapshot not on this machine") + + section("the split") + offenders = [] + for path in HERE.rglob("*.py"): + rel = path.relative_to(HERE) + if rel.parts[0] == "cli" or rel.as_posix() in ("__main__.py", "selftest.py"): + continue + for node in ast.walk(ast.parse(path.read_text())): + if isinstance(node, ast.Import) and any(a.name == "argparse" for a in node.names): + offenders.append(rel.as_posix()) + if isinstance(node, ast.ImportFrom) and node.module == "argparse": + offenders.append(rel.as_posix()) + check("nothing outside cli/ parses a command line", not offenders, offenders) + room = [p.relative_to(HERE).as_posix() for p in HERE.rglob("*") + if p.is_file() and p.suffix in (".py", ".json", ".toml") + and re.search(r"\broom", p.read_text(), re.I) + and p.name != "selftest.py"] + check("no file here uses the retired word", not room, room) + + print(f"\n{len(PASS)} passed, {len(FAIL)} failed") + return 1 if FAIL else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/soleprint/common/slice/slices.py b/soleprint/common/slice/slices.py new file mode 100644 index 0000000..19c45b4 --- /dev/null +++ b/soleprint/common/slice/slices.py @@ -0,0 +1,414 @@ +""" +Build a slice in memory, then write it — or say exactly why not. + +Building never touches the disk. Writing is a separate step that compares what +would be written with what is there and with the manifest, and refuses as a +whole: a run that overwrote half a copy and then stopped on a hand edit would +leave the consumer worse off than either state. + +Per file, one of: + + create not in dest yet + same dest already holds exactly this + update dest holds what the last run wrote; spr has moved on + delete the last run wrote it and the slice no longer needs it + conflict dest differs from what the last run wrote — edited by hand, or + never written by this slice. Refused unless --force. + +A vendored slice must also pass the scrub. That gate has no --force: a slice +that names spr is not a vendored slice. +""" + +import json +import re +from dataclasses import dataclass, field +from pathlib import Path + +from ..theme import bake +from . import graph, names, scan, tokens +from .manifest import (REPO, STAMP, file_sha, read, sha256, source_state, stamp, write) + +PARTS_DIR = graph.COMMON / "theme" / "parts" +UI_PACKAGE = graph.PACKAGE_JSON + +# What a dependency slice says at the top of every file it generates. No path +# and no command: the same text is right in every consumer, on every machine. +PROVENANCE = ("Generated by spr (`make ui slice`) from {source}. " + "Edits here are overwritten; change spr, or keep a change with --force.") + + +@dataclass +class Built: + entry: object + files: dict = field(default_factory=dict) # dest-relative path -> text + origins: dict = field(default_factory=dict) # dest-relative path -> where it came from + selected: list = field(default_factory=list) # export names, or part names + closure: list = field(default_factory=list) # source files, or parts + deps: dict = field(default_factory=dict) + carried: list = field(default_factory=list) # custom properties in tokens + fell_back: list = field(default_factory=list) + missing: list = field(default_factory=list) + unknown: list = field(default_factory=list) + hits: list = field(default_factory=list) # (file, line, term, text) + notes: list = field(default_factory=list) + + @property + def blocked(self) -> list[str]: + why = [] + if self.unknown: + why.append(f"unknown: {', '.join(sorted(self.unknown))}") + if self.hits: + why.append(f"scrub: {len(self.hits)} hit(s) in a vendored slice") + return why + + +def _comment(text: str, suffix: str) -> str: + if suffix in (".ts", ".js"): + return "".join(f"// {line}".rstrip() + "\n" for line in text.splitlines()) + if suffix in (".vue", ".html"): + return f"\n" + return f"/* {text} */\n" + + +def _header(entry, suffix: str, source: str) -> str: + if entry.header is not None: + return _comment(entry.header, suffix) if entry.header else "" + if entry.vendored: + return "" + return _comment(PROVENANCE.format(source=source), suffix) + + +def _finish(built: Built, profile) -> None: + """Vendored: rename through the profile, then scrub. Dependency: as is.""" + if not built.entry.vendored: + return + for rel, text in list(built.files.items()): + if rel.endswith(".json") and rel != "package.json": + continue + text = names.rename(text, profile) + built.files[rel] = text + for line, term, content in names.scrub(text, profile): + built.hits.append((rel, line, term, content)) + + +def _notice(entry, built: Built) -> None: + if entry.vendored and isinstance(entry.notice, Path): + built.files[entry.notice.name] = entry.notice.read_text() + built.origins[entry.notice.name] = str(entry.notice) + + +def _token_file(entry, built: Built, wanted: set, defined: set, texts: list, + header: str) -> str: + src = tokens.source(None if entry.theme in (None, "library") else entry.theme, entry.tokens) + block, missing = src.block(wanted) + built.carried = sorted(src.closure(wanted) - set(missing)) + built.fell_back = sorted(src.fell_back) + # A name the consumer declares itself is the consumer's, not a gap. + built.missing = sorted(set(missing) - defined) + rules = tokens.library_rules() if tokens.uses_waiting(texts) else "" + return header + block + (("\n" + rules) if rules else "") + + +def _defined(texts) -> set: + """Custom properties a consumer sets itself — in CSS, or from a Vue style + binding (`:style="{ '--toggle-color': c }"`), where the name is quoted.""" + return {m for _, t in texts for m in re.findall(r"(--[\w-]+)['\"]?\s*:", t)} + + +def build_vue(entry) -> Built: + built = Built(entry=entry) + profile = names.load(entry.rename) + nodes = graph.load() + exports = graph.barrel(nodes=nodes) + export_names = {e.name for e in exports} + + use_names = [u for u in entry.use if u in export_names] + use_files = [u for u in entry.use if u not in export_names] + usage = scan.usage(entry.scan, entry.alias, exclude=[entry.dest], nodes=nodes) \ + if entry.scan else scan.Usage() + + wanted_names = sorted(set(use_names) | usage.names) + wanted_files = sorted((set(use_files) | usage.files) - graph.GENERATED - {"base.css"}) + closure, unknown = graph.select(wanted_names, wanted_files, exports, nodes) + built.unknown = sorted(set(unknown) | usage.unknown) + built.selected = sorted(set(wanted_names) | set(wanted_files)) + built.closure = sorted(closure) + built.deps = graph.dependencies(closure, nodes) + + prefix = "src/" if entry.layout == "package" else "" + origin = "soleprint/common/ui/src" + + for rel in sorted(closure): + text = (graph.SRC / rel).read_text() + if entry.vendored: + text = names.strip_header(text, Path(rel).suffix) + built.files[prefix + rel] = text + built.origins[prefix + rel] = f"{origin}/{rel}" + + base = (graph.SRC / "base.css").read_text() + if entry.vendored: + base = names.strip_header(base, ".css") + built.files[prefix + "base.css"] = base + built.origins[prefix + "base.css"] = f"{origin}/base.css" + + wanted_tokens = graph.tokens_read(closure | {"base.css"}, nodes) | usage.tokens + texts = [(graph.SRC / rel, (graph.SRC / rel).read_text()) for rel in closure] + usage.texts + built.files[prefix + "tokens.css"] = _token_file( + entry, built, wanted_tokens, _defined(usage.texts), [t for _, t in texts], + _header(entry, ".css", f"{origin}/tokens.css")) + built.origins[prefix + "tokens.css"] = "generated" + + built.files[prefix + "theme.css"] = (_header(entry, ".css", f"{origin}/theme.css") + + "@import './tokens.css';\n@import './base.css';\n") + built.origins[prefix + "theme.css"] = "generated" + + built.files[prefix + "index.ts"] = _barrel(entry, exports, closure) + built.origins[prefix + "index.ts"] = "generated" + + if entry.delivery == "dependency" or entry.package: + built.files["package.json"] = _package_json(entry, built.deps, profile) + built.origins["package.json"] = "generated" + + _notice(entry, built) + _finish(built, profile) + return built + + +def _barrel(entry, exports, closure) -> str: + """index.ts exporting what the full barrel exports from the files held — no more.""" + header = _header(entry, ".ts", "soleprint/common/ui/src/index.ts") + lines = [header.rstrip("\n")] if header else [] + lines += ["import './theme.css'", ""] + by_file: dict[str, list] = {} + for e in exports: + if e.file in closure: + by_file.setdefault(e.file, []).append(e) + for file, items in by_file.items(): + spec = "./" + (file[:-3] if file.endswith(".ts") else file) + values = [e for e in items if not e.is_type] + types = [e for e in items if e.is_type] + if values: + names_ = ", ".join(e.local if e.local == e.name else f"{e.local} as {e.name}" + for e in values) + lines.append(f"export {{ {names_} }} from '{spec}'") + if types: + lines.append(f"export type {{ {', '.join(e.name for e in types)} }} from '{spec}'") + return "\n".join(lines).lstrip("\n") + "\n" + + +def _package_json(entry, deps: dict, profile) -> str: + version = json.loads(UI_PACKAGE.read_text()).get("version", "0.0.0") + if isinstance(entry.package, str): + name = entry.package + elif entry.vendored: + name = profile.identifiers.get("soleprint-ui", entry.name) + else: + name = "soleprint-ui" + src = "./src/" if entry.layout == "package" else "./" + data = { + "name": name, + "version": version, + "private": True, + "type": "module", + "main": "src/index.ts" if entry.layout == "package" else "index.ts", + "exports": { + ".": f"{src}index.ts", + "./tokens.css": f"{src}tokens.css", + "./theme.css": f"{src}theme.css", + "./base.css": f"{src}base.css", + **({"./src/*": "./src/*"} if entry.layout == "package" else {"./*": "./*"}), + }, + "dependencies": deps, + } + return json.dumps(data, indent=2) + "\n" + + +def _needs(name: str, parts: dict) -> list[str]: + header = bake.part_header(name, parts[name][0]) + line = bake.field(header, "NEEDS") + found = [w for w in re.findall(r"[a-z][\w-]*", line) if w in parts and w != name] + return list(dict.fromkeys(found)) + + +def build_plain(entry) -> Built: + built = Built(entry=entry) + profile = names.load(entry.rename) + parts = bake.load_parts() + reverse = profile.reverse() if entry.vendored else {} + + wanted = set() + for u in entry.use: + (wanted.add if u in parts else built.unknown.append)(u) + usage = scan.usage(entry.scan, (), exclude=[entry.dest]) if entry.scan else scan.Usage() + for path, text in usage.texts: + # A vendored page calls the renamed globals; read it back in spr's words + # so bake's DETECT lines still find them. + for new, old in reverse.items(): + text = re.sub(rf"(?, as bake places it. Joined into one file, `})(window)` + # followed by `(function …` parses as a call and the page dies — so + # every part is terminated before it is joined. + stripped += ";" + label = n if entry.vendored else f"part: {n} — common/theme/parts/{n}.{ext}" + return f"/* {label} */\n{stripped}\n" if stripped else "" + + css = [body(n, "css") for n in order] + js = [body(n, "js") for n in order] + wanted_tokens = set() + for n in order: + wanted_tokens |= tokens.read(parts[n][0]) + js_path = PARTS_DIR / f"{n}.js" + if js_path.exists(): + wanted_tokens |= tokens.read(js_path.read_text()) + wanted_tokens |= usage.tokens + + origin = "soleprint/common/theme/parts" + token_css = _token_file(entry, built, wanted_tokens, _defined(usage.texts), + [t for _, t in usage.texts], "") + header = _header(entry, ".css", origin) + built.files["ui.css"] = header + token_css + "\n" + "\n".join(c for c in css if c) + built.origins["ui.css"] = origin + if any(js): + built.files["ui.js"] = _header(entry, ".js", origin) + "\n".join(j for j in js if j) + built.origins["ui.js"] = origin + + _notice(entry, built) + _finish(built, profile) + return built + + +def build(entry) -> Built: + return build_vue(entry) if entry.target == "vue" else build_plain(entry) + + +@dataclass +class Plan: + actions: list = field(default_factory=list) # (rel, action, detail) + + def of(self, *kinds) -> list: + return [a for a in self.actions if a[1] in kinds] + + @property + def stale(self) -> bool: + return bool(self.of("create", "update", "delete", "conflict")) + + +def plan(entry, built: Built) -> Plan: + old = read(entry.manifest) or {} + recorded = {rel: meta.get("sha256") for rel, meta in old.get("files", {}).items()} + out = Plan() + for rel, text in built.files.items(): + current = file_sha(entry.dest / rel) + if current is None: + out.actions.append((rel, "create", "")) + elif current == sha256(text): + out.actions.append((rel, "same", "")) + elif recorded.get(rel) == current: + out.actions.append((rel, "update", "spr moved on")) + else: + why = "edited by hand since the last run" if rel in recorded \ + else "exists, and was not written by this slice" + out.actions.append((rel, "conflict", why)) + for rel in sorted(set(recorded) - set(built.files)): + current = file_sha(entry.dest / rel) + if current is None: + continue + if current == recorded[rel]: + out.actions.append((rel, "delete", "no longer part of the slice")) + else: + out.actions.append((rel, "conflict", "no longer part of the slice, and edited by hand")) + return out + + +def apply(entry, built: Built, steps: Plan, force: bool) -> list[str]: + """Write the slice. Returns what changed. Raises ValueError when refused.""" + if built.blocked: + raise ValueError("; ".join(built.blocked)) + conflicts = steps.of("conflict") + if conflicts and not force: + raise ValueError(f"{len(conflicts)} file(s) in dest differ from what this slice last " + f"wrote — nothing written. Bring a real change back to spr, or rerun " + f"with --force to overwrite: " + + ", ".join(rel for rel, _, _ in conflicts)) + changed = [] + for rel, action, _ in steps.actions: + target = entry.dest / rel + if action in ("create", "update") or (action == "conflict" and rel in built.files): + target.parent.mkdir(parents=True, exist_ok=True) + target.write_text(built.files[rel]) + changed.append(f"{'overwrite' if action == 'conflict' else action:<8} {rel}") + elif action == "delete" or (action == "conflict" and rel not in built.files): + target.unlink() + changed.append(f"delete {rel}") + + old = read(entry.manifest) or {} + inputs = [graph.SRC, PARTS_DIR, tokens.CONTRACT, tokens.THEMES, Path(__file__).parent] + manifest = { + "slice": entry.name, + "target": entry.target, + "delivery": entry.delivery, + "layout": entry.layout if entry.target == "vue" else None, + # The source only moves when the files do, so a new spr commit that + # changes nothing here does not churn the consumer's repository. + "source": source_state(inputs) if changed or "source" not in old else old["source"], + "selected": built.selected, + "closure": built.closure, + "deps": built.deps, + "tokens": {"source": entry.theme or "library", "carried": built.carried, + "fell_back": built.fell_back, "missing": built.missing, + "overrides": entry.tokens}, + "renames": names.load(entry.rename).identifiers if entry.vendored else {}, + "notice": (entry.notice.name if isinstance(entry.notice, Path) else entry.notice), + "files": {rel: {"from": built.origins.get(rel, "generated"), "sha256": sha256(text)} + for rel, text in sorted(built.files.items())}, + } + if manifest != old: + write(entry.manifest, manifest) + changed.append(f"manifest {entry.manifest}") + if entry.delivery == "dependency" and (changed or not (entry.dest / STAMP).exists()): + kind = "vue-slice" if entry.target == "vue" else "plain-slice" + version = json.loads(UI_PACKAGE.read_text()).get("version", "?") + source = graph.SRC if entry.target == "vue" else PARTS_DIR + (entry.dest / STAMP).parent.mkdir(parents=True, exist_ok=True) + (entry.dest / STAMP).write_text(stamp(entry.name, kind, version, source)) + return changed + + +def repo_relative(path: Path) -> str: + try: + return path.resolve().relative_to(REPO).as_posix() + except ValueError: + return str(path) diff --git a/soleprint/common/slice/tokens.py b/soleprint/common/slice/tokens.py new file mode 100644 index 0000000..7750184 --- /dev/null +++ b/soleprint/common/slice/tokens.py @@ -0,0 +1,162 @@ +""" +Where a slice's custom properties come from, and which of them it carries. + +Two sources, and the default differs by target on purpose: + + library common/ui/src/tokens.css — what every Vue consumer renders today. + The default for target = "vue", because swapping a consumer's copy + for a slice must not change a pixel. + common/theme/tokens.css + themes/.css — the theme contract + every plain page already uses. The default for target = "plain" + (as `soleprint`), and opt-in for Vue with `theme = "…"`. + +The two agree on almost every name but not all: the contract derives +--surface-3 from --border, and its font stacks are chosen for a locked-down +Windows box. Resolving a Vue slice through a theme is therefore a visible +change, which is exactly why it has to be asked for. + +A slice carries the declarations it reads **as written**, var() chains +included, plus everything those chains reach. Nothing is flattened to a literal: +a consumer that overrides --border still moves --panel-border with it, the same +as with the full file. A name the chosen source lacks falls back to the library +and is reported, because a fallback is a gap in the contract worth seeing. +""" + +import re +from dataclasses import dataclass, field +from pathlib import Path + +from ..theme import bake +from .graph import COMMON, SRC + +LIBRARY = SRC / "tokens.css" +THEME = COMMON / "theme" +CONTRACT = THEME / "tokens.css" +THEMES = THEME / "themes" + +COMMENT = re.compile(r"/\*.*?\*/", re.S) +REFERENCE = re.compile(r"var\(\s*(--[\w-]+)") +ROOT_BLOCK = re.compile(r":root\s*\{.*?\n\}", re.S) + + +def available_themes() -> list[str]: + return sorted(p.stem for p in THEMES.glob("*.css")) + + +def _declarations(css: str, selector: str) -> dict[str, str]: + """`--name: value;` pairs of one rule, comments removed first. + + bake.declarations reads a rule the same way; comments go first here because + prose inside a block ("Pages set --system-accent; …") must never be read as + a declaration. + """ + return bake.declarations(COMMENT.sub("", css), selector) + + +@dataclass +class Source: + """A resolved token source: the table a slice cuts its tokens from.""" + + name: str # "library" or a theme name + table: dict[str, str] # name -> value as written + fallback: dict[str, str] = field(default_factory=dict) + rules: str = "" # library extras outside :root + fell_back: set = field(default_factory=set) + + def closure(self, names: set) -> set: + """`names` plus every name their values reach through var().""" + seen, stack = set(), list(names) + while stack: + name = stack.pop() + if name in seen: + continue + seen.add(name) + value = self.table.get(name, self.fallback.get(name)) + if value: + stack.extend(set(REFERENCE.findall(value)) - seen) + return seen + + def value(self, name: str) -> str | None: + if name in self.table: + return self.table[name] + if name in self.fallback: + self.fell_back.add(name) + return self.fallback[name] + return None + + def block(self, names: set, indent: str = " ") -> tuple[str, list[str]]: + """The `:root { … }` a slice carries, and the names nobody defines. + + Order follows the source file, so the result reads like the file it was + cut from rather than like a sorted dump. + """ + wanted = self.closure(names) + order = list(self.table) + [n for n in self.fallback if n not in self.table] + lines, missing = [":root {"], [] + for name in order + sorted(wanted - set(order)): + if name not in wanted: + continue + value = self.value(name) + if value is None: + missing.append(name) + continue + lines.append(f"{indent}{name}: {value};") + wanted.discard(name) + lines.append("}") + return "\n".join(lines) + "\n", sorted(set(missing)) + + +def library_rules() -> str: + """What common/ui/src/tokens.css carries besides the :root block. + + Today that is the `.waiting` glow and its keyframes: a UI affordance that + happens to live in the token file. It travels only to a consumer that uses + the class — mpr's detection app does, the others do not. + """ + text = LIBRARY.read_text() + text = ROOT_BLOCK.sub("", text, count=1) + text = re.sub(r"\A\s*/\*.*?\*/", "", text, count=1, flags=re.S) # the file's header + return text.strip() + "\n" if text.strip() else "" + + +def source(theme: str | None, overrides: dict[str, str] | None = None) -> Source: + """The table for `theme` (None = the library), with overrides applied. + + Overrides go in before anything is read, so an alias that points at an + overridden name (`--amber: var(--accent)`) follows it. + """ + library = _declarations(LIBRARY.read_text(), ":root") + if theme is None or theme == "library": + table = dict(library) + table.update(overrides or {}) + return Source(name="library", table=table, rules=library_rules()) + + sheet = THEMES / f"{theme}.css" + if not sheet.exists(): + raise ValueError(f"no theme {theme!r} — have: {', '.join(available_themes())}") + table = _declarations(CONTRACT.read_text(), ":root") + table.update(_declarations(sheet.read_text(), f'[data-theme="{theme}"]')) + table.update(overrides or {}) + return Source(name=theme, table=table, fallback=library) + + +def resolved(src: Source, name: str) -> str | None: + """A token flattened to literals — for reports and comparisons only.""" + table = {**src.fallback, **src.table} + return bake.resolve(name, table) + + +def read(text: str) -> set: + """Custom properties a piece of text reads.""" + return set(REFERENCE.findall(text)) + + +def uses_waiting(texts) -> bool: + return any(re.search(r"\bwaiting\b", t) for t in texts) + + +def relpath(path: Path) -> str: + try: + return path.relative_to(COMMON.parent).as_posix() + except ValueError: + return str(path) diff --git a/soleprint/common/slice/ui.example.toml b/soleprint/common/slice/ui.example.toml new file mode 100644 index 0000000..88daa6f --- /dev/null +++ b/soleprint/common/slice/ui.example.toml @@ -0,0 +1,46 @@ +# A run file: every slice one consumer takes from spr, and how. +# +# ./ctrl/ui.sh slice soleprint/common/slice/ui.example.toml +# ./ctrl/ui.sh slice --list # resolved, nothing written +# ./ctrl/ui.sh check # nothing written; exit 1 if stale +# +# Copy it beside the consumer it describes. A real one names that consumer's +# paths, so it never lives in spr — and for a vendored slice keep it (and the +# manifest it writes) outside the consumer's repository, with `dest` pointing in. +# +# Paths are relative to THIS FILE. An entry's value replaces the default for +# every key. Unknown keys are refused, every problem in one pass. +# +# This one runs against spr's own tree and writes under out/ (gitignored). + +[defaults] +delivery = "dependency" + +# Vue, named outright: what a small live page takes. Drop-in `package` layout — +# package.json + src/ — the shape an app links as `soleprint-ui`. +[[slice]] +name = "panels" +target = "vue" +dest = "out/panels" +manifest = "out/panels.slice.json" +use = ["Panel", "SplitPane", "LogRenderer"] + +# The same pieces vendored: corp names, no headers, no stamp, scrubbed, and a +# `notice` stated on purpose (a path to a file, or "none"). +[[slice]] +name = "panels-vendored" +target = "vue" +delivery = "vendored" +notice = "none" +dest = "out/panels-vendored" +manifest = "out/panels-vendored.slice.json" +use = ["Panel", "SplitPane"] + +# Plain, read from a page's own markup: the jira vein's page uses panel and +# split, so that is what ui.css / ui.js carry — nothing else. +[[slice]] +name = "jira-page" +target = "plain" +dest = "out/jira-page" +manifest = "out/jira-page.slice.json" +scan = ["../../artery/veins/jira/ui/index.html"]