Files
soleprint/soleprint/station/tools/distill/explode.md
2026-09-16 05:27:53 -03:00

5.2 KiB

explode — one file back into the tree it describes

The other half of distill.sh. Together they are a round trip:

distill.sh digest  ->  one file  ->  paste into a chat  ->  the reply  ->  explode.sh

distill flattens repos into a single readable file so they fit somewhere that only takes text. explode takes the answer and writes it back onto disk. Neither is much use without the other.

./explode.sh --list reply.md            # what is in there; writes nothing
./explode.sh -o ./restored reply.md     # write the tree
./explode.sh -o ./restored --force x.md # overwrite what is already there
./explode.sh --raw -o ./restored x.md   # leave ⟪BT3⟫ escapes as they are
./explode.sh --contract > contract.txt  # the format to hand to the model
./explode.sh --selftest                 # check this copy against known input

Ask for @@, and attach it

--contract prints the output format to give whatever writes the reply. Attach that file; do not paste it into the message. A chat box renders markdown before the model sees it, and === alone under a line of text is setext syntax for a heading — so a pasted spec gets rendered as a title and the model is told nothing. --- is worse, ## is a heading, backticks open a fence. @@ means nothing in markdown, which is exactly why it is the marker to ask for.

Keeping the wording in --contract rather than in a note somewhere means what you ask for cannot drift from what the parser accepts.

Escaped fences

A chat UI shows its reply as rendered markdown, with each file inside a fence. The first ``` inside one of those files closes the fence: a README, a docstring example, the string "```json". Everything after it renders as prose. # turns into a heading, * into italics, <tag> disappears. The model copies what it was shown, so fences in the digest turn into fences in the reply.

So distill.sh digest escapes them. Inside file bodies, every run of three or more backticks is written as ⟪BTn⟫ and every run of tildes as ⟪TLn⟫, where n is the run length. A literal is written as ⟪LQ⟫, so a file that mentions the escape still comes back as itself. The digest says this at the top, and --contract asks for the same escapes in the reply. explode.sh turns them back into the real characters. It always does this for @@ replies, and does it for a digest only when the digest's header says it was escaped. --raw-fences on distill and --raw on explode turn it off. The tree copy is never escaped.

Nothing in the pipeline depends on the model obeying the formatting rules:

  • The contract asks for everything inside one ~~~~~~~~ block. Fence lines outside @@ blocks are prose to the parser and are ignored.
  • If the model wraps each file in its own fence anyway, a block whose first line opens a fence and whose last line closes one has both removed. Only the pair is removed, because a real file can end on a closing fence.
  • @@ END FILE: path must name the file it closes. If a close goes missing and two files end up in one block, the run is refused instead of gluing them together.

Copy the reply with Gemini's copy response button, not by selecting the rendered text. The button gives you the markdown as the model wrote it.

Layouts

Four shapes are recognised, picked automatically; --format overrides the guess.

shape when
@@ FILE: path@@ END FILE: path ask for this — explicit, and invisible to markdown; bare @@ END also read
=== FILE: path=== END the same thing, still read; do not ask for it
=== path marker a marker line, then the file until the next one
## path + fenced block distill.sh's own digest

Explicit open and close is worth insisting on: a writer emitting plain three-backtick fences silently truncates any file that itself contains a fence — every README with a shell example — because the nested fence looks exactly like the closing one.

One reply, several projects

A thread usually touches more than one repo, and produces one file regardless. The contract asks for paths that begin with the project name, so point -o at the directory those projects sit in and each file lands in its own worktree:

./explode.sh -o ~/wdir ~/Downloads/reply.md

No mapping table to maintain, and a new project needs no change here.

Refusals

Paths come out of a text file, so they are untrusted. Anything absolute or reaching upward with .. is refused and nothing is written — the check runs over the whole input before the first file is created. An unclosed block is refused too, rather than writing the file short. Existing files are never overwritten without --force.

A digest whose files distill clipped to fit a token budget is refused for the same reason: the document holds only their head and tail, and a truncated file that reads complete is the failure the whole format exists to prevent. The tree copy beside the digest has them whole — take them from there.

Two limits worth knowing: a file whose last line had no trailing newline comes back with one, and in the bare === path layout a line starting with === inside a file's own content cannot be told from a real marker. The digest and @@ layouts have no such ambiguity.