updated readme

2026-07-06 03:05:51 -03:00
parent ef72d06bfe
commit 1bad4e670b
2 changed files with 156 additions and 111 deletions
--- a/README.md
+++ b/README.md
@@ -1,115 +1,14 @@
-# meetus + doocus — a local, offline context extractor
+# meetus + doocus

-Turn a downloaded Google Drive (meetings **and** documents) into a **searchable,
-browsable, packageable** local knowledge base — then hand focused subsets to the
-AI services you're allowed to use (Gemini web, NotebookLM). Every extraction step
-is **deterministic and offline**; no cloud AI ever touches the raw source.
+A local, offline **context extractor**: turn a downloaded Google Drive — meetings
+**and** documents — into a searchable, browsable, packageable knowledge base, then
+hand focused subsets to the AI services you're allowed to use (Gemini web,
+NotebookLM). Every extraction step is deterministic and offline; no cloud AI ever
+touches the raw source.

-Two pipelines feed two local UIs over a shared component framework:
+- **meetus** — meeting recordings → enhanced transcripts (WhisperX + screen frames)
+- **doocus** — documents → textified content + metadata, indexed into a tree

- **meetus** — meeting recordings → enhanced transcripts (WhisperX + screen frames).
- **doocus** — documents → textified content + metadata, indexed into a tree.
+📖 **Documentation:** <https://docs.mcrn.ar/meetus/>

-![Architecture](docs/graphs/architecture.svg)
-
-> Diagram source: [`docs/graphs/architecture.dot`](docs/graphs/architecture.dot)
-> (regenerate with `make graphs`). Repo map: [`INDEX.md`](INDEX.md).
-
-## Install
-
-### System dependencies
-```bash
-sudo apt-get install ffmpeg graphviz     # ffmpeg: frames/thumbnails · graphviz: docs diagrams
-# optional: tesseract-ocr (doocus --ocr), poppler-utils (pdf page render)
-# macOS: brew install ffmpeg graphviz
-```
-
-### Python (uv feature groups)
-Dependencies live in `pyproject.toml` as [uv](https://docs.astral.sh/uv/) groups —
-install only what you need:
-```bash
-uv sync --group meetus                 # meeting pipeline (opencv, ffmpeg-python)
-uv sync --group doocus                 # document extraction (docx/pdf/pptx/xlsx/…)
-uv sync --group doocus --group ocr     # + OCR for images / scanned pdfs
-```
-Groups: `meetus`, `doocus`, `ocr`, `pdf-render`, `deprecated`. **whisper/whisperx**
-are external CLI tools (like ffmpeg) — install separately (often a GPU env):
-`pip install whisperx` (diarization needs a HuggingFace token with pyannote access).
-
-### UIs (Node)
-```bash
-cd ui/meetus-app && npm install
-cd ui/doocus-app && npm install
-```
-
-## The two pipelines
-
-### meetus — meetings
-```bash
-uv run process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
-```
-Runs WhisperX (speaker diarization) + FFmpeg scene-detection frames → an enhanced
-transcript with frame references, in `output/<YYYYMMDD-NNN-stem>/`. `--out-format`
-customizes the run-folder name (e.g. `{name}.meetus` for a docs-coherent sidecar).
-Batch a tree with `make batch IN=<dir>`. `process_meeting.py --help` is the flag
-source of truth; caching (`--no-cache`, `--skip-cache-frames/-whisper`) lets you
-iterate on scene thresholds without re-running Whisper.
-
-### doocus — documents
-```bash
-uv run process_tree.py "/path/to/drive" --output "doocus-data/<source>" --only docs
-```
-Replicates the whole tree into `index.json` (every file a node), extracting text
-sidecars (`content.md` + `meta.json`) only for `docx/pdf/pptx/xlsx`; other files
-are linked, not duplicated. `--only {all|docs|meetings}` scopes a collection.
-Full manual: [`doocus/README.md`](doocus/README.md).
-
-## Outputs & collections
-
-Each app writes to a **managed root** (gitignored), one subfolder per source,
-mirroring the source structure:
-
- `doocus-data/<source>/` — document collections (`index.json` + `.doocus/`)
- `meetus-data/<source>/` — meeting collections (`index.json` + `.meetus/`)
-
-Collections whose `index.json.root` matches are the **same source**: the doocus UI
-merges their docs + meetings into one tree, toggled as one source. A meeting's
-`.meetus` must sit in the **same folder** as its collection's `index.json`.
-
-## The UIs
-
-Local Vue 3 + Vite apps over `ui/framework/` (shared components + design tokens):
-
- **`ui/meetus-app`** — review a meeting: video + transcript (edit/read, speaker
-  merge, select mode) + frame selector, time-synced.
- **`ui/doocus-app`** — browse the merged tree, **search** cached text (transcripts,
-  extracted docs, raw files), per-type viewers, and **package** selected files.
-  Meetings **embed the meetus review** (`@review`, composed — not duplicated).
-```bash
-cd ui/doocus-app && DOOCUS_DATA="$PWD/../../doocus-data,$PWD/../../meetus-data" npm run dev
-cd ui/meetus-app && MEETUS_OUTPUT=/abs/path/to/output npm run dev
-```
-
-## Packaging (the point)
-
-The doocus package builder zips selected **originals** + their `content.md` for a
-target (Gemini web ≤10 files/≤100 MB, or NotebookLM). Originals are what the
-permitted services consume; the extracted text is the local **search index** and is
-never a replacement for the document.
-
-## Project layout
-
-See [`INDEX.md`](INDEX.md) for the full map. In brief:
-```
-process_meeting.py   meetus CLI            process_tree.py / process_doc.py   doocus CLIs
-meetus/              meetus core           doocus/                            doocus core
-ctrl/                batch + ops scripts   ui/{meetus-app,doocus-app,framework}
-Makefile             batch · docs · merge · graphs        docs/graphs/       architecture diagram
-def/                 design notes          samples/ output/                   (gitignored)
-```
-
-Deprecated OCR/vision code lives unwired in `meetus/deprecated/` (the only user of
-`ollama`, kept out of every default install). See [`INDEX.md`](INDEX.md).
-
-## License
-For personal use.
+🗺️ Repo map for contributors: [`INDEX.md`](INDEX.md)
--- a/docs/index.html
+++ b/docs/index.html
@@ -0,0 +1,146 @@
+<!DOCTYPE html>
+<html lang="en">
+<head>
+<meta charset="UTF-8">
+<meta name="viewport" content="width=device-width, initial-scale=1.0">
+<title>meetus + doocus — local offline context extractor</title>
+<style>
+  @import url('https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600&family=JetBrains+Mono:wght@400;500&display=swap');
+  * { margin: 0; padding: 0; box-sizing: border-box; }
+  :root {
+    --bg: #0a0e17; --panel: #121829; --border: #1e2a4a;
+    --text: #e8eaf0; --dim: #8a93a8; --accent: #4a8cff; --accent2: #2ecc9a;
+  }
+  body { background: var(--bg); color: var(--text); font-family: 'Inter', sans-serif; line-height: 1.65; }
+  a { color: var(--accent); text-decoration: none; }
+  a:hover { text-decoration: underline; }
+  code, pre { font-family: 'JetBrains Mono', monospace; }
+  code { background: #0e1524; border: 1px solid var(--border); border-radius: 4px; padding: 1px 5px; font-size: .85em; }
+  pre { background: #0e1524; border: 1px solid var(--border); border-radius: 8px; padding: 14px 16px; overflow-x: auto; margin: 10px 0; font-size: 13px; }
+  pre code { background: none; border: 0; padding: 0; }
+
+  header { padding: 28px 0 20px; border-bottom: 1px solid var(--border); }
+  header h1 { font-family: 'JetBrains Mono', monospace; font-size: 26px; letter-spacing: 2px; }
+  header h1 .a { color: var(--accent); } header h1 .b { color: var(--accent2); }
+  header .sub { color: var(--dim); font-size: 14px; margin-top: 6px; }
+
+  .wrap { max-width: 900px; margin: 0 auto; padding: 0 24px 80px; }
+  section { padding: 26px 0; border-bottom: 1px solid var(--border); }
+  h2 { font-size: 19px; margin-bottom: 10px; }
+  h3 { font-size: 15px; margin: 16px 0 6px; color: var(--text); }
+  p { color: #cdd3e0; margin: 8px 0; }
+  ul { margin: 8px 0 8px 22px; color: #cdd3e0; }
+  li { margin: 4px 0; }
+  .diagram { background: #fbfbfd; border: 1px solid var(--border); border-radius: 10px; padding: 16px; margin: 14px 0; text-align: center; }
+  .diagram img { max-width: 100%; }
+  .tag { display: inline-block; font-size: 11px; color: var(--dim); border: 1px solid var(--border); border-radius: 20px; padding: 1px 9px; margin-right: 6px; }
+  .grid2 { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; }
+  @media (max-width: 680px) { .grid2 { grid-template-columns: 1fr; } }
+  footer { color: var(--dim); font-size: 13px; padding: 22px 0; }
+</style>
+</head>
+<body>
+<div class="wrap">
+  <header>
+    <h1><span class="a">meetus</span> + <span class="b">doocus</span></h1>
+    <div class="sub">A local, offline context extractor · deterministic · no cloud AI touches the source</div>
+  </header>
+
+  <section>
+    <p>Turn a downloaded Google Drive — meetings <strong>and</strong> documents — into a
+      searchable, browsable, packageable knowledge base, then hand focused subsets to the
+      AI services you're allowed to use (Gemini web, NotebookLM). Two offline pipelines
+      feed two local UIs over a shared component framework.</p>
+    <p>
+      <span class="tag">meetus → meeting transcripts (WhisperX + frames)</span>
+      <span class="tag">doocus → documents → text + metadata tree</span>
+    </p>
+    <div class="diagram"><img src="graphs/architecture.svg" alt="meetus + doocus architecture"></div>
+  </section>
+
+  <section>
+    <h2>The two pipelines</h2>
+    <div class="grid2">
+      <div>
+        <h3>meetus — meetings</h3>
+        <p>WhisperX (speaker diarization) + FFmpeg scene-detection frames → an enhanced
+          transcript with frame references.</p>
+        <pre><code>uv run process_meeting.py meeting.mkv \
+  --embed-images --scene-detection \
+  --scene-threshold 10 --diarize</code></pre>
+        <p>Batch a tree with <code>make batch IN=&lt;dir&gt;</code>. <code>--out-format</code>
+          controls the run-folder name (e.g. <code>{name}.meetus</code>).</p>
+      </div>
+      <div>
+        <h3>doocus — documents</h3>
+        <p>Replicates the whole tree into <code>index.json</code>; extracts text sidecars
+          (<code>content.md</code> + <code>meta.json</code>) only for docx/pdf/pptx/xlsx —
+          other files are linked, not duplicated.</p>
+        <pre><code>uv run process_tree.py "/path/to/drive" \
+  --output "doocus-data/&lt;source&gt;" --only docs</code></pre>
+        <p><code>--only {all|docs|meetings}</code> scopes a collection.</p>
+      </div>
+    </div>
+  </section>
+
+  <section>
+    <h2>Outputs &amp; collections</h2>
+    <p>Each app writes to a <strong>managed root</strong> (gitignored), one subfolder per
+      source, mirroring the source structure:</p>
+    <ul>
+      <li><code>doocus-data/&lt;source&gt;/</code> — document collections (<code>index.json</code> + <code>.doocus/</code>)</li>
+      <li><code>meetus-data/&lt;source&gt;/</code> — meeting collections (<code>index.json</code> + <code>.meetus/</code>)</li>
+    </ul>
+    <p>Collections whose <code>index.json.root</code> match are the <strong>same source</strong>:
+      the doocus UI merges their docs + meetings into one tree, toggled as one source. When a
+      file exists in two collections, the copy whose sidecar resolves wins. A meeting's
+      <code>.meetus</code> must sit in the <strong>same folder</strong> as its collection's index.</p>
+  </section>
+
+  <section>
+    <h2>The UIs</h2>
+    <p>Local Vue 3 + Vite apps over <code>ui/framework/</code> (shared components + tokens):</p>
+    <ul>
+      <li><strong>meetus-app</strong> — review a meeting: video + transcript (edit/read, speaker
+        merge, select mode) + frame selector, time-synced.</li>
+      <li><strong>doocus-app</strong> — browse the merged tree, <strong>search</strong> the cached
+        text (transcripts, extracted docs, raw files), per-type viewers, and <strong>package</strong>
+        selected files. Meetings <strong>embed the meetus review</strong> (<code>@review</code>,
+        composed — not duplicated).</li>
+    </ul>
+    <pre><code>cd ui/doocus-app &amp;&amp; DOOCUS_DATA="/abs/doocus-data,/abs/meetus-data" npm run dev
+cd ui/meetus-app &amp;&amp; MEETUS_OUTPUT=/abs/output npm run dev</code></pre>
+  </section>
+
+  <section>
+    <h2>Packaging</h2>
+    <p>The doocus package builder zips selected <strong>originals</strong> + their
+      <code>content.md</code> for a target (Gemini web ≤10 files/≤100 MB, or NotebookLM).
+      Originals are what the permitted services consume; the extracted text is the local
+      <strong>search index</strong> — never a replacement for the document.</p>
+  </section>
+
+  <section>
+    <h2>Install</h2>
+    <pre><code># system
+sudo apt-get install ffmpeg graphviz          # + tesseract-ocr, poppler-utils (optional)
+
+# python (uv feature groups)
+uv sync --group meetus                         # meeting pipeline
+uv sync --group doocus --group ocr             # docs + OCR
+
+# whisper/whisperx are external CLIs (install separately, often a GPU env)
+pip install whisperx                           # diarization needs a HuggingFace token
+
+# UIs
+cd ui/meetus-app &amp;&amp; npm install
+cd ui/doocus-app &amp;&amp; npm install</code></pre>
+  </section>
+
+  <footer>
+    Repo map &amp; contributor docs: <code>INDEX.md</code> · <code>doocus/README.md</code> ·
+    diagram source in <code>docs/graphs/</code> (<code>make graphs</code>).
+  </footer>
+</div>
+</body>
+</html>