mariano/meetus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs

Browse Source
This commit is contained in:
Mariano Gabriel
2026-07-06 02:41:50 -03:00
parent 31ad8e5928
commit ef72d06bfe
6 changed files with 379 additions and 272 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
INDEX.md
View File

@@ -103,15 +103,30 @@ ctrl/ control plane / operational scripts
└─ interleave_cht_frames.py whisperx JSON + cht frames/index.json → enhanced.txt └─ interleave_cht_frames.py whisperx JSON + cht frames/index.json → enhanced.txt
ui/ local browser UIs (Vue 3 + Vite), shared framework ui/ local browser UIs (Vue 3 + Vite), shared framework
├─ framework/ ✅ shared component/renderer library + design tokens ├─ framework/ ✅ shared component/renderer library + design tokens
├─ meetus-app/ ✅ review meeting runs (was ui/app/) ├─ meetus-app/ ✅ review meeting runs (video · transcript · frames; @review source)
└─ doocus-app/ ✅ browse extracted docs + build packages (Gemini/NotebookLM) └─ doocus-app/ ✅ browse tree · search · package; embeds the meetus review
doocus-data/ meetus-data/ ✅ managed collection roots (gitignored): <root>/<source>/
docs/graphs/ 📄 architecture diagram (.dot → .svg, `make graphs`)
def/ 📄 design/decision notes, in order (the feature history) def/ 📄 design/decision notes, in order (the feature history)
README.md 📄 manual for process_meeting.py README.md 📄 project overview (meetus + doocus) + architecture diagram
INDEX.md 📄 this file INDEX.md 📄 this file
MARIAN.md 📄 genesis brainstorm (explains why the deprecated OCR path exists) MARIAN.md 📄 genesis brainstorm (explains why the deprecated OCR path exists)
local-run.sh 📄 personal scratch invocations (gitignored) local-run.sh 📄 personal scratch invocations (gitignored)
``` ```
## Collections & UIs
- **Managed roots** (gitignored): `doocus-data/<source>/` (docs) and
`meetus-data/<source>/` (meetings), each `<root>/<source>/` mirroring the source.
`process_tree.py --only {all|docs|meetings}` scopes a collection.
- **doocus-app** discovers both roots (`DOOCUS_DATA` overrides, comma-separated),
**merges collections that share `index.json.root`** into one source, searches the
cached text, and can **scan a folder** (runs `process_tree` via `uv`) from the UI.
- **meetus-app** is a thin shell over `@review` (`ui/meetus-app/src/components/ReviewBody.vue`),
the review composition doocus embeds — meetus stays the single source.
- **Diagram**: `docs/graphs/architecture.dot` → `docs/graphs/architecture.svg`
(`make graphs`; needs system graphviz). Both committed.
## Notes ## Notes
- **Default flow** uses `--embed-images` (frames referenced for the LLM to read) + - **Default flow** uses `--embed-images` (frames referenced for the LLM to read) +

13
Makefile
View File

@@ -67,7 +67,12 @@ PY := $(if $(PYTHON),$(PYTHON),python3)
MERGE_SRC ?= MERGE_SRC ?=
MERGE_DST ?= docs-output MERGE_DST ?= docs-output
.PHONY: batch dry docs docs-dry merge merge-dry help # Diagrams: graphviz .dot → .svg (both committed). Requires system graphviz
# (apt install graphviz). `make graphs` renders all docs/graphs/*.dot.
DOT_SRC := $(wildcard docs/graphs/*.dot)
SVG_OUT := $(DOT_SRC:.dot=.svg)
.PHONY: batch dry docs docs-dry merge merge-dry graphs help
batch: batch:
@ctrl/batch.sh -i "$(IN)" -o "$(OUT)" $(if $(EXT),-e "$(EXT)") $(if $(LIST),-l "$(LIST)") -- \ @ctrl/batch.sh -i "$(IN)" -o "$(OUT)" $(if $(EXT),-e "$(EXT)") $(if $(LIST),-l "$(LIST)") -- \
@@ -100,5 +105,11 @@ merge-dry:
@test -n "$(MERGE_SRC)" || { echo "ERROR: MERGE_SRC is required" >&2; exit 1; } @test -n "$(MERGE_SRC)" || { echo "ERROR: MERGE_SRC is required" >&2; exit 1; }
@rsync -avn $(MERGE_EXCLUDES) "$(MERGE_SRC)/" "$(MERGE_DST)/" @rsync -avn $(MERGE_EXCLUDES) "$(MERGE_SRC)/" "$(MERGE_DST)/"
docs/graphs/%.svg: docs/graphs/%.dot
dot -Tsvg $< -o $@
graphs: $(SVG_OUT)
@echo "rendered $(words $(SVG_OUT)) svg(s) from $(words $(DOT_SRC)) dot file(s)"
help: help:
@sed -n '3,14p' Makefile @sed -n '3,14p' Makefile

340
README.md
View File

@@ -1,293 +1,115 @@
# Meeting Processor # meetus + doocus — a local, offline context extractor
Extract screen content from meeting recordings and merge with Whisper/WhisperX transcripts for better AI summarization. Turn a downloaded Google Drive (meetings **and** documents) into a **searchable,
browsable, packageable** local knowledge base — then hand focused subsets to the
AI services you're allowed to use (Gemini web, NotebookLM). Every extraction step
is **deterministic and offline**; no cloud AI ever touches the raw source.
## Overview Two pipelines feed two local UIs over a shared component framework:
This tool enhances meeting transcripts by combining: - **meetus** — meeting recordings → enhanced transcripts (WhisperX + screen frames).
- **Audio transcription** (Whisper or WhisperX with speaker diarization) - **doocus** — documents → textified content + metadata, indexed into a tree.
- **Screen content extraction** via FFmpeg scene detection
- **Frame embedding** for direct LLM analysis
The result is a rich, timestamped transcript with embedded screen frames that provides full context for AI summarization. ![Architecture](docs/graphs/architecture.svg)
## Installation > Diagram source: [`docs/graphs/architecture.dot`](docs/graphs/architecture.dot)
> (regenerate with `make graphs`). Repo map: [`INDEX.md`](INDEX.md).
### 1. System Dependencies ## Install
**FFmpeg** (required for scene detection and frame extraction): ### System dependencies
```bash ```bash
# Ubuntu/Debian sudo apt-get install ffmpeg graphviz # ffmpeg: frames/thumbnails · graphviz: docs diagrams
sudo apt-get install ffmpeg # optional: tesseract-ocr (doocus --ocr), poppler-utils (pdf page render)
# macOS: brew install ffmpeg graphviz
# macOS
brew install ffmpeg
``` ```
### 2. Python Dependencies (uv) ### Python (uv feature groups)
Dependencies live in `pyproject.toml` as [uv](https://docs.astral.sh/uv/) groups —
Dependencies live in `pyproject.toml` as [uv](https://docs.astral.sh/uv/) install only what you need:
feature groups — install only what you need:
```bash ```bash
uv sync --group meetus # meeting pipeline (frame extraction) uv sync --group meetus # meeting pipeline (opencv, ffmpeg-python)
uv sync --group doocus # document extraction (see doocus/README.md) uv sync --group doocus # document extraction (docx/pdf/pptx/xlsx/…)
uv sync --group doocus --group ocr # + OCR for images / scanned pdfs uv sync --group doocus --group ocr # + OCR for images / scanned pdfs
uv run process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize
``` ```
Groups: `meetus`, `doocus`, `ocr`, `pdf-render`, `deprecated`. **whisper/whisperx**
are external CLI tools (like ffmpeg) — install separately (often a GPU env):
`pip install whisperx` (diarization needs a HuggingFace token with pyannote access).
Groups: `meetus`, `doocus`, `ocr`, `pdf-render`, `deprecated` (the last is the ### UIs (Node)
unwired OCR/vision path — the only user of `ollama`, kept out of every default
install).
### 3. Whisper or WhisperX (for audio transcription)
meetus calls these as **external CLI tools** (like ffmpeg), so they are *not*
uv-managed — install them however suits your machine (often a separate GPU env):
```bash ```bash
# standard whisper cd ui/meetus-app && npm install
pip install openai-whisper cd ui/doocus-app && npm install
# or WhisperX (recommended - adds speaker diarization)
pip install whisperx
``` ```
For speaker diarization, you'll need a HuggingFace token with access to pyannote models. ## The two pipelines
## Quick Start
### Recommended Usage
### meetus — meetings
```bash ```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize uv run process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
``` ```
Runs WhisperX (speaker diarization) + FFmpeg scene-detection frames → an enhanced
transcript with frame references, in `output/<YYYYMMDD-NNN-stem>/`. `--out-format`
customizes the run-folder name (e.g. `{name}.meetus` for a docs-coherent sidecar).
Batch a tree with `make batch IN=<dir>`. `process_meeting.py --help` is the flag
source of truth; caching (`--no-cache`, `--skip-cache-frames/-whisper`) lets you
iterate on scene thresholds without re-running Whisper.
This will: ### doocus — documents
1. Run WhisperX transcription with speaker diarization
2. Extract frames at scene changes (threshold 10 = moderately sensitive)
3. Create an enhanced transcript with frame file references
4. Save everything to `output/` folder
The `--embed-images` flag adds frame paths to the transcript (e.g., `Frame: frames/video_00257.jpg`), keeping the transcript small while frames stay in `frames/` folder for LLM access.
### Re-run with Cached Results
Already ran it once? Re-run instantly using cached results:
```bash ```bash
# Uses cached transcript and frames uv run process_tree.py "/path/to/drive" --output "doocus-data/<source>" --only docs
python process_meeting.py samples/meeting.mkv --embed-images
# Skip only specific cached items
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-frames
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper
# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache
``` ```
Replicates the whole tree into `index.json` (every file a node), extracting text
sidecars (`content.md` + `meta.json`) only for `docx/pdf/pptx/xlsx`; other files
are linked, not duplicated. `--only {all|docs|meetings}` scopes a collection.
Full manual: [`doocus/README.md`](doocus/README.md).
## Usage Examples ## Outputs & collections
### Scene Detection Options Each app writes to a **managed root** (gitignored), one subfolder per source,
mirroring the source structure:
- `doocus-data/<source>/` — document collections (`index.json` + `.doocus/`)
- `meetus-data/<source>/` — meeting collections (`index.json` + `.meetus/`)
Collections whose `index.json.root` matches are the **same source**: the doocus UI
merges their docs + meetings into one tree, toggled as one source. A meeting's
`.meetus` must sit in the **same folder** as its collection's `index.json`.
## The UIs
Local Vue 3 + Vite apps over `ui/framework/` (shared components + design tokens):
- **`ui/meetus-app`** — review a meeting: video + transcript (edit/read, speaker
merge, select mode) + frame selector, time-synced.
- **`ui/doocus-app`** — browse the merged tree, **search** cached text (transcripts,
extracted docs, raw files), per-type viewers, and **package** selected files.
Meetings **embed the meetus review** (`@review`, composed — not duplicated).
```bash ```bash
# Default threshold (15) cd ui/doocus-app && DOOCUS_DATA="$PWD/../../doocus-data,$PWD/../../meetus-data" npm run dev
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize cd ui/meetus-app && MEETUS_OUTPUT=/abs/path/to/output npm run dev
# More sensitive (more frames, threshold: 5)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --diarize
# Less sensitive (fewer frames, threshold: 30)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 30 --diarize
``` ```
### Fixed Interval Extraction (alternative to scene detection) ## Packaging (the point)
```bash
# Every 10 seconds
python process_meeting.py samples/meeting.mkv --embed-images --interval 10 --diarize
# Every 3 seconds (more detailed) The doocus package builder zips selected **originals** + their `content.md` for a
python process_meeting.py samples/meeting.mkv --embed-images --interval 3 --diarize target (Gemini web ≤10 files/≤100 MB, or NotebookLM). Originals are what the
permitted services consume; the extracted text is the local **search index** and is
never a replacement for the document.
## Project layout
See [`INDEX.md`](INDEX.md) for the full map. In brief:
```
process_meeting.py meetus CLI process_tree.py / process_doc.py doocus CLIs
meetus/ meetus core doocus/ doocus core
ctrl/ batch + ops scripts ui/{meetus-app,doocus-app,framework}
Makefile batch · docs · merge · graphs docs/graphs/ architecture diagram
def/ design notes samples/ output/ (gitignored)
``` ```
### Caching Examples Deprecated OCR/vision code lives unwired in `meetus/deprecated/` (the only user of
```bash `ollama`, kept out of every default install). See [`INDEX.md`](INDEX.md).
# First run - processes everything
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
# Iterate on scene threshold (reuse whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
# Re-run whisper only
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper
# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache
```
### Custom output location
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --output-dir my_outputs/
```
### Enable verbose logging
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --verbose
```
## Output Files
Each video gets its own timestamped output directory:
```
output/
└── 20241019_143022-meeting/
├── manifest.json # Processing configuration
├── meeting_enhanced.txt # Enhanced transcript for AI
├── meeting.json # Whisper/WhisperX transcript
└── frames/ # Extracted video frames
├── frame_00001_5.00s.jpg
├── frame_00002_10.00s.jpg
└── ...
```
### Caching Behavior
The tool automatically reuses the most recent output directory for the same video:
- **First run**: Creates new timestamped directory (e.g., `20241019_143022-meeting/`)
- **Subsequent runs**: Reuses the same directory and cached results
- **Cached items**: Whisper transcript, extracted frames, analysis results
**Fine-grained cache control:**
- `--no-cache`: Force complete reprocessing
- `--skip-cache-frames`: Re-extract frames only
- `--skip-cache-whisper`: Re-run transcription only
This allows you to iterate on scene detection thresholds without re-running Whisper!
## Workflow for Meeting Analysis
### Complete Workflow (One Command!)
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
```
### Typical Iterative Workflow
```bash
# First run - full processing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
# Adjust scene threshold (keeps cached whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
```
### Example Prompt for Claude
```
Please summarize this meeting transcript. Pay special attention to:
1. Key decisions made
2. Action items
3. Technical details shown on screen
4. Any metrics or data presented
[Paste enhanced transcript here]
```
## Command Reference
`process_meeting.py --help` is the source of truth for flags — run it rather than
relying on a copy here. The essentials:
- `--diarize` — WhisperX with speaker diarization (needs a HuggingFace token)
- `--embed-images` — reference frames in the transcript for the LLM (default behavior)
- `--scene-detection` / `--scene-threshold N` — frame extraction (lower = more frames)
- `--interval N` — fixed-interval extraction instead of scene detection
- `--transcript-formats srt,vtt,…` — extra transcript formats alongside JSON
- `--no-cache` / `--skip-cache-frames` / `--skip-cache-whisper` — cache control
For batches, prefer `make batch IN=<dir>` (see [`INDEX.md`](INDEX.md)).
## Tips for Best Results
### Scene Detection vs Interval
- **Scene detection** (`--scene-detection`): Recommended. Captures frames when content changes. More efficient.
- **Interval extraction** (`--interval N`): Alternative for continuous content. Captures every N seconds.
### Scene Detection Threshold
- Lower values (5-10): More sensitive, captures more frames
- Default (15): Good balance for most meetings
- Higher values (20-30): Less sensitive, fewer frames
### Whisper vs WhisperX
- **Whisper** (`--run-whisper`): Standard transcription, fast
- **WhisperX** (`--run-whisper --diarize`): Adds speaker identification, requires HuggingFace token
## Troubleshooting
### Frame Extraction Issues
**"No frames extracted"**
- Check video file is valid: `ffmpeg -i video.mkv`
- Try lower scene threshold: `--scene-threshold 5`
- Try interval extraction: `--interval 3`
- Check disk space in output directory
**Scene detection not working**
- Ensure FFmpeg is installed
- Falls back to interval extraction automatically
- Try manual interval: `--interval 5`
### Whisper/WhisperX Issues
**WhisperX diarization not working**
- Ensure you have a HuggingFace token set
- Token needs access to pyannote models
- Fall back to standard Whisper without `--diarize`
### Cache Issues
**Cache not being used**
- Ensure you're using the same video filename
- Check that output directory contains cached files
- Use `--verbose` to see what's being cached/loaded
**Want to re-run specific steps**
- `--skip-cache-frames`: Re-extract frames
- `--skip-cache-whisper`: Re-run transcription
- `--no-cache`: Force complete reprocessing
## Deprecated Features (kept for reference)
### OCR and Vision Analysis
OCR, Vision and Hybrid screen-text analysis were the original approach but went nowhere. They have been **removed from the CLI** (the `--ocr-engine` / `--use-vision` / `--use-hybrid` flags no longer exist) and now live, unwired, in `meetus/deprecated/` for reference only. The tool always references frames (`--embed-images`) so your LLM reads them directly. The realtime continuation of the idea is the separate `cht` project. See [`INDEX.md`](INDEX.md).
## Project Structure
See [`INDEX.md`](INDEX.md) for the full repo map. In brief:
```
meetus/
├── process_meeting.py # Main CLI script (entry point)
├── Makefile # `make batch` convenience wrapper
├── meetus/ # Core package
│ ├── workflow.py # Processing orchestrator
│ ├── frame_extractor.py # Frame extraction (FFmpeg scene detection)
│ ├── transcript_merger.py # Transcript + frame-ref merging
│ ├── output_manager.py # Run dirs (YYYYMMDD-NNN-<stem>) & manifest
│ ├── cache_manager.py # Per-step caching
│ └── deprecated/ # Old OCR/vision/hybrid analysis (reference only)
├── ctrl/ # Control plane / operational scripts
│ ├── batch.sh # Recursive batch runner
│ ├── transcribe_oneoff.sh # High-quality re-transcription
│ ├── summarize/ # Local-LLM summarization (WIP, on hold)
│ └── cht/ # Bridge to the realtime `cht` project
├── def/ # Design/decision notes
├── output/ # Run directories (gitignored)
├── samples/ # Sample inputs (gitignored)
└── README.md # This file
```
## License ## License
For personal use. For personal use.

53
docs/graphs/architecture.dot Normal file
View File

@@ -0,0 +1,53 @@
// Architecture of the meetus + doocus toolkit.
// Regenerate the SVG with: make graphs (or: dot -Tsvg docs/graphs/architecture.dot -o docs/graphs/architecture.svg)
digraph meetus_doocus {
rankdir=LR
splines=ortho
fontname="Helvetica"
node [shape=box, style="rounded,filled", fontname="Helvetica", fontsize=11, fillcolor="#f5f5f5", color="#cccccc"]
edge [fontname="Helvetica", fontsize=9, color="#888888"]
drive [label="Downloaded Drive\n(local · sensitive)", fillcolor="#e8eef7"]
subgraph cluster_pipe {
label="Deterministic, offline pipelines — no cloud AI touches the source"
labeljust=l ; style=dashed ; color="#bbbbbb" ; fontsize=10
meetus_cli [label="meetus\nprocess_meeting.py · make batch\nwhisperx + ffmpeg frames"]
doocus_cli [label="doocus\nprocess_tree.py · process_doc.py\nlocal text extraction"]
}
subgraph cluster_out {
label="Managed collection roots (gitignored) — <root>/<source>/, mirrored"
labeljust=l ; style=dashed ; color="#bbbbbb" ; fontsize=10
meetus_data [label="meetus-data/<source>/\nindex.json + <file>.meetus/\ntranscript · frames", fillcolor="#fdf0e3"]
doocus_data [label="doocus-data/<source>/\nindex.json + <file>.doocus/\ncontent.md · meta.json", fillcolor="#eaf6ea"]
}
subgraph cluster_ui {
label="Local UIs (Vue 3 + Vite)"
labeljust=l ; style=dashed ; color="#bbbbbb" ; fontsize=10
doocus_app [label="doocus-app\ntree · search · package\nembeds the meeting review"]
meetus_app [label="meetus-app\nmeeting review\nvideo · transcript · frames"]
review [label="@review\n(meetus ReviewBody — composed by both)", fillcolor="#f3ecfa"]
framework [label="ui/framework\nshared components + tokens", fillcolor="#f3ecfa"]
}
package [label="Package (zip)\noriginals + content.md", fillcolor="#e8eef7"]
services [label="Gemini web · NotebookLM\n(permitted services)", shape=ellipse, fillcolor="#e8eef7"]
drive -> meetus_cli
drive -> doocus_cli
meetus_cli -> meetus_data
doocus_cli -> doocus_data
doocus_data -> doocus_app
meetus_data -> doocus_app [xlabel="merge by\nshared root"]
meetus_data -> meetus_app
framework -> review [style=dashed, arrowhead=none]
review -> meetus_app [style=dashed, xlabel="standalone"]
review -> doocus_app [style=dashed, xlabel="embedded"]
doocus_app -> package
package -> services
}

186
docs/graphs/architecture.svg Normal file
View File

@@ -0,0 +1,186 @@
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN"
"http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
<!-- Generated by graphviz version 2.42.4 (0)
-->
<!-- Title: meetus_doocus Pages: 1 -->
<svg width="1517pt" height="314pt"
viewBox="0.00 0.00 1516.69 314.00" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g id="graph0" class="graph" transform="scale(1 1) rotate(0) translate(4 310)">
<title>meetus_doocus</title>
<polygon fill="white" stroke="transparent" points="-4,4 -4,-310 1512.69,-310 1512.69,4 -4,4"/>
<g id="clust1" class="cluster">
<title>cluster_pipe</title>
<polygon fill="none" stroke="#bbbbbb" stroke-dasharray="5,2" points="145,-157 145,-298 477,-298 477,-157 145,-157"/>
<text text-anchor="middle" x="311" y="-286" font-family="Helvetica,sans-Serif" font-size="10.00">Deterministic, offline pipelines — no cloud AI touches the source</text>
</g>
<g id="clust2" class="cluster">
<title>cluster_out</title>
<polygon fill="none" stroke="#bbbbbb" stroke-dasharray="5,2" points="505,-157 505,-298 859,-298 859,-157 505,-157"/>
<text text-anchor="middle" x="682" y="-286" font-family="Helvetica,sans-Serif" font-size="10.00">Managed collection roots (gitignored) — &lt;root&gt;/&lt;source&gt;/, mirrored</text>
</g>
<g id="clust3" class="cluster">
<title>cluster_ui</title>
<polygon fill="none" stroke="#bbbbbb" stroke-dasharray="5,2" points="213,-8 213,-149 1074,-149 1074,-8 213,-8"/>
<text text-anchor="middle" x="279" y="-137" font-family="Helvetica,sans-Serif" font-size="10.00">Local UIs (Vue 3 + Vite)</text>
</g>
<!-- drive -->
<g id="node1" class="node">
<title>drive</title>
<path fill="#e8eef7" stroke="#cccccc" d="M105,-236C105,-236 12,-236 12,-236 6,-236 0,-230 0,-224 0,-224 0,-212 0,-212 0,-206 6,-200 12,-200 12,-200 105,-200 105,-200 111,-200 117,-206 117,-212 117,-212 117,-224 117,-224 117,-230 111,-236 105,-236"/>
<text text-anchor="middle" x="58.5" y="-221.2" font-family="Helvetica,sans-Serif" font-size="11.00">Downloaded Drive</text>
<text text-anchor="middle" x="58.5" y="-209.2" font-family="Helvetica,sans-Serif" font-size="11.00">(local · sensitive)</text>
</g>
<!-- meetus_cli -->
<g id="node2" class="node">
<title>meetus_cli</title>
<path fill="#f5f5f5" stroke="#cccccc" d="M400,-209C400,-209 221,-209 221,-209 215,-209 209,-203 209,-197 209,-197 209,-177 209,-177 209,-171 215,-165 221,-165 221,-165 400,-165 400,-165 406,-165 412,-171 412,-177 412,-177 412,-197 412,-197 412,-203 406,-209 400,-209"/>
<text text-anchor="middle" x="310.5" y="-196.2" font-family="Helvetica,sans-Serif" font-size="11.00">meetus</text>
<text text-anchor="middle" x="310.5" y="-184.2" font-family="Helvetica,sans-Serif" font-size="11.00">process_meeting.py · make batch</text>
<text text-anchor="middle" x="310.5" y="-172.2" font-family="Helvetica,sans-Serif" font-size="11.00">whisperx + ffmpeg frames</text>
</g>
<!-- drive&#45;&gt;meetus_cli -->
<g id="edge1" class="edge">
<title>drive&#45;&gt;meetus_cli</title>
<path fill="none" stroke="#888888" d="M117.26,-204.5C117.26,-204.5 198.64,-204.5 198.64,-204.5"/>
<polygon fill="#888888" stroke="#888888" points="198.64,-208 208.64,-204.5 198.64,-201 198.64,-208"/>
</g>
<!-- doocus_cli -->
<g id="node3" class="node">
<title>doocus_cli</title>
<path fill="#f5f5f5" stroke="#cccccc" d="M398.5,-271C398.5,-271 222.5,-271 222.5,-271 216.5,-271 210.5,-265 210.5,-259 210.5,-259 210.5,-239 210.5,-239 210.5,-233 216.5,-227 222.5,-227 222.5,-227 398.5,-227 398.5,-227 404.5,-227 410.5,-233 410.5,-239 410.5,-239 410.5,-259 410.5,-259 410.5,-265 404.5,-271 398.5,-271"/>
<text text-anchor="middle" x="310.5" y="-258.2" font-family="Helvetica,sans-Serif" font-size="11.00">doocus</text>
<text text-anchor="middle" x="310.5" y="-246.2" font-family="Helvetica,sans-Serif" font-size="11.00">process_tree.py · process_doc.py</text>
<text text-anchor="middle" x="310.5" y="-234.2" font-family="Helvetica,sans-Serif" font-size="11.00">local text extraction</text>
</g>
<!-- drive&#45;&gt;doocus_cli -->
<g id="edge2" class="edge">
<title>drive&#45;&gt;doocus_cli</title>
<path fill="none" stroke="#888888" d="M117.26,-231.5C117.26,-231.5 200.13,-231.5 200.13,-231.5"/>
<polygon fill="#888888" stroke="#888888" points="200.14,-235 210.13,-231.5 200.13,-228 200.14,-235"/>
</g>
<!-- meetus_data -->
<g id="node4" class="node">
<title>meetus_data</title>
<path fill="#fdf0e3" stroke="#cccccc" d="M756,-209C756,-209 607,-209 607,-209 601,-209 595,-203 595,-197 595,-197 595,-177 595,-177 595,-171 601,-165 607,-165 607,-165 756,-165 756,-165 762,-165 768,-171 768,-177 768,-177 768,-197 768,-197 768,-203 762,-209 756,-209"/>
<text text-anchor="middle" x="681.5" y="-196.2" font-family="Helvetica,sans-Serif" font-size="11.00">meetus&#45;data/&lt;source&gt;/</text>
<text text-anchor="middle" x="681.5" y="-184.2" font-family="Helvetica,sans-Serif" font-size="11.00">index.json + &lt;file&gt;.meetus/</text>
<text text-anchor="middle" x="681.5" y="-172.2" font-family="Helvetica,sans-Serif" font-size="11.00">transcript · frames</text>
</g>
<!-- meetus_cli&#45;&gt;meetus_data -->
<g id="edge3" class="edge">
<title>meetus_cli&#45;&gt;meetus_data</title>
<path fill="none" stroke="#888888" d="M412.37,-187C412.37,-187 584.95,-187 584.95,-187"/>
<polygon fill="#888888" stroke="#888888" points="584.95,-190.5 594.95,-187 584.95,-183.5 584.95,-190.5"/>
</g>
<!-- doocus_data -->
<g id="node5" class="node">
<title>doocus_data</title>
<path fill="#eaf6ea" stroke="#cccccc" d="M754.5,-271C754.5,-271 608.5,-271 608.5,-271 602.5,-271 596.5,-265 596.5,-259 596.5,-259 596.5,-239 596.5,-239 596.5,-233 602.5,-227 608.5,-227 608.5,-227 754.5,-227 754.5,-227 760.5,-227 766.5,-233 766.5,-239 766.5,-239 766.5,-259 766.5,-259 766.5,-265 760.5,-271 754.5,-271"/>
<text text-anchor="middle" x="681.5" y="-258.2" font-family="Helvetica,sans-Serif" font-size="11.00">doocus&#45;data/&lt;source&gt;/</text>
<text text-anchor="middle" x="681.5" y="-246.2" font-family="Helvetica,sans-Serif" font-size="11.00">index.json + &lt;file&gt;.doocus/</text>
<text text-anchor="middle" x="681.5" y="-234.2" font-family="Helvetica,sans-Serif" font-size="11.00">content.md · meta.json</text>
</g>
<!-- doocus_cli&#45;&gt;doocus_data -->
<g id="edge4" class="edge">
<title>doocus_cli&#45;&gt;doocus_data</title>
<path fill="none" stroke="#888888" d="M410.9,-249C410.9,-249 586.43,-249 586.43,-249"/>
<polygon fill="#888888" stroke="#888888" points="586.43,-252.5 596.43,-249 586.43,-245.5 586.43,-252.5"/>
</g>
<!-- doocus_app -->
<g id="node6" class="node">
<title>doocus_app</title>
<path fill="#f5f5f5" stroke="#cccccc" d="M1054,-122C1054,-122 907,-122 907,-122 901,-122 895,-116 895,-110 895,-110 895,-90 895,-90 895,-84 901,-78 907,-78 907,-78 1054,-78 1054,-78 1060,-78 1066,-84 1066,-90 1066,-90 1066,-110 1066,-110 1066,-116 1060,-122 1054,-122"/>
<text text-anchor="middle" x="980.5" y="-109.2" font-family="Helvetica,sans-Serif" font-size="11.00">doocus&#45;app</text>
<text text-anchor="middle" x="980.5" y="-97.2" font-family="Helvetica,sans-Serif" font-size="11.00">tree · search · package</text>
<text text-anchor="middle" x="980.5" y="-85.2" font-family="Helvetica,sans-Serif" font-size="11.00">embeds the meeting review</text>
</g>
<!-- meetus_data&#45;&gt;doocus_app -->
<g id="edge6" class="edge">
<title>meetus_data&#45;&gt;doocus_app</title>
<path fill="none" stroke="#888888" d="M681.69,-164.89C681.69,-140.45 681.69,-104.5 681.69,-104.5 681.69,-104.5 884.87,-104.5 884.87,-104.5"/>
<polygon fill="#888888" stroke="#888888" points="884.87,-108 894.87,-104.5 884.87,-101 884.87,-108"/>
<text text-anchor="middle" x="727.08" y="-117.3" font-family="Helvetica,sans-Serif" font-size="9.00">merge by</text>
<text text-anchor="middle" x="727.08" y="-107.3" font-family="Helvetica,sans-Serif" font-size="9.00">shared root</text>
</g>
<!-- meetus_app -->
<g id="node7" class="node">
<title>meetus_app</title>
<path fill="#f5f5f5" stroke="#cccccc" d="M1049,-60C1049,-60 912,-60 912,-60 906,-60 900,-54 900,-48 900,-48 900,-28 900,-28 900,-22 906,-16 912,-16 912,-16 1049,-16 1049,-16 1055,-16 1061,-22 1061,-28 1061,-28 1061,-48 1061,-48 1061,-54 1055,-60 1049,-60"/>
<text text-anchor="middle" x="980.5" y="-47.2" font-family="Helvetica,sans-Serif" font-size="11.00">meetus&#45;app</text>
<text text-anchor="middle" x="980.5" y="-35.2" font-family="Helvetica,sans-Serif" font-size="11.00">meeting review</text>
<text text-anchor="middle" x="980.5" y="-23.2" font-family="Helvetica,sans-Serif" font-size="11.00">video · transcript · frames</text>
</g>
<!-- meetus_data&#45;&gt;meetus_app -->
<g id="edge7" class="edge">
<title>meetus_data&#45;&gt;meetus_app</title>
<path fill="none" stroke="#888888" d="M768.1,-187C810.98,-187 853.69,-187 853.69,-187 853.69,-187 853.69,-57 853.69,-57 853.69,-57 889.83,-57 889.83,-57"/>
<polygon fill="#888888" stroke="#888888" points="889.83,-60.5 899.83,-57 889.83,-53.5 889.83,-60.5"/>
</g>
<!-- doocus_data&#45;&gt;doocus_app -->
<g id="edge5" class="edge">
<title>doocus_data&#45;&gt;doocus_app</title>
<path fill="none" stroke="#888888" d="M766.83,-249C855.27,-249 980.69,-249 980.69,-249 980.69,-249 980.69,-132.31 980.69,-132.31"/>
<polygon fill="#888888" stroke="#888888" points="984.19,-132.31 980.69,-122.31 977.19,-132.31 984.19,-132.31"/>
</g>
<!-- package -->
<g id="node10" class="node">
<title>package</title>
<path fill="#e8eef7" stroke="#cccccc" d="M1233,-118C1233,-118 1114,-118 1114,-118 1108,-118 1102,-112 1102,-106 1102,-106 1102,-94 1102,-94 1102,-88 1108,-82 1114,-82 1114,-82 1233,-82 1233,-82 1239,-82 1245,-88 1245,-94 1245,-94 1245,-106 1245,-106 1245,-112 1239,-118 1233,-118"/>
<text text-anchor="middle" x="1173.5" y="-103.2" font-family="Helvetica,sans-Serif" font-size="11.00">Package (zip)</text>
<text text-anchor="middle" x="1173.5" y="-91.2" font-family="Helvetica,sans-Serif" font-size="11.00">originals + content.md</text>
</g>
<!-- doocus_app&#45;&gt;package -->
<g id="edge11" class="edge">
<title>doocus_app&#45;&gt;package</title>
<path fill="none" stroke="#888888" d="M1066.28,-100C1066.28,-100 1091.84,-100 1091.84,-100"/>
<polygon fill="#888888" stroke="#888888" points="1091.84,-103.5 1101.84,-100 1091.84,-96.5 1091.84,-103.5"/>
</g>
<!-- review -->
<g id="node8" class="node">
<title>review</title>
<path fill="#f3ecfa" stroke="#cccccc" d="M799,-87C799,-87 564,-87 564,-87 558,-87 552,-81 552,-75 552,-75 552,-63 552,-63 552,-57 558,-51 564,-51 564,-51 799,-51 799,-51 805,-51 811,-57 811,-63 811,-63 811,-75 811,-75 811,-81 805,-87 799,-87"/>
<text text-anchor="middle" x="681.5" y="-72.2" font-family="Helvetica,sans-Serif" font-size="11.00">@review</text>
<text text-anchor="middle" x="681.5" y="-60.2" font-family="Helvetica,sans-Serif" font-size="11.00">(meetus ReviewBody — composed by both)</text>
</g>
<!-- review&#45;&gt;doocus_app -->
<g id="edge10" class="edge">
<title>review&#45;&gt;doocus_app</title>
<path fill="none" stroke="#888888" stroke-dasharray="5,2" d="M811.34,-82.5C811.34,-82.5 884.71,-82.5 884.71,-82.5"/>
<polygon fill="#888888" stroke="#888888" points="884.71,-86 894.71,-82.5 884.71,-79 884.71,-86"/>
<text text-anchor="middle" x="872.53" y="-75.3" font-family="Helvetica,sans-Serif" font-size="9.00">embedded</text>
</g>
<!-- review&#45;&gt;meetus_app -->
<g id="edge9" class="edge">
<title>review&#45;&gt;meetus_app</title>
<path fill="none" stroke="#888888" stroke-dasharray="5,2" d="M811.34,-54C811.34,-54 889.9,-54 889.9,-54"/>
<polygon fill="#888888" stroke="#888888" points="889.9,-57.5 899.9,-54 889.9,-50.5 889.9,-57.5"/>
<text text-anchor="middle" x="875.62" y="-46.8" font-family="Helvetica,sans-Serif" font-size="9.00">standalone</text>
</g>
<!-- framework -->
<g id="node9" class="node">
<title>framework</title>
<path fill="#f3ecfa" stroke="#cccccc" d="M388,-87C388,-87 233,-87 233,-87 227,-87 221,-81 221,-75 221,-75 221,-63 221,-63 221,-57 227,-51 233,-51 233,-51 388,-51 388,-51 394,-51 400,-57 400,-63 400,-63 400,-75 400,-75 400,-81 394,-87 388,-87"/>
<text text-anchor="middle" x="310.5" y="-72.2" font-family="Helvetica,sans-Serif" font-size="11.00">ui/framework</text>
<text text-anchor="middle" x="310.5" y="-60.2" font-family="Helvetica,sans-Serif" font-size="11.00">shared components + tokens</text>
</g>
<!-- framework&#45;&gt;review -->
<g id="edge8" class="edge">
<title>framework&#45;&gt;review</title>
<path fill="none" stroke="#888888" stroke-dasharray="5,2" d="M400.31,-69C445.77,-69 501.8,-69 551.84,-69"/>
</g>
<!-- services -->
<g id="node11" class="node">
<title>services</title>
<ellipse fill="#e8eef7" stroke="#cccccc" cx="1394.84" cy="-100" rx="113.69" ry="22.76"/>
<text text-anchor="middle" x="1394.84" y="-103.2" font-family="Helvetica,sans-Serif" font-size="11.00">Gemini web · NotebookLM</text>
<text text-anchor="middle" x="1394.84" y="-91.2" font-family="Helvetica,sans-Serif" font-size="11.00">(permitted services)</text>
</g>
<!-- package&#45;&gt;services -->
<g id="edge12" class="edge">
<title>package&#45;&gt;services</title>
<path fill="none" stroke="#888888" d="M1245.06,-100C1245.06,-100 1270.89,-100 1270.89,-100"/>
<polygon fill="#888888" stroke="#888888" points="1270.89,-103.5 1280.89,-100 1270.89,-96.5 1270.89,-103.5"/>
</g>
</g>
</svg>
Side by Side

After

Width:  |  Height:  |  Size: 13 KiB

38
doocus/README.md
View File

@@ -100,18 +100,38 @@ uv sync --group doocus --group ocr # + pytesseract (needs system `tesseract`
(`--render`) additionally needs the `pdf-render` group (`pdf2image`) + system (`--render`) additionally needs the `pdf-render` group (`pdf2image`) + system
`poppler`. `poppler`.
## Collections (multiple sources)
doocus-app discovers collections from **managed roots** (gitignored),
`<root>/<source>/` per source:
- `doocus-data/<source>/` — document collections
- `meetus-data/<source>/` — meeting collections (`.meetus` + a `--only meetings` index)
Override the roots with `DOOCUS_DATA` (comma-separated; `DOOCUS_OUTPUT` still works
for a single dir). **Collections whose `index.json.root` match are the same source**
— the UI merges their docs + meetings into one interleaved tree, toggled together
from the `⧉` sources menu. When the same file exists in two collections, the copy
whose sidecar resolves wins (a `meetus-data` meeting with its `.meetus` overrides a
bare `doocus` copy).
## Browser UI ## Browser UI
`ui/doocus-app/` (sibling of `ui/meetus-app/`, shares `ui/framework/`) reads `ui/doocus-app/` (sibling of `ui/meetus-app/`, shares `ui/framework/`):
`index.json` and shows the **full tree** (folders preserved). Select a file →
detail view: for extracted docs, a split of the markdown-rendered **extracted - **Tree** (folders preserved, source-grouped) with a **search bar** — server-side
text** (clearly labelled "search index, not the document") beside the **native search over the cached text (transcripts, extracted `content.md`, raw files);
view of the original** (pdf inline; docx/xlsx → download); for linked files the matches **prune the tree** and show a file count + size.
original renders directly (image / html / text / markdown). A package builder - **Detail**: for extracted docs, markdown-rendered **extracted text** (labelled
zips selected **originals** (+ the `content.md` for extracted ones) for a target "search index, not the document") beside the **native original** (pdf inline;
(Gemini web / NotebookLM), and will emit Drive **links** once nodes carry a `url`. docx/xlsx rendered via mammoth/SheetJS); linked files render directly; **meetings
embed the meetus review** (video + transcript + frames, from `@review`).
- **Scan** a folder from the UI (`⧉` menu → Scan) — runs `process_tree` via `uv`
into `doocus-data/`; **⟳** re-discovers newly-scanned sources.
- **Package** builder zips selected **originals** (+ `content.md` for extracted) for
a target (Gemini web / NotebookLM); Drive **links** once nodes carry a `url`.
```bash ```bash
cd ui/doocus-app && npm install cd ui/doocus-app && npm install
DOOCUS_OUTPUT=/abs/path/to/docs-output npm run dev DOOCUS_DATA="/abs/doocus-data,/abs/meetus-data" npm run dev
``` ```