124 lines
6.8 KiB
Markdown
124 lines
6.8 KiB
Markdown
# meetus — repo index
|
|
|
|
A standalone, local-first **command-line tool**: turn screen-share meeting/training
|
|
recordings into a rich, AI-summarizable transcript. The extraction pipeline is
|
|
deterministic and offline; the summarization step is the only LLM-facing part.
|
|
|
|
meetus stands on its own. The separate **`cht`** project does the same thing in
|
|
realtime; meetus may or may not feed it, so the two are kept formally apart (the
|
|
cht-facing bits here live under `ctrl/cht/`).
|
|
|
|
> This file maps the whole repo. [`README.md`](README.md) is the detailed manual for
|
|
> the main `process_meeting.py` CLI.
|
|
|
|
## Data flow
|
|
|
|
```
|
|
video / audio
|
|
│ process_meeting.py (deterministic, offline)
|
|
▼
|
|
output/<run>/ run folder: YYYYMMDD-NNN-<stem>/
|
|
├── <stem>.json Whisper/WhisperX transcript (base)
|
|
├── <stem>.srt / .vtt / ... extra transcript formats (optional)
|
|
├── frames/ extracted scene/interval frames
|
|
├── <stem>_enhanced.txt transcript + frame refs ◀── the product
|
|
└── manifest.json
|
|
│ ctrl/summarize/*.py (local LLM — WIP, on hold)
|
|
▼
|
|
<stem>_summary_simple.md / _reference.md
|
|
```
|
|
|
|
`make batch` (→ `ctrl/batch.sh`) runs the pipeline over a whole directory tree,
|
|
mirroring the input folder structure into the output.
|
|
|
|
## doocus — document extraction (alongside meetus)
|
|
|
|
The same idea for documents. `process_doc.py` turns local files (Drive exports,
|
|
PDFs, images, ...) into a textified product + metadata, deterministically and
|
|
offline. Its output is what feeds permitted services (Gemini web, NotebookLM)
|
|
and the local search index; the raw source never leaves the machine.
|
|
|
|
```
|
|
document (docx/pdf/xlsx/img/...)
|
|
│ process_doc.py (deterministic, offline, no cloud AI)
|
|
▼
|
|
docs-output/<run>/ run folder: YYYYMMDD-NNN-<stem>/
|
|
├── content.md textified document ◀── the product
|
|
├── meta.json source/sha/dates/author/pages/... metadata
|
|
├── thumb.jpg optional single preview
|
|
└── manifest.json
|
|
```
|
|
|
|
`make docs` (→ `ctrl/batch_docs.sh`) runs it over a whole tree. Full doocus docs:
|
|
[`doocus/README.md`](doocus/README.md). Cross-search (meetus transcripts + doocus
|
|
docs, textified-only) is a later phase. 🚧
|
|
|
|
## Status legend
|
|
|
|
✅ active · 🚧 WIP, on hold · 🗄️ deprecated (kept for reference) · 🧰 one-off / niche · 📄 docs
|
|
|
|
## Layout
|
|
|
|
```
|
|
process_meeting.py ✅ the CLI tool — entry point (stays at root)
|
|
process_doc.py ✅ doocus CLI — document extraction entry point
|
|
Makefile ✅ batch convenience wrapper (make batch / make docs)
|
|
meetus/ ✅ core package
|
|
├─ workflow.py orchestrator (whisper → frames → merge)
|
|
├─ frame_extractor.py FFmpeg scene-detection / interval frames
|
|
├─ transcript_merger.py interleave transcript + frame refs → enhanced.txt
|
|
├─ output_manager.py run-folder naming + manifest.json
|
|
├─ cache_manager.py per-step caching (skip done work on rerun)
|
|
└─ deprecated/ 🗄️ old screen-text idea (OCR/vision/hybrid) — unwired, reference only
|
|
├─ ocr_processor.py (was --ocr-engine)
|
|
├─ vision_processor.py (was --use-vision)
|
|
├─ hybrid_processor.py (was --use-hybrid)
|
|
└─ prompts/ vision context prompts
|
|
doocus/ ✅ document-extraction package (see doocus/README.md)
|
|
├─ workflow.py DocConfig + DocWorkflow (dispatch → extract → write → manifest)
|
|
├─ registry.py extension → extractor family dispatch (lazy, graceful)
|
|
├─ output_manager.py run-folder + content.md/meta.json/thumb writer
|
|
├─ naming.py YYYYMMDD-NNN naming + sha256 (reuses meetus convention)
|
|
└─ extractors/ text, tabular(csv), office(docx/pptx/xlsx), pdf, web(html), image, media(mp4)
|
|
ctrl/ control plane / operational scripts
|
|
├─ batch.sh ✅ recursive batch runner (mirrors tree, continues past failures)
|
|
├─ batch_docs.sh ✅ doocus twin of batch.sh (make docs)
|
|
├─ transcribe_oneoff.sh 🧰 high-quality re-transcription over an existing run
|
|
├─ summarize/ 🚧 last step — local-LLM summarization (WIP, on hold)
|
|
│ ├─ summarize_simple.py minimal map-and-append; reads every referenced frame
|
|
│ ├─ compile_meeting.py REFINE-pattern technical reference; on-demand frames
|
|
│ └─ summarize_meeting.py map→extract(validated facts)→reduce
|
|
└─ cht/ 🧰 bridge to the separate realtime `cht` project
|
|
└─ interleave_cht_frames.py whisperx JSON + cht frames/index.json → enhanced.txt
|
|
ui/ local browser UIs (Vue 3 + Vite), shared framework
|
|
├─ framework/ ✅ shared component/renderer library + design tokens
|
|
├─ meetus-app/ ✅ review meeting runs (was ui/app/)
|
|
└─ doocus-app/ ✅ browse extracted docs + build packages (Gemini/NotebookLM)
|
|
def/ 📄 design/decision notes, in order (the feature history)
|
|
README.md 📄 manual for process_meeting.py
|
|
INDEX.md 📄 this file
|
|
MARIAN.md 📄 genesis brainstorm (explains why the deprecated OCR path exists)
|
|
local-run.sh 📄 personal scratch invocations (gitignored)
|
|
```
|
|
|
|
## Notes
|
|
|
|
- **Default flow** uses `--embed-images` (frames referenced for the LLM to read) +
|
|
`--scene-detection --scene-threshold 10 --diarize`. `make batch` adds
|
|
`--transcript-formats srt` and writes outputs next to the sources. See `README.md`.
|
|
- **`meetus/deprecated/`** is no longer imported or reachable from the CLI (its flags
|
|
were removed and `workflow.py` no longer imports it). Kept for reference only; the
|
|
realtime continuation of the idea is the separate `cht` project. Its only unique
|
|
dependency (`ollama`) is isolated to the `deprecated` uv group in `pyproject.toml`,
|
|
so a normal `uv sync --group meetus`/`--group doocus` never installs it.
|
|
- **Dependencies** are uv feature groups in `pyproject.toml` (no `requirements.txt`):
|
|
`meetus`, `doocus`, `ocr`, `pdf-render`, `deprecated`. whisper/whisperx are
|
|
external CLIs (like ffmpeg), installed separately — not uv-managed.
|
|
- **`ctrl/summarize/`** scripts are standalone; run them under an env with the `openai`
|
|
client (e.g. `~/wdir/llm/.venv`) against a local OpenAI-compatible server.
|
|
|
|
## Loose ends
|
|
|
|
- `README.md`'s "Output Files" example still shows the old run-folder format
|
|
`YYYYMMDD_HHMMSS-video` (actual: `YYYYMMDD-NNN-<stem>`). Minor; worth a tidy.
|