mariano/meetus
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs

Browse Source
This commit is contained in:
Mariano Gabriel
2026-07-06 02:41:50 -03:00
parent 31ad8e5928
commit ef72d06bfe
6 changed files with 379 additions and 272 deletions
Download Patch File Download Diff File Expand all files Collapse all files

340
README.md
View File

@@ -1,293 +1,115 @@
# Meeting Processor
# meetus + doocus — a local, offline context extractor
Extract screen content from meeting recordings and merge with Whisper/WhisperX transcripts for better AI summarization.
Turn a downloaded Google Drive (meetings **and** documents) into a **searchable,
browsable, packageable** local knowledge base — then hand focused subsets to the
AI services you're allowed to use (Gemini web, NotebookLM). Every extraction step
is **deterministic and offline**; no cloud AI ever touches the raw source.
## Overview
Two pipelines feed two local UIs over a shared component framework:
This tool enhances meeting transcripts by combining:
- **Audio transcription** (Whisper or WhisperX with speaker diarization)
- **Screen content extraction** via FFmpeg scene detection
- **Frame embedding** for direct LLM analysis
- **meetus** — meeting recordings → enhanced transcripts (WhisperX + screen frames).
- **doocus** — documents → textified content + metadata, indexed into a tree.
The result is a rich, timestamped transcript with embedded screen frames that provides full context for AI summarization.
![Architecture](docs/graphs/architecture.svg)
## Installation
> Diagram source: [`docs/graphs/architecture.dot`](docs/graphs/architecture.dot)
> (regenerate with `make graphs`). Repo map: [`INDEX.md`](INDEX.md).
### 1. System Dependencies
## Install
**FFmpeg** (required for scene detection and frame extraction):
### System dependencies
```bash
# Ubuntu/Debian
sudo apt-get install ffmpeg
# macOS
brew install ffmpeg
sudo apt-get install ffmpeg graphviz # ffmpeg: frames/thumbnails · graphviz: docs diagrams
# optional: tesseract-ocr (doocus --ocr), poppler-utils (pdf page render)
# macOS: brew install ffmpeg graphviz
```
### 2. Python Dependencies (uv)
Dependencies live in `pyproject.toml` as [uv](https://docs.astral.sh/uv/)
feature groups — install only what you need:
### Python (uv feature groups)
Dependencies live in `pyproject.toml` as [uv](https://docs.astral.sh/uv/) groups —
install only what you need:
```bash
uv sync --group meetus # meeting pipeline (frame extraction)
uv sync --group doocus # document extraction (see doocus/README.md)
uv sync --group meetus # meeting pipeline (opencv, ffmpeg-python)
uv sync --group doocus # document extraction (docx/pdf/pptx/xlsx/…)
uv sync --group doocus --group ocr # + OCR for images / scanned pdfs
uv run process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize
```
Groups: `meetus`, `doocus`, `ocr`, `pdf-render`, `deprecated`. **whisper/whisperx**
are external CLI tools (like ffmpeg) — install separately (often a GPU env):
`pip install whisperx` (diarization needs a HuggingFace token with pyannote access).
Groups: `meetus`, `doocus`, `ocr`, `pdf-render`, `deprecated` (the last is the
unwired OCR/vision path — the only user of `ollama`, kept out of every default
install).
### 3. Whisper or WhisperX (for audio transcription)
meetus calls these as **external CLI tools** (like ffmpeg), so they are *not*
uv-managed — install them however suits your machine (often a separate GPU env):
### UIs (Node)
```bash
# standard whisper
pip install openai-whisper
# or WhisperX (recommended - adds speaker diarization)
pip install whisperx
cd ui/meetus-app && npm install
cd ui/doocus-app && npm install
```
For speaker diarization, you'll need a HuggingFace token with access to pyannote models.
## Quick Start
### Recommended Usage
## The two pipelines
### meetus — meetings
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
uv run process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
```
Runs WhisperX (speaker diarization) + FFmpeg scene-detection frames → an enhanced
transcript with frame references, in `output/<YYYYMMDD-NNN-stem>/`. `--out-format`
customizes the run-folder name (e.g. `{name}.meetus` for a docs-coherent sidecar).
Batch a tree with `make batch IN=<dir>`. `process_meeting.py --help` is the flag
source of truth; caching (`--no-cache`, `--skip-cache-frames/-whisper`) lets you
iterate on scene thresholds without re-running Whisper.
This will:
1. Run WhisperX transcription with speaker diarization
2. Extract frames at scene changes (threshold 10 = moderately sensitive)
3. Create an enhanced transcript with frame file references
4. Save everything to `output/` folder
The `--embed-images` flag adds frame paths to the transcript (e.g., `Frame: frames/video_00257.jpg`), keeping the transcript small while frames stay in `frames/` folder for LLM access.
### Re-run with Cached Results
Already ran it once? Re-run instantly using cached results:
### doocus — documents
```bash
# Uses cached transcript and frames
python process_meeting.py samples/meeting.mkv --embed-images
# Skip only specific cached items
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-frames
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper
# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache
uv run process_tree.py "/path/to/drive" --output "doocus-data/<source>" --only docs
```
Replicates the whole tree into `index.json` (every file a node), extracting text
sidecars (`content.md` + `meta.json`) only for `docx/pdf/pptx/xlsx`; other files
are linked, not duplicated. `--only {all|docs|meetings}` scopes a collection.
Full manual: [`doocus/README.md`](doocus/README.md).
## Usage Examples
## Outputs & collections
### Scene Detection Options
Each app writes to a **managed root** (gitignored), one subfolder per source,
mirroring the source structure:
- `doocus-data/<source>/` — document collections (`index.json` + `.doocus/`)
- `meetus-data/<source>/` — meeting collections (`index.json` + `.meetus/`)
Collections whose `index.json.root` matches are the **same source**: the doocus UI
merges their docs + meetings into one tree, toggled as one source. A meeting's
`.meetus` must sit in the **same folder** as its collection's `index.json`.
## The UIs
Local Vue 3 + Vite apps over `ui/framework/` (shared components + design tokens):
- **`ui/meetus-app`** — review a meeting: video + transcript (edit/read, speaker
merge, select mode) + frame selector, time-synced.
- **`ui/doocus-app`** — browse the merged tree, **search** cached text (transcripts,
extracted docs, raw files), per-type viewers, and **package** selected files.
Meetings **embed the meetus review** (`@review`, composed — not duplicated).
```bash
# Default threshold (15)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize
# More sensitive (more frames, threshold: 5)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --diarize
# Less sensitive (fewer frames, threshold: 30)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 30 --diarize
cd ui/doocus-app && DOOCUS_DATA="$PWD/../../doocus-data,$PWD/../../meetus-data" npm run dev
cd ui/meetus-app && MEETUS_OUTPUT=/abs/path/to/output npm run dev
```
### Fixed Interval Extraction (alternative to scene detection)
```bash
# Every 10 seconds
python process_meeting.py samples/meeting.mkv --embed-images --interval 10 --diarize
## Packaging (the point)
# Every 3 seconds (more detailed)
python process_meeting.py samples/meeting.mkv --embed-images --interval 3 --diarize
The doocus package builder zips selected **originals** + their `content.md` for a
target (Gemini web ≤10 files/≤100 MB, or NotebookLM). Originals are what the
permitted services consume; the extracted text is the local **search index** and is
never a replacement for the document.
## Project layout
See [`INDEX.md`](INDEX.md) for the full map. In brief:
```
process_meeting.py meetus CLI process_tree.py / process_doc.py doocus CLIs
meetus/ meetus core doocus/ doocus core
ctrl/ batch + ops scripts ui/{meetus-app,doocus-app,framework}
Makefile batch · docs · merge · graphs docs/graphs/ architecture diagram
def/ design notes samples/ output/ (gitignored)
```
### Caching Examples
```bash
# First run - processes everything
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
# Iterate on scene threshold (reuse whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
# Re-run whisper only
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper
# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache
```
### Custom output location
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --output-dir my_outputs/
```
### Enable verbose logging
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --verbose
```
## Output Files
Each video gets its own timestamped output directory:
```
output/
└── 20241019_143022-meeting/
├── manifest.json # Processing configuration
├── meeting_enhanced.txt # Enhanced transcript for AI
├── meeting.json # Whisper/WhisperX transcript
└── frames/ # Extracted video frames
├── frame_00001_5.00s.jpg
├── frame_00002_10.00s.jpg
└── ...
```
### Caching Behavior
The tool automatically reuses the most recent output directory for the same video:
- **First run**: Creates new timestamped directory (e.g., `20241019_143022-meeting/`)
- **Subsequent runs**: Reuses the same directory and cached results
- **Cached items**: Whisper transcript, extracted frames, analysis results
**Fine-grained cache control:**
- `--no-cache`: Force complete reprocessing
- `--skip-cache-frames`: Re-extract frames only
- `--skip-cache-whisper`: Re-run transcription only
This allows you to iterate on scene detection thresholds without re-running Whisper!
## Workflow for Meeting Analysis
### Complete Workflow (One Command!)
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
```
### Typical Iterative Workflow
```bash
# First run - full processing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
# Adjust scene threshold (keeps cached whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
```
### Example Prompt for Claude
```
Please summarize this meeting transcript. Pay special attention to:
1. Key decisions made
2. Action items
3. Technical details shown on screen
4. Any metrics or data presented
[Paste enhanced transcript here]
```
## Command Reference
`process_meeting.py --help` is the source of truth for flags — run it rather than
relying on a copy here. The essentials:
- `--diarize` — WhisperX with speaker diarization (needs a HuggingFace token)
- `--embed-images` — reference frames in the transcript for the LLM (default behavior)
- `--scene-detection` / `--scene-threshold N` — frame extraction (lower = more frames)
- `--interval N` — fixed-interval extraction instead of scene detection
- `--transcript-formats srt,vtt,…` — extra transcript formats alongside JSON
- `--no-cache` / `--skip-cache-frames` / `--skip-cache-whisper` — cache control
For batches, prefer `make batch IN=<dir>` (see [`INDEX.md`](INDEX.md)).
## Tips for Best Results
### Scene Detection vs Interval
- **Scene detection** (`--scene-detection`): Recommended. Captures frames when content changes. More efficient.
- **Interval extraction** (`--interval N`): Alternative for continuous content. Captures every N seconds.
### Scene Detection Threshold
- Lower values (5-10): More sensitive, captures more frames
- Default (15): Good balance for most meetings
- Higher values (20-30): Less sensitive, fewer frames
### Whisper vs WhisperX
- **Whisper** (`--run-whisper`): Standard transcription, fast
- **WhisperX** (`--run-whisper --diarize`): Adds speaker identification, requires HuggingFace token
## Troubleshooting
### Frame Extraction Issues
**"No frames extracted"**
- Check video file is valid: `ffmpeg -i video.mkv`
- Try lower scene threshold: `--scene-threshold 5`
- Try interval extraction: `--interval 3`
- Check disk space in output directory
**Scene detection not working**
- Ensure FFmpeg is installed
- Falls back to interval extraction automatically
- Try manual interval: `--interval 5`
### Whisper/WhisperX Issues
**WhisperX diarization not working**
- Ensure you have a HuggingFace token set
- Token needs access to pyannote models
- Fall back to standard Whisper without `--diarize`
### Cache Issues
**Cache not being used**
- Ensure you're using the same video filename
- Check that output directory contains cached files
- Use `--verbose` to see what's being cached/loaded
**Want to re-run specific steps**
- `--skip-cache-frames`: Re-extract frames
- `--skip-cache-whisper`: Re-run transcription
- `--no-cache`: Force complete reprocessing
## Deprecated Features (kept for reference)
### OCR and Vision Analysis
OCR, Vision and Hybrid screen-text analysis were the original approach but went nowhere. They have been **removed from the CLI** (the `--ocr-engine` / `--use-vision` / `--use-hybrid` flags no longer exist) and now live, unwired, in `meetus/deprecated/` for reference only. The tool always references frames (`--embed-images`) so your LLM reads them directly. The realtime continuation of the idea is the separate `cht` project. See [`INDEX.md`](INDEX.md).
## Project Structure
See [`INDEX.md`](INDEX.md) for the full repo map. In brief:
```
meetus/
├── process_meeting.py # Main CLI script (entry point)
├── Makefile # `make batch` convenience wrapper
├── meetus/ # Core package
│ ├── workflow.py # Processing orchestrator
│ ├── frame_extractor.py # Frame extraction (FFmpeg scene detection)
│ ├── transcript_merger.py # Transcript + frame-ref merging
│ ├── output_manager.py # Run dirs (YYYYMMDD-NNN-<stem>) & manifest
│ ├── cache_manager.py # Per-step caching
│ └── deprecated/ # Old OCR/vision/hybrid analysis (reference only)
├── ctrl/ # Control plane / operational scripts
│ ├── batch.sh # Recursive batch runner
│ ├── transcribe_oneoff.sh # High-quality re-transcription
│ ├── summarize/ # Local-LLM summarization (WIP, on hold)
│ └── cht/ # Bridge to the realtime `cht` project
├── def/ # Design/decision notes
├── output/ # Run directories (gitignored)
├── samples/ # Sample inputs (gitignored)
└── README.md # This file
```
Deprecated OCR/vision code lives unwired in `meetus/deprecated/` (the only user of
`ollama`, kept out of every default install). See [`INDEX.md`](INDEX.md).
## License
For personal use.