meetus/README.md

# Meeting Processor

Extract screen content from meeting recordings and merge with Whisper/WhisperX transcripts for better AI summarization.

## Overview

This tool enhances meeting transcripts by combining:
- **Audio transcription** (Whisper or WhisperX with speaker diarization)
- **Screen content extraction** via FFmpeg scene detection
- **Frame embedding** for direct LLM analysis

The result is a rich, timestamped transcript with embedded screen frames that provides full context for AI summarization.

## Installation

### 1. System Dependencies

**FFmpeg** (required for scene detection and frame extraction):
```bash
# Ubuntu/Debian
sudo apt-get install ffmpeg

# macOS
brew install ffmpeg
```

### 2. Python Dependencies (uv)

Dependencies live in `pyproject.toml` as [uv](https://docs.astral.sh/uv/)
feature groups — install only what you need:

```bash
uv sync --group meetus                 # meeting pipeline (frame extraction)
uv sync --group doocus                 # document extraction (see doocus/README.md)
uv sync --group doocus --group ocr     # + OCR for images / scanned pdfs
uv run process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize
```

Groups: `meetus`, `doocus`, `ocr`, `pdf-render`, `deprecated` (the last is the
unwired OCR/vision path — the only user of `ollama`, kept out of every default
install).

### 3. Whisper or WhisperX (for audio transcription)

meetus calls these as **external CLI tools** (like ffmpeg), so they are *not*
uv-managed — install them however suits your machine (often a separate GPU env):

```bash
# standard whisper
pip install openai-whisper
# or WhisperX (recommended - adds speaker diarization)
pip install whisperx
```

For speaker diarization, you'll need a HuggingFace token with access to pyannote models.

## Quick Start

### Recommended Usage

```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
```

This will:
1. Run WhisperX transcription with speaker diarization
2. Extract frames at scene changes (threshold 10 = moderately sensitive)
3. Create an enhanced transcript with frame file references
4. Save everything to `output/` folder

The `--embed-images` flag adds frame paths to the transcript (e.g., `Frame: frames/video_00257.jpg`), keeping the transcript small while frames stay in `frames/` folder for LLM access.

### Re-run with Cached Results

Already ran it once? Re-run instantly using cached results:
```bash
# Uses cached transcript and frames
python process_meeting.py samples/meeting.mkv --embed-images

# Skip only specific cached items
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-frames
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper

# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache
```

## Usage Examples

### Scene Detection Options
```bash
# Default threshold (15)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize

# More sensitive (more frames, threshold: 5)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --diarize

# Less sensitive (fewer frames, threshold: 30)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 30 --diarize
```

### Fixed Interval Extraction (alternative to scene detection)
```bash
# Every 10 seconds
python process_meeting.py samples/meeting.mkv --embed-images --interval 10 --diarize

# Every 3 seconds (more detailed)
python process_meeting.py samples/meeting.mkv --embed-images --interval 3 --diarize
```

### Caching Examples
```bash
# First run - processes everything
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

# Iterate on scene threshold (reuse whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames

# Re-run whisper only
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper

# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache
```

### Custom output location
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --output-dir my_outputs/
```

### Enable verbose logging
```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --verbose
```

## Output Files

Each video gets its own timestamped output directory:

```
output/
└── 20241019_143022-meeting/
    ├── manifest.json                    # Processing configuration
    ├── meeting_enhanced.txt             # Enhanced transcript for AI
    ├── meeting.json                     # Whisper/WhisperX transcript
    └── frames/                          # Extracted video frames
        ├── frame_00001_5.00s.jpg
        ├── frame_00002_10.00s.jpg
        └── ...
```

### Caching Behavior

The tool automatically reuses the most recent output directory for the same video:
- **First run**: Creates new timestamped directory (e.g., `20241019_143022-meeting/`)
- **Subsequent runs**: Reuses the same directory and cached results
- **Cached items**: Whisper transcript, extracted frames, analysis results

**Fine-grained cache control:**
- `--no-cache`: Force complete reprocessing
- `--skip-cache-frames`: Re-extract frames only
- `--skip-cache-whisper`: Re-run transcription only

This allows you to iterate on scene detection thresholds without re-running Whisper!

## Workflow for Meeting Analysis

### Complete Workflow (One Command!)

```bash
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
```

### Typical Iterative Workflow

```bash
# First run - full processing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

# Adjust scene threshold (keeps cached whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
```

### Example Prompt for Claude

```
Please summarize this meeting transcript. Pay special attention to:
1. Key decisions made
2. Action items
3. Technical details shown on screen
4. Any metrics or data presented

[Paste enhanced transcript here]
```

## Command Reference

`process_meeting.py --help` is the source of truth for flags — run it rather than
relying on a copy here. The essentials:

- `--diarize` — WhisperX with speaker diarization (needs a HuggingFace token)
- `--embed-images` — reference frames in the transcript for the LLM (default behavior)
- `--scene-detection` / `--scene-threshold N` — frame extraction (lower = more frames)
- `--interval N` — fixed-interval extraction instead of scene detection
- `--transcript-formats srt,vtt,…` — extra transcript formats alongside JSON
- `--no-cache` / `--skip-cache-frames` / `--skip-cache-whisper` — cache control

For batches, prefer `make batch IN=<dir>` (see [`INDEX.md`](INDEX.md)).

## Tips for Best Results

### Scene Detection vs Interval
- **Scene detection** (`--scene-detection`): Recommended. Captures frames when content changes. More efficient.
- **Interval extraction** (`--interval N`): Alternative for continuous content. Captures every N seconds.

### Scene Detection Threshold
- Lower values (5-10): More sensitive, captures more frames
- Default (15): Good balance for most meetings
- Higher values (20-30): Less sensitive, fewer frames

### Whisper vs WhisperX
- **Whisper** (`--run-whisper`): Standard transcription, fast
- **WhisperX** (`--run-whisper --diarize`): Adds speaker identification, requires HuggingFace token

## Troubleshooting

### Frame Extraction Issues

**"No frames extracted"**
- Check video file is valid: `ffmpeg -i video.mkv`
- Try lower scene threshold: `--scene-threshold 5`
- Try interval extraction: `--interval 3`
- Check disk space in output directory

**Scene detection not working**
- Ensure FFmpeg is installed
- Falls back to interval extraction automatically
- Try manual interval: `--interval 5`

### Whisper/WhisperX Issues

**WhisperX diarization not working**
- Ensure you have a HuggingFace token set
- Token needs access to pyannote models
- Fall back to standard Whisper without `--diarize`

### Cache Issues

**Cache not being used**
- Ensure you're using the same video filename
- Check that output directory contains cached files
- Use `--verbose` to see what's being cached/loaded

**Want to re-run specific steps**
- `--skip-cache-frames`: Re-extract frames
- `--skip-cache-whisper`: Re-run transcription
- `--no-cache`: Force complete reprocessing

## Deprecated Features (kept for reference)

### OCR and Vision Analysis

OCR, Vision and Hybrid screen-text analysis were the original approach but went nowhere. They have been **removed from the CLI** (the `--ocr-engine` / `--use-vision` / `--use-hybrid` flags no longer exist) and now live, unwired, in `meetus/deprecated/` for reference only. The tool always references frames (`--embed-images`) so your LLM reads them directly. The realtime continuation of the idea is the separate `cht` project. See [`INDEX.md`](INDEX.md).

## Project Structure

See [`INDEX.md`](INDEX.md) for the full repo map. In brief:

```
meetus/
├── process_meeting.py          # Main CLI script (entry point)
├── Makefile                    # `make batch` convenience wrapper
├── meetus/                     # Core package
│   ├── workflow.py             # Processing orchestrator
│   ├── frame_extractor.py      # Frame extraction (FFmpeg scene detection)
│   ├── transcript_merger.py    # Transcript + frame-ref merging
│   ├── output_manager.py       # Run dirs (YYYYMMDD-NNN-<stem>) & manifest
│   ├── cache_manager.py        # Per-step caching
│   └── deprecated/             # Old OCR/vision/hybrid analysis (reference only)
├── ctrl/                       # Control plane / operational scripts
│   ├── batch.sh                # Recursive batch runner
│   ├── transcribe_oneoff.sh    # High-quality re-transcription
│   ├── summarize/              # Local-LLM summarization (WIP, on hold)
│   └── cht/                    # Bridge to the realtime `cht` project
├── def/                        # Design/decision notes
├── output/                     # Run directories (gitignored)
├── samples/                    # Sample inputs (gitignored)
└── README.md                   # This file
```

## License

For personal use.