Meeting Processor

Extract screen content from meeting recordings and merge with Whisper/WhisperX transcripts for better AI summarization.

Overview

This tool enhances meeting transcripts by combining:

• Audio transcription (Whisper or WhisperX with speaker diarization)
• Screen content extraction via FFmpeg scene detection
• Frame embedding for direct LLM analysis

The result is a rich, timestamped transcript with embedded screen frames that provides full context for AI summarization.

Installation

1. System Dependencies

FFmpeg (required for scene detection and frame extraction):

# Ubuntu/Debian
sudo apt-get install ffmpeg

# macOS
brew install ffmpeg

2. Python Dependencies (uv)

Dependencies live in pyproject.toml as uv feature groups — install only what you need:

uv sync --group meetus                 # meeting pipeline (frame extraction)
uv sync --group doocus                 # document extraction (see doocus/README.md)
uv sync --group doocus --group ocr     # + OCR for images / scanned pdfs
uv run process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize

Groups: meetus, doocus, ocr, pdf-render, deprecated (the last is the unwired OCR/vision path — the only user of ollama, kept out of every default install).

3. Whisper or WhisperX (for audio transcription)

meetus calls these as external CLI tools (like ffmpeg), so they are not uv-managed — install them however suits your machine (often a separate GPU env):

# standard whisper
pip install openai-whisper
# or WhisperX (recommended - adds speaker diarization)
pip install whisperx

For speaker diarization, you'll need a HuggingFace token with access to pyannote models.

Quick Start

Recommended Usage

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

This will:

1. Run WhisperX transcription with speaker diarization
2. Extract frames at scene changes (threshold 10 = moderately sensitive)
3. Create an enhanced transcript with frame file references
4. Save everything to output/ folder

The --embed-images flag adds frame paths to the transcript (e.g., Frame: frames/video_00257.jpg), keeping the transcript small while frames stay in frames/ folder for LLM access.

Re-run with Cached Results

Already ran it once? Re-run instantly using cached results:

# Uses cached transcript and frames
python process_meeting.py samples/meeting.mkv --embed-images

# Skip only specific cached items
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-frames
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper

# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache

Usage Examples

Scene Detection Options

# Default threshold (15)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize

# More sensitive (more frames, threshold: 5)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --diarize

# Less sensitive (fewer frames, threshold: 30)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 30 --diarize

Fixed Interval Extraction (alternative to scene detection)

# Every 10 seconds
python process_meeting.py samples/meeting.mkv --embed-images --interval 10 --diarize

# Every 3 seconds (more detailed)
python process_meeting.py samples/meeting.mkv --embed-images --interval 3 --diarize

Caching Examples

# First run - processes everything
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

# Iterate on scene threshold (reuse whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames

# Re-run whisper only
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper

# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache

Custom output location

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --output-dir my_outputs/

Enable verbose logging

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --verbose

Output Files

Each video gets its own timestamped output directory:

output/
└── 20241019_143022-meeting/
    ├── manifest.json                    # Processing configuration
    ├── meeting_enhanced.txt             # Enhanced transcript for AI
    ├── meeting.json                     # Whisper/WhisperX transcript
    └── frames/                          # Extracted video frames
        ├── frame_00001_5.00s.jpg
        ├── frame_00002_10.00s.jpg
        └── ...

Caching Behavior

The tool automatically reuses the most recent output directory for the same video:

• First run: Creates new timestamped directory (e.g., 20241019_143022-meeting/)
• Subsequent runs: Reuses the same directory and cached results
• Cached items: Whisper transcript, extracted frames, analysis results

Fine-grained cache control:

• --no-cache: Force complete reprocessing
• --skip-cache-frames: Re-extract frames only
• --skip-cache-whisper: Re-run transcription only

This allows you to iterate on scene detection thresholds without re-running Whisper!

Workflow for Meeting Analysis

Complete Workflow (One Command!)

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

Typical Iterative Workflow

# First run - full processing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

# Adjust scene threshold (keeps cached whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames

Example Prompt for Claude

Please summarize this meeting transcript. Pay special attention to:
1. Key decisions made
2. Action items
3. Technical details shown on screen
4. Any metrics or data presented

[Paste enhanced transcript here]

Command Reference

process_meeting.py --help is the source of truth for flags — run it rather than relying on a copy here. The essentials:

• --diarize — WhisperX with speaker diarization (needs a HuggingFace token)
• --embed-images — reference frames in the transcript for the LLM (default behavior)
• --scene-detection / --scene-threshold N — frame extraction (lower = more frames)
• --interval N — fixed-interval extraction instead of scene detection
• --transcript-formats srt,vtt,… — extra transcript formats alongside JSON
• --no-cache / --skip-cache-frames / --skip-cache-whisper — cache control

For batches, prefer make batch IN=<dir> (see INDEX.md).

Tips for Best Results

Scene Detection vs Interval

• Scene detection (--scene-detection): Recommended. Captures frames when content changes. More efficient.
• Interval extraction (--interval N): Alternative for continuous content. Captures every N seconds.

Scene Detection Threshold

• Lower values (5-10): More sensitive, captures more frames
• Default (15): Good balance for most meetings
• Higher values (20-30): Less sensitive, fewer frames

Whisper vs WhisperX

• Whisper (--run-whisper): Standard transcription, fast
• WhisperX (--run-whisper --diarize): Adds speaker identification, requires HuggingFace token

Troubleshooting

Frame Extraction Issues

"No frames extracted"

• Check video file is valid: ffmpeg -i video.mkv
• Try lower scene threshold: --scene-threshold 5
• Try interval extraction: --interval 3
• Check disk space in output directory

Scene detection not working

• Ensure FFmpeg is installed
• Falls back to interval extraction automatically
• Try manual interval: --interval 5

Whisper/WhisperX Issues

WhisperX diarization not working

• Ensure you have a HuggingFace token set
• Token needs access to pyannote models
• Fall back to standard Whisper without --diarize

Cache Issues

Cache not being used

• Ensure you're using the same video filename
• Check that output directory contains cached files
• Use --verbose to see what's being cached/loaded

Want to re-run specific steps

• --skip-cache-frames: Re-extract frames
• --skip-cache-whisper: Re-run transcription
• --no-cache: Force complete reprocessing

Deprecated Features (kept for reference)

OCR and Vision Analysis

OCR, Vision and Hybrid screen-text analysis were the original approach but went nowhere. They have been removed from the CLI (the --ocr-engine / --use-vision / --use-hybrid flags no longer exist) and now live, unwired, in meetus/deprecated/ for reference only. The tool always references frames (--embed-images) so your LLM reads them directly. The realtime continuation of the idea is the separate cht project. See INDEX.md.

Project Structure

See INDEX.md for the full repo map. In brief:

meetus/
├── process_meeting.py          # Main CLI script (entry point)
├── Makefile                    # `make batch` convenience wrapper
├── meetus/                     # Core package
│   ├── workflow.py             # Processing orchestrator
│   ├── frame_extractor.py      # Frame extraction (FFmpeg scene detection)
│   ├── transcript_merger.py    # Transcript + frame-ref merging
│   ├── output_manager.py       # Run dirs (YYYYMMDD-NNN-<stem>) & manifest
│   ├── cache_manager.py        # Per-step caching
│   └── deprecated/             # Old OCR/vision/hybrid analysis (reference only)
├── ctrl/                       # Control plane / operational scripts
│   ├── batch.sh                # Recursive batch runner
│   ├── transcribe_oneoff.sh    # High-quality re-transcription
│   ├── summarize/              # Local-LLM summarization (WIP, on hold)
│   └── cht/                    # Bridge to the realtime `cht` project
├── def/                        # Design/decision notes
├── output/                     # Run directories (gitignored)
├── samples/                    # Sample inputs (gitignored)
└── README.md                   # This file

License

For personal use.