Meeting Processor

Extract screen content from meeting recordings and merge with Whisper/WhisperX transcripts for better AI summarization.

Overview

This tool enhances meeting transcripts by combining:

• Audio transcription (Whisper or WhisperX with speaker diarization)
• Screen content extraction via FFmpeg scene detection
• Frame embedding for direct LLM analysis

The result is a rich, timestamped transcript with embedded screen frames that provides full context for AI summarization.

Installation

1. System Dependencies

FFmpeg (required for scene detection and frame extraction):

# Ubuntu/Debian
sudo apt-get install ffmpeg

# macOS
brew install ffmpeg

2. Python Dependencies

pip install -r requirements.txt

3. Whisper or WhisperX (for audio transcription)

Standard Whisper:

pip install openai-whisper

WhisperX (recommended - includes speaker diarization):

pip install whisperx

For speaker diarization, you'll need a HuggingFace token with access to pyannote models.

Quick Start

Recommended Usage

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

This will:

1. Run WhisperX transcription with speaker diarization
2. Extract frames at scene changes (threshold 10 = moderately sensitive)
3. Create an enhanced transcript with frame file references
4. Save everything to output/ folder

The --embed-images flag adds frame paths to the transcript (e.g., Frame: frames/video_00257.jpg), keeping the transcript small while frames stay in frames/ folder for LLM access.

Re-run with Cached Results

Already ran it once? Re-run instantly using cached results:

# Uses cached transcript and frames
python process_meeting.py samples/meeting.mkv --embed-images

# Skip only specific cached items
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-frames
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper

# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache

Usage Examples

Scene Detection Options

# Default threshold (15)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize

# More sensitive (more frames, threshold: 5)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --diarize

# Less sensitive (fewer frames, threshold: 30)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 30 --diarize

Fixed Interval Extraction (alternative to scene detection)

# Every 10 seconds
python process_meeting.py samples/meeting.mkv --embed-images --interval 10 --diarize

# Every 3 seconds (more detailed)
python process_meeting.py samples/meeting.mkv --embed-images --interval 3 --diarize

Caching Examples

# First run - processes everything
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

# Iterate on scene threshold (reuse whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames

# Re-run whisper only
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper

# Force complete reprocessing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --no-cache

Custom output location

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --output-dir my_outputs/

Enable verbose logging

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --diarize --verbose

Output Files

Each video gets its own timestamped output directory:

output/
└── 20241019_143022-meeting/
    ├── manifest.json                    # Processing configuration
    ├── meeting_enhanced.txt             # Enhanced transcript for AI
    ├── meeting.json                     # Whisper/WhisperX transcript
    └── frames/                          # Extracted video frames
        ├── frame_00001_5.00s.jpg
        ├── frame_00002_10.00s.jpg
        └── ...

Caching Behavior

The tool automatically reuses the most recent output directory for the same video:

• First run: Creates new timestamped directory (e.g., 20241019_143022-meeting/)
• Subsequent runs: Reuses the same directory and cached results
• Cached items: Whisper transcript, extracted frames, analysis results

Fine-grained cache control:

• --no-cache: Force complete reprocessing
• --skip-cache-frames: Re-extract frames only
• --skip-cache-whisper: Re-run transcription only

This allows you to iterate on scene detection thresholds without re-running Whisper!

Workflow for Meeting Analysis

Complete Workflow (One Command!)

python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

Typical Iterative Workflow

# First run - full processing
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize

# Adjust scene threshold (keeps cached whisper transcript)
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames

Example Prompt for Claude

Please summarize this meeting transcript. Pay special attention to:
1. Key decisions made
2. Action items
3. Technical details shown on screen
4. Any metrics or data presented

[Paste enhanced transcript here]

Command Reference

process_meeting.py --help is the source of truth for flags — run it rather than relying on a copy here. The essentials:

• --diarize — WhisperX with speaker diarization (needs a HuggingFace token)
• --embed-images — reference frames in the transcript for the LLM (default behavior)
• --scene-detection / --scene-threshold N — frame extraction (lower = more frames)
• --interval N — fixed-interval extraction instead of scene detection
• --transcript-formats srt,vtt,… — extra transcript formats alongside JSON
• --no-cache / --skip-cache-frames / --skip-cache-whisper — cache control

For batches, prefer make batch IN=<dir> (see INDEX.md).

Tips for Best Results

Scene Detection vs Interval

• Scene detection (--scene-detection): Recommended. Captures frames when content changes. More efficient.
• Interval extraction (--interval N): Alternative for continuous content. Captures every N seconds.

Scene Detection Threshold

• Lower values (5-10): More sensitive, captures more frames
• Default (15): Good balance for most meetings
• Higher values (20-30): Less sensitive, fewer frames

Whisper vs WhisperX

• Whisper (--run-whisper): Standard transcription, fast
• WhisperX (--run-whisper --diarize): Adds speaker identification, requires HuggingFace token

Troubleshooting

Frame Extraction Issues

"No frames extracted"

• Check video file is valid: ffmpeg -i video.mkv
• Try lower scene threshold: --scene-threshold 5
• Try interval extraction: --interval 3
• Check disk space in output directory

Scene detection not working

• Ensure FFmpeg is installed
• Falls back to interval extraction automatically
• Try manual interval: --interval 5

Whisper/WhisperX Issues

WhisperX diarization not working

• Ensure you have a HuggingFace token set
• Token needs access to pyannote models
• Fall back to standard Whisper without --diarize

Cache Issues

Cache not being used

• Ensure you're using the same video filename
• Check that output directory contains cached files
• Use --verbose to see what's being cached/loaded

Want to re-run specific steps

• --skip-cache-frames: Re-extract frames
• --skip-cache-whisper: Re-run transcription
• --no-cache: Force complete reprocessing

Deprecated Features (kept for reference)

OCR and Vision Analysis

OCR, Vision and Hybrid screen-text analysis were the original approach but went nowhere. They have been removed from the CLI (the --ocr-engine / --use-vision / --use-hybrid flags no longer exist) and now live, unwired, in meetus/deprecated/ for reference only. The tool always references frames (--embed-images) so your LLM reads them directly. The realtime continuation of the idea is the separate cht project. See INDEX.md.

Project Structure

See INDEX.md for the full repo map. In brief:

meetus/
├── process_meeting.py          # Main CLI script (entry point)
├── Makefile                    # `make batch` convenience wrapper
├── meetus/                     # Core package
│   ├── workflow.py             # Processing orchestrator
│   ├── frame_extractor.py      # Frame extraction (FFmpeg scene detection)
│   ├── transcript_merger.py    # Transcript + frame-ref merging
│   ├── output_manager.py       # Run dirs (YYYYMMDD-NNN-<stem>) & manifest
│   ├── cache_manager.py        # Per-step caching
│   └── deprecated/             # Old OCR/vision/hybrid analysis (reference only)
├── ctrl/                       # Control plane / operational scripts
│   ├── batch.sh                # Recursive batch runner
│   ├── transcribe_oneoff.sh    # High-quality re-transcription
│   ├── summarize/              # Local-LLM summarization (WIP, on hold)
│   └── cht/                    # Bridge to the realtime `cht` project
├── def/                        # Design/decision notes
├── output/                     # Run directories (gitignored)
├── samples/                    # Sample inputs (gitignored)
└── README.md                   # This file

License

For personal use.