refactor (untested)
This commit is contained in:
84
INDEX.md
Normal file
84
INDEX.md
Normal file
@@ -0,0 +1,84 @@
|
||||
# meetus — repo index
|
||||
|
||||
A standalone, local-first **command-line tool**: turn screen-share meeting/training
|
||||
recordings into a rich, AI-summarizable transcript. The extraction pipeline is
|
||||
deterministic and offline; the summarization step is the only LLM-facing part.
|
||||
|
||||
meetus stands on its own. The separate **`cht`** project does the same thing in
|
||||
realtime; meetus may or may not feed it, so the two are kept formally apart (the
|
||||
cht-facing bits here live under `ctrl/cht/`).
|
||||
|
||||
> This file maps the whole repo. [`README.md`](README.md) is the detailed manual for
|
||||
> the main `process_meeting.py` CLI.
|
||||
|
||||
## Data flow
|
||||
|
||||
```
|
||||
video / audio
|
||||
│ process_meeting.py (deterministic, offline)
|
||||
▼
|
||||
output/<run>/ run folder: YYYYMMDD-NNN-<stem>/
|
||||
├── <stem>.json Whisper/WhisperX transcript (base)
|
||||
├── <stem>.srt / .vtt / ... extra transcript formats (optional)
|
||||
├── frames/ extracted scene/interval frames
|
||||
├── <stem>_enhanced.txt transcript + frame refs ◀── the product
|
||||
└── manifest.json
|
||||
│ ctrl/summarize/*.py (local LLM — WIP, on hold)
|
||||
▼
|
||||
<stem>_summary_simple.md / _reference.md
|
||||
```
|
||||
|
||||
`make batch` (→ `ctrl/batch.sh`) runs the pipeline over a whole directory tree,
|
||||
mirroring the input folder structure into the output.
|
||||
|
||||
## Status legend
|
||||
|
||||
✅ active · 🚧 WIP, on hold · 🗄️ deprecated (kept for reference) · 🧰 one-off / niche · 📄 docs
|
||||
|
||||
## Layout
|
||||
|
||||
```
|
||||
process_meeting.py ✅ the CLI tool — entry point (stays at root)
|
||||
Makefile ✅ batch convenience wrapper
|
||||
meetus/ ✅ core package
|
||||
├─ workflow.py orchestrator (whisper → frames → merge)
|
||||
├─ frame_extractor.py FFmpeg scene-detection / interval frames
|
||||
├─ transcript_merger.py interleave transcript + frame refs → enhanced.txt
|
||||
├─ output_manager.py run-folder naming + manifest.json
|
||||
├─ cache_manager.py per-step caching (skip done work on rerun)
|
||||
└─ deprecated/ 🗄️ old screen-text idea (OCR/vision/hybrid) — unwired, reference only
|
||||
├─ ocr_processor.py (was --ocr-engine)
|
||||
├─ vision_processor.py (was --use-vision)
|
||||
├─ hybrid_processor.py (was --use-hybrid)
|
||||
└─ prompts/ vision context prompts
|
||||
ctrl/ control plane / operational scripts
|
||||
├─ batch.sh ✅ recursive batch runner (mirrors tree, continues past failures)
|
||||
├─ transcribe_oneoff.sh 🧰 high-quality re-transcription over an existing run
|
||||
├─ summarize/ 🚧 last step — local-LLM summarization (WIP, on hold)
|
||||
│ ├─ summarize_simple.py minimal map-and-append; reads every referenced frame
|
||||
│ ├─ compile_meeting.py REFINE-pattern technical reference; on-demand frames
|
||||
│ └─ summarize_meeting.py map→extract(validated facts)→reduce
|
||||
└─ cht/ 🧰 bridge to the separate realtime `cht` project
|
||||
└─ interleave_cht_frames.py whisperx JSON + cht frames/index.json → enhanced.txt
|
||||
def/ 📄 design/decision notes, in order (the feature history)
|
||||
README.md 📄 manual for process_meeting.py
|
||||
INDEX.md 📄 this file
|
||||
MARIAN.md 📄 genesis brainstorm (explains why the deprecated OCR path exists)
|
||||
local-run.sh 📄 personal scratch invocations (gitignored)
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- **Default flow** uses `--embed-images` (frames referenced for the LLM to read) +
|
||||
`--scene-detection --scene-threshold 10 --diarize`. `make batch` adds
|
||||
`--transcript-formats srt` and writes outputs next to the sources. See `README.md`.
|
||||
- **`meetus/deprecated/`** is no longer imported or reachable from the CLI (its flags
|
||||
were removed and `workflow.py` no longer imports it). Kept for reference only; the
|
||||
realtime continuation of the idea is the separate `cht` project.
|
||||
- **`ctrl/summarize/`** scripts are standalone; run them under an env with the `openai`
|
||||
client (e.g. `~/wdir/llm/.venv`) against a local OpenAI-compatible server.
|
||||
|
||||
## Loose ends
|
||||
|
||||
- `README.md`'s "Output Files" example still shows the old run-folder format
|
||||
`YYYYMMDD_HHMMSS-video` (actual: `YYYYMMDD-NNN-<stem>`). Minor; worth a tidy.
|
||||
99
README.md
99
README.md
@@ -104,7 +104,7 @@ python process_meeting.py samples/meeting.mkv --embed-images --interval 3 --diar
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
|
||||
|
||||
# Iterate on scene threshold (reuse whisper transcript)
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames --skip-cache-analysis
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
|
||||
|
||||
# Re-run whisper only
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --skip-cache-whisper
|
||||
@@ -150,7 +150,6 @@ The tool automatically reuses the most recent output directory for the same vide
|
||||
- `--no-cache`: Force complete reprocessing
|
||||
- `--skip-cache-frames`: Re-extract frames only
|
||||
- `--skip-cache-whisper`: Re-run transcription only
|
||||
- `--skip-cache-analysis`: Re-run analysis only
|
||||
|
||||
This allows you to iterate on scene detection thresholds without re-running Whisper!
|
||||
|
||||
@@ -169,7 +168,7 @@ python process_meeting.py samples/meeting.mkv --embed-images --scene-detection -
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 10 --diarize
|
||||
|
||||
# Adjust scene threshold (keeps cached whisper transcript)
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames --skip-cache-analysis
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
|
||||
```
|
||||
|
||||
### Example Prompt for Claude
|
||||
@@ -186,42 +185,17 @@ Please summarize this meeting transcript. Pay special attention to:
|
||||
|
||||
## Command Reference
|
||||
|
||||
```
|
||||
usage: process_meeting.py [-h] [--transcript TRANSCRIPT] [--run-whisper]
|
||||
[--whisper-model {tiny,base,small,medium,large}]
|
||||
[--diarize] [--output OUTPUT] [--output-dir OUTPUT_DIR]
|
||||
[--interval INTERVAL] [--scene-detection]
|
||||
[--scene-threshold SCENE_THRESHOLD]
|
||||
[--embed-images] [--embed-quality EMBED_QUALITY]
|
||||
[--no-cache] [--skip-cache-frames] [--skip-cache-whisper]
|
||||
[--skip-cache-analysis] [--no-deduplicate]
|
||||
[--extract-only] [--format {detailed,compact}]
|
||||
[--verbose] video
|
||||
`process_meeting.py --help` is the source of truth for flags — run it rather than
|
||||
relying on a copy here. The essentials:
|
||||
|
||||
Main Options:
|
||||
video Path to video file
|
||||
--diarize Use WhisperX with speaker diarization
|
||||
--embed-images Add frame file references to transcript (recommended)
|
||||
- `--diarize` — WhisperX with speaker diarization (needs a HuggingFace token)
|
||||
- `--embed-images` — reference frames in the transcript for the LLM (default behavior)
|
||||
- `--scene-detection` / `--scene-threshold N` — frame extraction (lower = more frames)
|
||||
- `--interval N` — fixed-interval extraction instead of scene detection
|
||||
- `--transcript-formats srt,vtt,…` — extra transcript formats alongside JSON
|
||||
- `--no-cache` / `--skip-cache-frames` / `--skip-cache-whisper` — cache control
|
||||
|
||||
Frame Extraction:
|
||||
--scene-detection Use FFmpeg scene detection (recommended)
|
||||
--scene-threshold Detection sensitivity 0-100 (default: 15, lower=more sensitive)
|
||||
--interval Extract frame every N seconds (alternative to scene detection)
|
||||
|
||||
Caching:
|
||||
--no-cache Force complete reprocessing
|
||||
--skip-cache-frames Re-extract frames only
|
||||
--skip-cache-whisper Re-run transcription only
|
||||
--skip-cache-analysis Re-run analysis only
|
||||
|
||||
Other:
|
||||
--run-whisper Run Whisper (without diarization)
|
||||
--whisper-model Whisper model: tiny, base, small, medium, large (default: medium)
|
||||
--transcript, -t Path to existing Whisper transcript (JSON or TXT)
|
||||
--output, -o Output file for enhanced transcript
|
||||
--output-dir Directory for output files (default: output/)
|
||||
--verbose, -v Enable verbose logging
|
||||
```
|
||||
For batches, prefer `make batch IN=<dir>` (see [`INDEX.md`](INDEX.md)).
|
||||
|
||||
## Tips for Best Results
|
||||
|
||||
@@ -238,10 +212,6 @@ Other:
|
||||
- **Whisper** (`--run-whisper`): Standard transcription, fast
|
||||
- **WhisperX** (`--run-whisper --diarize`): Adds speaker identification, requires HuggingFace token
|
||||
|
||||
### Deduplication
|
||||
- Enabled by default - removes similar consecutive frames
|
||||
- Disable with `--no-deduplicate` if slides/screens change subtly
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Frame Extraction Issues
|
||||
@@ -274,44 +244,37 @@ Other:
|
||||
**Want to re-run specific steps**
|
||||
- `--skip-cache-frames`: Re-extract frames
|
||||
- `--skip-cache-whisper`: Re-run transcription
|
||||
- `--skip-cache-analysis`: Re-run analysis
|
||||
- `--no-cache`: Force complete reprocessing
|
||||
|
||||
## Experimental Features
|
||||
## Deprecated Features (kept for reference)
|
||||
|
||||
### OCR and Vision Analysis
|
||||
|
||||
OCR (`--ocr-engine`) and Vision analysis (`--use-vision`) options are available but experimental. The recommended approach is to use `--embed-images` which embeds frame references directly in the transcript, letting your LLM analyze the images.
|
||||
|
||||
```bash
|
||||
# Experimental: OCR extraction
|
||||
python process_meeting.py samples/meeting.mkv --run-whisper --ocr-engine tesseract
|
||||
|
||||
# Experimental: Vision model analysis
|
||||
python process_meeting.py samples/meeting.mkv --run-whisper --use-vision --vision-model llava:13b
|
||||
|
||||
# Experimental: Hybrid OpenCV + OCR
|
||||
python process_meeting.py samples/meeting.mkv --run-whisper --use-hybrid
|
||||
```
|
||||
OCR, Vision and Hybrid screen-text analysis were the original approach but went nowhere. They have been **removed from the CLI** (the `--ocr-engine` / `--use-vision` / `--use-hybrid` flags no longer exist) and now live, unwired, in `meetus/deprecated/` for reference only. The tool always references frames (`--embed-images`) so your LLM reads them directly. The realtime continuation of the idea is the separate `cht` project. See [`INDEX.md`](INDEX.md).
|
||||
|
||||
## Project Structure
|
||||
|
||||
See [`INDEX.md`](INDEX.md) for the full repo map. In brief:
|
||||
|
||||
```
|
||||
meetus/
|
||||
├── meetus/ # Main package
|
||||
│ ├── __init__.py
|
||||
├── process_meeting.py # Main CLI script (entry point)
|
||||
├── Makefile # `make batch` convenience wrapper
|
||||
├── meetus/ # Core package
|
||||
│ ├── workflow.py # Processing orchestrator
|
||||
│ ├── output_manager.py # Output directory & manifest management
|
||||
│ ├── cache_manager.py # Caching logic
|
||||
│ ├── frame_extractor.py # Video frame extraction (FFmpeg scene detection)
|
||||
│ ├── vision_processor.py # Vision model analysis (experimental)
|
||||
│ ├── ocr_processor.py # OCR processing (experimental)
|
||||
│ └── transcript_merger.py # Transcript merging
|
||||
├── process_meeting.py # Main CLI script
|
||||
├── requirements.txt # Python dependencies
|
||||
├── output/ # Timestamped output directories
|
||||
│ └── YYYYMMDD_HHMMSS-video/ # Auto-generated per video
|
||||
├── samples/ # Sample videos (gitignored)
|
||||
│ ├── frame_extractor.py # Frame extraction (FFmpeg scene detection)
|
||||
│ ├── transcript_merger.py # Transcript + frame-ref merging
|
||||
│ ├── output_manager.py # Run dirs (YYYYMMDD-NNN-<stem>) & manifest
|
||||
│ ├── cache_manager.py # Per-step caching
|
||||
│ └── deprecated/ # Old OCR/vision/hybrid analysis (reference only)
|
||||
├── ctrl/ # Control plane / operational scripts
|
||||
│ ├── batch.sh # Recursive batch runner
|
||||
│ ├── transcribe_oneoff.sh # High-quality re-transcription
|
||||
│ ├── summarize/ # Local-LLM summarization (WIP, on hold)
|
||||
│ └── cht/ # Bridge to the realtime `cht` project
|
||||
├── def/ # Design/decision notes
|
||||
├── output/ # Run directories (gitignored)
|
||||
├── samples/ # Sample inputs (gitignored)
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ frames/index.json. Frames are placed at their real timestamps rather
|
||||
than appended at the end.
|
||||
|
||||
Usage:
|
||||
python interleave_cht_frames.py \\
|
||||
python ctrl/cht/interleave_cht_frames.py \\
|
||||
<transcript.json> <cht_frames_index.json> [output.txt]
|
||||
"""
|
||||
import json
|
||||
@@ -18,7 +18,7 @@ OpenAI-compatible server (vLLM or llama.cpp — see ~/wdir/llm/serve.sh); --base
|
||||
swaps it.
|
||||
|
||||
Usage (start `~/wdir/llm/serve.sh qwen-vl` first, then):
|
||||
~/wdir/llm/.venv/bin/python compile_meeting.py \\
|
||||
~/wdir/llm/.venv/bin/python ctrl/summarize/compile_meeting.py \\
|
||||
output/<run>/<stem>_enhanced.txt \\
|
||||
"compile every deployment/data-flow workflow and the system architecture \\
|
||||
as a technical reference; note the [mm:ss] each was shown on screen" \\
|
||||
@@ -22,7 +22,7 @@ summarization failure mode — hallucinated names/facts + long-input drift):
|
||||
|
||||
Usage:
|
||||
# start the local server first (`~/wdir/llm/serve.sh qwen7b`), then:
|
||||
python summarize_meeting.py output/<run>/<stem>_enhanced.txt \\
|
||||
python ctrl/summarize/summarize_meeting.py output/<run>/<stem>_enhanced.txt \\
|
||||
"focus on the names mentioned and their roles, output in English" \\
|
||||
-o output/<run>/summary_en.md
|
||||
|
||||
@@ -8,7 +8,7 @@ Edit SYS below to change the steer.
|
||||
Talks to a local OpenAI-compatible endpoint (ollama / vLLM / llama.cpp).
|
||||
|
||||
Usage:
|
||||
~/wdir/llm/.venv/bin/python summarize_simple.py <stem>_enhanced.txt \\
|
||||
~/wdir/llm/.venv/bin/python ctrl/summarize/summarize_simple.py <stem>_enhanced.txt \\
|
||||
--base-url http://localhost:11434/v1 --model gemma3-27b-16k
|
||||
"""
|
||||
import argparse, base64, io, re, sys
|
||||
@@ -4,7 +4,7 @@
|
||||
# merger so the enhanced transcript is regenerated using the existing frames.
|
||||
#
|
||||
# Usage:
|
||||
# ./transcribe_oneoff.sh <video> [language]
|
||||
# ./ctrl/transcribe_oneoff.sh <video> [language]
|
||||
# language: optional ISO code (es, en). Omit for auto-detect.
|
||||
#
|
||||
# What this does differently from the main pipeline:
|
||||
11
meetus/deprecated/__init__.py
Normal file
11
meetus/deprecated/__init__.py
Normal file
@@ -0,0 +1,11 @@
|
||||
"""
|
||||
Deprecated frame-analysis code, kept for reference only.
|
||||
|
||||
These were the original approach (extract screen text via OCR / vision / hybrid
|
||||
and bake it into the transcript). They were superseded by the --embed-images flow,
|
||||
where frames are referenced and the summarizing LLM reads them directly.
|
||||
|
||||
No longer wired into the CLI: the --use-vision / --use-hybrid / --ocr-engine flags
|
||||
were removed and workflow.py no longer imports this package. Kept for reference only.
|
||||
The realtime continuation of this idea lives in the separate `cht` project.
|
||||
"""
|
||||
@@ -1,6 +1,10 @@
|
||||
"""
|
||||
Hybrid frame analysis: OpenCV text detection + OCR for accurate extraction.
|
||||
Better than pure vision models which tend to hallucinate text content.
|
||||
|
||||
STATUS: deprecated, reference only — the hybrid path was superseded by --embed-images
|
||||
(frames are referenced so the summarizing LLM reads them directly). The --use-hybrid /
|
||||
--hybrid-llm-cleanup flags were removed; this is no longer wired into the CLI. See INDEX.md.
|
||||
"""
|
||||
from typing import List, Tuple, Dict, Optional
|
||||
from pathlib import Path
|
||||
@@ -1,6 +1,10 @@
|
||||
"""
|
||||
OCR processing for extracted video frames.
|
||||
Supports multiple OCR engines and text deduplication.
|
||||
|
||||
STATUS: deprecated, reference only — the OCR path was superseded by --embed-images
|
||||
(frames are referenced so the summarizing LLM reads them directly). The --ocr-engine
|
||||
flag was removed; this is no longer wired into the CLI. See INDEX.md.
|
||||
"""
|
||||
from typing import List, Tuple, Dict, Optional
|
||||
from pathlib import Path
|
||||
@@ -1,6 +1,10 @@
|
||||
"""
|
||||
Vision-based frame analysis using local vision-language models via Ollama.
|
||||
Better than OCR for understanding dashboards, code, and console output.
|
||||
|
||||
STATUS: deprecated, reference only — the vision path was superseded by --embed-images
|
||||
(frames are referenced so the summarizing LLM reads them directly). The --use-vision
|
||||
flag was removed; this is no longer wired into the CLI. See INDEX.md.
|
||||
"""
|
||||
from typing import List, Tuple, Dict, Optional
|
||||
from pathlib import Path
|
||||
@@ -110,8 +110,7 @@ class OutputManager:
|
||||
"outputs": {
|
||||
"frames": str(self.frames_dir.relative_to(self.output_dir)),
|
||||
"enhanced_transcript": f"{self.video_path.stem}_enhanced.txt",
|
||||
"whisper_transcript": f"{self.video_path.stem}.json" if config.get("run_whisper") else None,
|
||||
"analysis": f"{self.video_path.stem}_{'vision' if config.get('use_vision') else 'ocr'}.json"
|
||||
"whisper_transcript": f"{self.video_path.stem}.json" if config.get("run_whisper") else None
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -12,8 +12,6 @@ from typing import Dict, Any, Optional
|
||||
from .output_manager import OutputManager
|
||||
from .cache_manager import CacheManager
|
||||
from .frame_extractor import FrameExtractor
|
||||
from .ocr_processor import OCRProcessor
|
||||
from .vision_processor import VisionProcessor
|
||||
from .transcript_merger import TranscriptMerger
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -51,25 +49,7 @@ class WorkflowConfig:
|
||||
self.scene_threshold = kwargs.get('scene_threshold', 15.0)
|
||||
self.interval = kwargs.get('interval', 5)
|
||||
|
||||
# Analysis options
|
||||
self.use_vision = kwargs.get('use_vision', False)
|
||||
self.use_hybrid = kwargs.get('use_hybrid', False)
|
||||
self.hybrid_llm_cleanup = kwargs.get('hybrid_llm_cleanup', False)
|
||||
self.hybrid_llm_model = kwargs.get('hybrid_llm_model', 'llama3.2:3b')
|
||||
self.vision_model = kwargs.get('vision_model', 'llava:13b')
|
||||
self.vision_context = kwargs.get('vision_context', 'meeting')
|
||||
self.ocr_engine = kwargs.get('ocr_engine', 'tesseract')
|
||||
|
||||
# Validation: can't use both vision and hybrid
|
||||
if self.use_vision and self.use_hybrid:
|
||||
raise ValueError("Cannot use both --use-vision and --use-hybrid. Choose one.")
|
||||
|
||||
# Validation: LLM cleanup requires hybrid mode
|
||||
if self.hybrid_llm_cleanup and not self.use_hybrid:
|
||||
raise ValueError("--hybrid-llm-cleanup requires --use-hybrid")
|
||||
|
||||
# Processing options
|
||||
self.no_deduplicate = kwargs.get('no_deduplicate', False)
|
||||
self.no_cache = kwargs.get('no_cache', False)
|
||||
self.skip_cache_frames = kwargs.get('skip_cache_frames', False)
|
||||
self.skip_cache_whisper = kwargs.get('skip_cache_whisper', False)
|
||||
@@ -124,11 +104,7 @@ class WorkflowConfig:
|
||||
"scene_threshold": self.scene_threshold if self.scene_detection else None
|
||||
},
|
||||
"analysis": {
|
||||
"method": "vision" if self.use_vision else ("hybrid" if self.use_hybrid else "ocr"),
|
||||
"vision_model": self.vision_model if self.use_vision else None,
|
||||
"vision_context": self.vision_context if self.use_vision else None,
|
||||
"ocr_engine": self.ocr_engine if (not self.use_vision) else None,
|
||||
"deduplication": not self.no_deduplicate
|
||||
"method": "embed-images"
|
||||
},
|
||||
"output_format": self.format
|
||||
}
|
||||
@@ -172,18 +148,7 @@ class ProcessingWorkflow:
|
||||
logger.info("=" * 80)
|
||||
logger.info(f"Video: {self.config.video_path.name}")
|
||||
|
||||
# Determine analysis method
|
||||
if self.config.use_vision:
|
||||
analysis_method = f"Vision Model ({self.config.vision_model})"
|
||||
logger.info(f"Analysis: {analysis_method}")
|
||||
logger.info(f"Context: {self.config.vision_context}")
|
||||
elif self.config.use_hybrid:
|
||||
analysis_method = f"Hybrid (OpenCV + {self.config.ocr_engine})"
|
||||
logger.info(f"Analysis: {analysis_method}")
|
||||
else:
|
||||
analysis_method = f"OCR ({self.config.ocr_engine})"
|
||||
logger.info(f"Analysis: {analysis_method}")
|
||||
|
||||
logger.info("Screen content: frame references (embed-images)")
|
||||
logger.info(f"Frame extraction: {'Scene detection' if self.config.scene_detection else f'Every {self.config.interval}s'}")
|
||||
logger.info(f"Caching: {'Disabled' if self.config.no_cache else 'Enabled'}")
|
||||
logger.info("=" * 80)
|
||||
@@ -198,8 +163,8 @@ class ProcessingWorkflow:
|
||||
logger.error("No frames extracted")
|
||||
raise RuntimeError("Frame extraction failed")
|
||||
|
||||
# Step 2: Analyze frames
|
||||
screen_segments = self._analyze_frames(frames_info)
|
||||
# Step 2: Prepare frame references
|
||||
screen_segments = self._prepare_frames(frames_info)
|
||||
|
||||
if self.config.extract_only:
|
||||
logger.info("Done! (extract-only mode)")
|
||||
@@ -243,16 +208,6 @@ class ProcessingWorkflow:
|
||||
raise RuntimeError("Whisper not installed")
|
||||
transcribe_cmd = "whisper"
|
||||
|
||||
# Unload Ollama model to free GPU memory for Whisper (if using vision)
|
||||
if self.config.use_vision:
|
||||
logger.info("Freeing GPU memory for Whisper...")
|
||||
try:
|
||||
subprocess.run(["ollama", "stop", self.config.vision_model],
|
||||
capture_output=True, check=False)
|
||||
logger.info("✓ Ollama model unloaded")
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not unload Ollama model: {e}")
|
||||
|
||||
if use_diarize:
|
||||
logger.info(f"Running WhisperX transcription with diarization (model: {self.config.whisper_model})...")
|
||||
else:
|
||||
@@ -391,155 +346,19 @@ class ProcessingWorkflow:
|
||||
logger.info(f"✓ Extracted {len(frames_info)} frames")
|
||||
return frames_info
|
||||
|
||||
def _analyze_frames(self, frames_info):
|
||||
"""Analyze frames with vision, hybrid, or OCR."""
|
||||
# Skip analysis if just embedding images
|
||||
if self.config.embed_images:
|
||||
logger.info("Step 2: Skipping analysis (images will be embedded)")
|
||||
# Create minimal segments with just frame paths and timestamps
|
||||
def _prepare_frames(self, frames_info):
|
||||
"""Build screen segments that reference each extracted frame by path.
|
||||
|
||||
The transcript merger embeds these references so the summarizing LLM reads
|
||||
the frames directly. This is the only screen-content mode (the old in-process
|
||||
OCR / vision / hybrid analysis lives in meetus/deprecated/, unwired)."""
|
||||
screen_segments = [
|
||||
{
|
||||
'timestamp': timestamp,
|
||||
'text': '', # No text extraction needed
|
||||
'frame_path': frame_path
|
||||
}
|
||||
{'timestamp': timestamp, 'text': '', 'frame_path': frame_path}
|
||||
for frame_path, timestamp in frames_info
|
||||
]
|
||||
logger.info(f"✓ Prepared {len(screen_segments)} frames for embedding")
|
||||
logger.info(f"Step 2: Prepared {len(screen_segments)} frame references")
|
||||
return screen_segments
|
||||
|
||||
# Determine analysis type
|
||||
if self.config.use_vision:
|
||||
analysis_type = 'vision'
|
||||
elif self.config.use_hybrid:
|
||||
analysis_type = 'hybrid'
|
||||
else:
|
||||
analysis_type = 'ocr'
|
||||
|
||||
# Check cache
|
||||
cached_analysis = self.cache_mgr.get_analysis_cache(analysis_type)
|
||||
if cached_analysis:
|
||||
return cached_analysis
|
||||
|
||||
if self.config.use_vision:
|
||||
return self._run_vision_analysis(frames_info)
|
||||
elif self.config.use_hybrid:
|
||||
return self._run_hybrid_analysis(frames_info)
|
||||
else:
|
||||
return self._run_ocr_analysis(frames_info)
|
||||
|
||||
def _run_vision_analysis(self, frames_info):
|
||||
"""Run vision analysis on frames."""
|
||||
logger.info("Step 2: Running vision analysis on extracted frames...")
|
||||
logger.info(f"Loading vision model {self.config.vision_model} to GPU...")
|
||||
|
||||
# Load audio segments for context if transcript exists
|
||||
audio_segments = []
|
||||
transcript_path = self.config.transcript_path or self._get_cached_transcript()
|
||||
|
||||
if transcript_path:
|
||||
transcript_file = Path(transcript_path)
|
||||
if transcript_file.exists():
|
||||
logger.info("Loading audio transcript for context...")
|
||||
merger = TranscriptMerger()
|
||||
audio_segments = merger.load_whisper_transcript(str(transcript_file))
|
||||
logger.info(f"✓ Loaded {len(audio_segments)} audio segments for context")
|
||||
|
||||
try:
|
||||
vision = VisionProcessor(model=self.config.vision_model)
|
||||
screen_segments = vision.process_frames(
|
||||
frames_info,
|
||||
context=self.config.vision_context,
|
||||
deduplicate=not self.config.no_deduplicate,
|
||||
audio_segments=audio_segments
|
||||
)
|
||||
logger.info(f"✓ Analyzed {len(screen_segments)} frames with vision model")
|
||||
|
||||
# Debug: Show sample analysis results
|
||||
if screen_segments:
|
||||
logger.debug(f"First analysis result: timestamp={screen_segments[0].get('timestamp')}, text_length={len(screen_segments[0].get('text', ''))}")
|
||||
logger.debug(f"First analysis text preview: {screen_segments[0].get('text', '')[:200]}...")
|
||||
if len(screen_segments) > 1:
|
||||
logger.debug(f"Last analysis result: timestamp={screen_segments[-1].get('timestamp')}, text_length={len(screen_segments[-1].get('text', ''))}")
|
||||
|
||||
# Cache results
|
||||
self.cache_mgr.save_analysis('vision', screen_segments)
|
||||
return screen_segments
|
||||
|
||||
except ImportError as e:
|
||||
logger.error(f"{e}")
|
||||
raise
|
||||
|
||||
def _get_cached_transcript(self) -> Optional[str]:
|
||||
"""Get cached Whisper transcript if available."""
|
||||
cached = self.cache_mgr.get_whisper_cache()
|
||||
return str(cached) if cached else None
|
||||
|
||||
def _run_hybrid_analysis(self, frames_info):
|
||||
"""Run hybrid analysis on frames (OpenCV + OCR)."""
|
||||
if self.config.hybrid_llm_cleanup:
|
||||
logger.info("Step 2: Running hybrid analysis (OpenCV + OCR + LLM cleanup)...")
|
||||
else:
|
||||
logger.info("Step 2: Running hybrid analysis (OpenCV text detection + OCR)...")
|
||||
|
||||
try:
|
||||
from .hybrid_processor import HybridProcessor
|
||||
|
||||
hybrid = HybridProcessor(
|
||||
ocr_engine=self.config.ocr_engine,
|
||||
use_llm_cleanup=self.config.hybrid_llm_cleanup,
|
||||
llm_model=self.config.hybrid_llm_model
|
||||
)
|
||||
screen_segments = hybrid.process_frames(
|
||||
frames_info,
|
||||
deduplicate=not self.config.no_deduplicate
|
||||
)
|
||||
logger.info(f"✓ Processed {len(screen_segments)} frames with hybrid analysis")
|
||||
|
||||
# Debug: Show sample hybrid results
|
||||
if screen_segments:
|
||||
logger.debug(f"First hybrid result: timestamp={screen_segments[0].get('timestamp')}, text_length={len(screen_segments[0].get('text', ''))}")
|
||||
logger.debug(f"First hybrid text preview: {screen_segments[0].get('text', '')[:200]}...")
|
||||
if len(screen_segments) > 1:
|
||||
logger.debug(f"Last hybrid result: timestamp={screen_segments[-1].get('timestamp')}, text_length={len(screen_segments[-1].get('text', ''))}")
|
||||
|
||||
# Cache results
|
||||
self.cache_mgr.save_analysis('hybrid', screen_segments)
|
||||
return screen_segments
|
||||
|
||||
except ImportError as e:
|
||||
logger.error(f"{e}")
|
||||
raise
|
||||
|
||||
def _run_ocr_analysis(self, frames_info):
|
||||
"""Run OCR analysis on frames."""
|
||||
logger.info("Step 2: Running OCR on extracted frames...")
|
||||
|
||||
try:
|
||||
ocr = OCRProcessor(engine=self.config.ocr_engine)
|
||||
screen_segments = ocr.process_frames(
|
||||
frames_info,
|
||||
deduplicate=not self.config.no_deduplicate
|
||||
)
|
||||
logger.info(f"✓ Processed {len(screen_segments)} frames with OCR")
|
||||
|
||||
# Debug: Show sample OCR results
|
||||
if screen_segments:
|
||||
logger.debug(f"First OCR result: timestamp={screen_segments[0].get('timestamp')}, text_length={len(screen_segments[0].get('text', ''))}")
|
||||
logger.debug(f"First OCR text preview: {screen_segments[0].get('text', '')[:200]}...")
|
||||
if len(screen_segments) > 1:
|
||||
logger.debug(f"Last OCR result: timestamp={screen_segments[-1].get('timestamp')}, text_length={len(screen_segments[-1].get('text', ''))}")
|
||||
|
||||
# Cache results
|
||||
self.cache_mgr.save_analysis('ocr', screen_segments)
|
||||
return screen_segments
|
||||
|
||||
except ImportError as e:
|
||||
logger.error(f"{e}")
|
||||
logger.error(f"To install {self.config.ocr_engine}:")
|
||||
logger.error(f" pip install {self.config.ocr_engine}")
|
||||
raise
|
||||
|
||||
def _merge_transcripts(self, transcript_path, screen_segments):
|
||||
"""Merge audio and screen transcripts."""
|
||||
merger = TranscriptMerger(
|
||||
@@ -586,18 +405,9 @@ class ProcessingWorkflow:
|
||||
|
||||
def _build_result(self, transcript_path=None, screen_segments=None, enhanced_transcript=None):
|
||||
"""Build result dictionary."""
|
||||
# Determine analysis filename
|
||||
if self.config.use_vision:
|
||||
analysis_type = 'vision'
|
||||
elif self.config.use_hybrid:
|
||||
analysis_type = 'hybrid'
|
||||
else:
|
||||
analysis_type = 'ocr'
|
||||
|
||||
return {
|
||||
"output_dir": str(self.output_mgr.output_dir),
|
||||
"transcript": transcript_path,
|
||||
"analysis": f"{self.config.video_path.stem}_{analysis_type}.json",
|
||||
"frames_count": len(screen_segments) if screen_segments else 0,
|
||||
"enhanced_transcript": enhanced_transcript,
|
||||
"manifest": str(self.output_mgr.get_path("manifest.json"))
|
||||
|
||||
@@ -38,14 +38,8 @@ Examples:
|
||||
# Adjust frame extraction quality (lower = smaller files)
|
||||
python process_meeting.py samples/meeting.mkv --run-whisper --embed-images --embed-quality 60 --scene-detection
|
||||
|
||||
# Hybrid approach: OpenCV + OCR (extracts text from frames)
|
||||
python process_meeting.py samples/meeting.mkv --run-whisper --use-hybrid --scene-detection
|
||||
|
||||
# Hybrid + LLM cleanup (best for code formatting)
|
||||
python process_meeting.py samples/meeting.mkv --run-whisper --use-hybrid --hybrid-llm-cleanup --scene-detection
|
||||
|
||||
# Iterate on scene threshold (reuse whisper transcript)
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames --skip-cache-analysis
|
||||
python process_meeting.py samples/meeting.mkv --embed-images --scene-detection --scene-threshold 5 --skip-cache-frames
|
||||
"""
|
||||
)
|
||||
|
||||
@@ -123,45 +117,6 @@ Examples:
|
||||
default=15.0
|
||||
)
|
||||
|
||||
# Analysis options
|
||||
parser.add_argument(
|
||||
'--ocr-engine',
|
||||
choices=['tesseract', 'easyocr', 'paddleocr'],
|
||||
help='OCR engine to use (default: tesseract)',
|
||||
default='tesseract'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--use-vision',
|
||||
action='store_true',
|
||||
help='Use local vision model (Ollama) instead of OCR for better context understanding'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--use-hybrid',
|
||||
action='store_true',
|
||||
help='Use hybrid approach: OpenCV text detection + OCR (more accurate than vision models)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--hybrid-llm-cleanup',
|
||||
action='store_true',
|
||||
help='Use LLM to clean up OCR output and preserve code formatting (requires --use-hybrid)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--hybrid-llm-model',
|
||||
help='LLM model for cleanup (default: llama3.2:3b)',
|
||||
default='llama3.2:3b'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--vision-model',
|
||||
help='Vision model to use with Ollama (default: llava:13b)',
|
||||
default='llava:13b'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--vision-context',
|
||||
choices=['meeting', 'dashboard', 'code', 'console'],
|
||||
help='Context hint for vision analysis (default: meeting)',
|
||||
default='meeting'
|
||||
)
|
||||
|
||||
# Processing options
|
||||
parser.add_argument(
|
||||
'--no-cache',
|
||||
@@ -171,27 +126,17 @@ Examples:
|
||||
parser.add_argument(
|
||||
'--skip-cache-frames',
|
||||
action='store_true',
|
||||
help='Skip cached frames, re-extract from video (but keep whisper/analysis cache)'
|
||||
help='Skip cached frames, re-extract from video (but keep whisper cache)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--skip-cache-whisper',
|
||||
action='store_true',
|
||||
help='Skip cached whisper transcript, re-run transcription (but keep frames/analysis cache)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--skip-cache-analysis',
|
||||
action='store_true',
|
||||
help='Skip cached analysis, re-run OCR/vision (but keep frames/whisper cache)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--no-deduplicate',
|
||||
action='store_true',
|
||||
help='Disable text deduplication'
|
||||
help='Skip cached whisper transcript, re-run transcription (but keep frames cache)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--extract-only',
|
||||
action='store_true',
|
||||
help='Only extract frames and analyze, skip transcript merging'
|
||||
help='Only extract frames + transcript, skip the enhanced-transcript merge'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--format',
|
||||
@@ -202,7 +147,8 @@ Examples:
|
||||
parser.add_argument(
|
||||
'--embed-images',
|
||||
action='store_true',
|
||||
help='Skip OCR/vision analysis and reference frame files directly (faster, lets LLM analyze images)'
|
||||
help='Reference extracted frames in the transcript for the LLM to read '
|
||||
'(now the default behavior; flag kept for compatibility)'
|
||||
)
|
||||
parser.add_argument(
|
||||
'--embed-quality',
|
||||
|
||||
29
tests/__init__.py
Normal file
29
tests/__init__.py
Normal file
@@ -0,0 +1,29 @@
|
||||
"""Smoke + focused tests for meetus.
|
||||
|
||||
Run from the repo root:
|
||||
python -m unittest discover -s tests -t . -v
|
||||
|
||||
This package's import makes the repo importable and provides a `cv2` stub (only
|
||||
cv2 is missing in minimal envs; numpy/PIL are real and not shadowed).
|
||||
"""
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
REPO_ROOT = Path(__file__).resolve().parent.parent
|
||||
STUBS_DIR = Path(__file__).resolve().parent / "_stubs"
|
||||
|
||||
for _p in (str(STUBS_DIR), str(REPO_ROOT)):
|
||||
if _p not in sys.path:
|
||||
sys.path.insert(0, _p)
|
||||
|
||||
|
||||
def subprocess_env(extra=None):
|
||||
"""Env for subprocess tests: repo root + cv2 stub on PYTHONPATH."""
|
||||
env = dict(os.environ)
|
||||
pp = env.get("PYTHONPATH", "")
|
||||
parts = [str(REPO_ROOT), str(STUBS_DIR)] + ([pp] if pp else [])
|
||||
env["PYTHONPATH"] = os.pathsep.join(parts)
|
||||
if extra:
|
||||
env.update(extra)
|
||||
return env
|
||||
2
tests/_stubs/cv2.py
Normal file
2
tests/_stubs/cv2.py
Normal file
@@ -0,0 +1,2 @@
|
||||
# Empty stub so meetus.frame_extractor (which does `import cv2`) imports in test
|
||||
# environments without OpenCV. Tests never exercise cv2 functionality.
|
||||
109
tests/test_batch.py
Normal file
109
tests/test_batch.py
Normal file
@@ -0,0 +1,109 @@
|
||||
"""Deeper tests for ctrl/batch.sh: recursive + space-safe discovery, audio
|
||||
inclusion, structure mirroring, flag forwarding, failure resilience, and the
|
||||
stdin-isolation fix (a greedy child must not stall the batch)."""
|
||||
import os
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from tests import REPO_ROOT
|
||||
|
||||
BATCH = REPO_ROOT / "ctrl" / "batch.sh"
|
||||
|
||||
|
||||
def run_batch(args, env_extra=None):
|
||||
env = dict(os.environ)
|
||||
if env_extra:
|
||||
env.update(env_extra)
|
||||
return subprocess.run(["bash", str(BATCH), *args], cwd=REPO_ROOT, env=env,
|
||||
capture_output=True, text=True)
|
||||
|
||||
|
||||
def stub(d, body):
|
||||
p = d / "pystub"
|
||||
p.write_text("#!/bin/bash\n" + body + "\n")
|
||||
p.chmod(0o755)
|
||||
return str(p)
|
||||
|
||||
|
||||
class TestBatchDry(unittest.TestCase):
|
||||
def test_recursive_spaces_audio_and_mirror(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp = Path(tmp)
|
||||
ind = tmp / "in"
|
||||
(ind / "team a" / "sub one").mkdir(parents=True)
|
||||
(ind / "team a" / "sub one" / "session 1.mkv").touch()
|
||||
(ind / "clip.MP4").touch() # case-insensitive
|
||||
(ind / "radio.ogg").touch() # audio
|
||||
(ind / "notes.txt").touch() # ignored
|
||||
r = run_batch(["-i", str(ind), "-o", str(tmp / "out"), "-n"])
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
out = r.stdout
|
||||
self.assertIn("session 1.mkv", out)
|
||||
self.assertIn("clip.MP4", out)
|
||||
self.assertIn("radio.ogg", out)
|
||||
self.assertNotIn("notes.txt", out)
|
||||
self.assertIn("Found 3 video", out)
|
||||
self.assertIn(str(tmp / "out" / "team a" / "sub one"), out)
|
||||
|
||||
|
||||
class TestBatchRun(unittest.TestCase):
|
||||
def test_forward_flags_and_mirror(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp = Path(tmp)
|
||||
ind = tmp / "in" / "team a"
|
||||
ind.mkdir(parents=True)
|
||||
(ind / "session 1.mkv").touch()
|
||||
r = run_batch(["-i", str(tmp / "in"), "-o", str(tmp / "out"),
|
||||
"--", "--embed-images", "--diarize"],
|
||||
env_extra={"PYTHON": "echo"})
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
line = next(l for l in r.stdout.splitlines() if "process_meeting.py" in l)
|
||||
self.assertIn("--output-dir", line)
|
||||
self.assertIn(str(tmp / "out" / "team a"), line)
|
||||
self.assertIn("--embed-images", line)
|
||||
self.assertIn("session 1.mkv", line)
|
||||
|
||||
def test_empty_forward_does_not_crash(self):
|
||||
# set -u + empty FORWARD array guard.
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp = Path(tmp)
|
||||
ind = tmp / "in"
|
||||
ind.mkdir()
|
||||
(ind / "x.mp4").touch()
|
||||
r = run_batch(["-i", str(ind), "-o", str(tmp / "out")],
|
||||
env_extra={"PYTHON": "echo"})
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertIn("process_meeting.py", r.stdout)
|
||||
|
||||
def test_continues_past_failures(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp = Path(tmp)
|
||||
ind = tmp / "in"
|
||||
ind.mkdir()
|
||||
(ind / "a.mp4").touch()
|
||||
(ind / "b.mp4").touch()
|
||||
r = run_batch(["-i", str(ind), "-o", str(tmp / "out")],
|
||||
env_extra={"PYTHON": stub(tmp, "exit 1")})
|
||||
self.assertNotEqual(r.returncode, 0) # nonzero when any failed
|
||||
self.assertIn("0 ok, 2 failed", r.stdout)
|
||||
self.assertIn("FAILED", r.stderr)
|
||||
|
||||
def test_stdin_isolation_processes_all_items(self):
|
||||
# A child that drains stdin (like ffmpeg) must NOT eat the file list.
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tmp = Path(tmp)
|
||||
ind = tmp / "in"
|
||||
ind.mkdir()
|
||||
for n in ("a.mp4", "b.mp4", "c.mp4"):
|
||||
(ind / n).touch()
|
||||
greedy = stub(tmp, 'shift; cat >/dev/null; echo "DID $1"')
|
||||
r = run_batch(["-i", str(ind), "-o", str(tmp / "out")],
|
||||
env_extra={"PYTHON": greedy})
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertEqual(r.stdout.count("DID "), 3)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
77
tests/test_config.py
Normal file
77
tests/test_config.py
Normal file
@@ -0,0 +1,77 @@
|
||||
"""Deeper tests for WorkflowConfig — the --transcript-formats parsing and the
|
||||
post-cleanup shape (embed-images only; deprecated analysis attrs gone)."""
|
||||
import unittest
|
||||
|
||||
from meetus.workflow import WorkflowConfig
|
||||
|
||||
|
||||
def cfg(**kw):
|
||||
kw.setdefault("video", "sample.mp4")
|
||||
return WorkflowConfig(**kw)
|
||||
|
||||
|
||||
class TestTranscriptFormats(unittest.TestCase):
|
||||
def test_default_is_json_only(self):
|
||||
self.assertEqual(cfg().transcript_formats, ["json"])
|
||||
|
||||
def test_srt_implies_json(self):
|
||||
# User shouldn't have to list json; it's always the base.
|
||||
self.assertEqual(cfg(transcript_formats="srt").transcript_formats,
|
||||
["json", "srt"])
|
||||
|
||||
def test_multiple(self):
|
||||
self.assertEqual(cfg(transcript_formats="srt,txt").transcript_formats,
|
||||
["json", "srt", "txt"])
|
||||
|
||||
def test_all_expands(self):
|
||||
self.assertEqual(set(cfg(transcript_formats="all").transcript_formats),
|
||||
{"json", "srt", "vtt", "txt", "tsv"})
|
||||
|
||||
def test_dedup_and_json_kept(self):
|
||||
self.assertEqual(cfg(transcript_formats="json,srt,srt").transcript_formats,
|
||||
["json", "srt"])
|
||||
|
||||
def test_whitespace_and_case(self):
|
||||
self.assertEqual(cfg(transcript_formats=" SRT , Txt ").transcript_formats,
|
||||
["json", "srt", "txt"])
|
||||
|
||||
def test_invalid_raises(self):
|
||||
with self.assertRaises(ValueError):
|
||||
cfg(transcript_formats="srt,bogus")
|
||||
|
||||
|
||||
class TestDefaults(unittest.TestCase):
|
||||
def test_scalar_defaults(self):
|
||||
c = cfg()
|
||||
self.assertEqual(c.embed_quality, 80)
|
||||
self.assertEqual(c.format, "detailed")
|
||||
self.assertEqual(c.whisper_model, "medium")
|
||||
self.assertEqual(c.scene_threshold, 15.0)
|
||||
|
||||
def test_to_dict_analysis_is_embed_images(self):
|
||||
self.assertEqual(cfg().to_dict()["analysis"]["method"], "embed-images")
|
||||
|
||||
def test_to_dict_whisper_has_diarize_and_formats(self):
|
||||
d = cfg(diarize=True, transcript_formats="srt").to_dict()
|
||||
self.assertTrue(d["whisper"]["diarize"])
|
||||
self.assertEqual(d["whisper"]["transcript_formats"], ["json", "srt"])
|
||||
|
||||
|
||||
class TestDeprecatedRemoved(unittest.TestCase):
|
||||
"""The OCR/vision/hybrid surface was removed in the cleanup."""
|
||||
|
||||
def test_no_deprecated_attrs(self):
|
||||
c = cfg()
|
||||
for attr in ("use_vision", "use_hybrid", "hybrid_llm_cleanup",
|
||||
"hybrid_llm_model", "vision_model", "vision_context",
|
||||
"ocr_engine", "no_deduplicate"):
|
||||
self.assertFalse(hasattr(c, attr), f"{attr} should be removed")
|
||||
|
||||
def test_extra_kwargs_are_ignored(self):
|
||||
# Stray kwargs (e.g. an old flag) must not crash construction.
|
||||
c = cfg(use_vision=True, ocr_engine="tesseract")
|
||||
self.assertFalse(hasattr(c, "use_vision"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
55
tests/test_makefile.py
Normal file
55
tests/test_makefile.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""Smoke for the Makefile wrapper: OUT defaults to IN, the baked-in FLAGS, and
|
||||
the AUDIO_LANG knob (named to avoid the LANG locale env var)."""
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from tests import REPO_ROOT
|
||||
|
||||
|
||||
def make(target, **vars):
|
||||
args = ["make", target] + [f"{k}={v}" for k, v in vars.items()]
|
||||
return subprocess.run(args, cwd=REPO_ROOT, capture_output=True, text=True,
|
||||
env=dict(os.environ))
|
||||
|
||||
|
||||
@unittest.skipUnless(shutil.which("make"), "make not installed")
|
||||
class TestMakefile(unittest.TestCase):
|
||||
def test_dry_out_defaults_to_in(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
ind = Path(tmp) / "in"
|
||||
ind.mkdir()
|
||||
(ind / "x.mp4").touch()
|
||||
r = make("dry", IN=str(ind))
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertIn(f"Output: {ind}", r.stdout)
|
||||
self.assertIn("Found 1 video", r.stdout)
|
||||
|
||||
def test_batch_baked_in_flags(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
ind = Path(tmp) / "in"
|
||||
ind.mkdir()
|
||||
(ind / "x.mp4").touch()
|
||||
r = make("batch", IN=str(ind), PYTHON="echo")
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
line = next(l for l in r.stdout.splitlines() if "process_meeting.py" in l)
|
||||
self.assertIn("--scene-threshold 10", line)
|
||||
self.assertIn("--transcript-formats srt", line)
|
||||
self.assertIn("--embed-images", line)
|
||||
self.assertNotIn("--language", line) # no AUDIO_LANG -> no --language
|
||||
|
||||
def test_audio_lang_appends_language(self):
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
ind = Path(tmp) / "in"
|
||||
ind.mkdir()
|
||||
(ind / "x.mp4").touch()
|
||||
r = make("batch", IN=str(ind), PYTHON="echo", AUDIO_LANG="es")
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertIn("--language es", r.stdout)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
57
tests/test_smoke.py
Normal file
57
tests/test_smoke.py
Normal file
@@ -0,0 +1,57 @@
|
||||
"""Smoke: the wired package imports, deprecated stays unwired, and the cleaned
|
||||
CLI surface is what we expect."""
|
||||
import subprocess
|
||||
import sys
|
||||
import unittest
|
||||
|
||||
from tests import REPO_ROOT, subprocess_env
|
||||
|
||||
|
||||
class TestImports(unittest.TestCase):
|
||||
def test_core_modules_import(self):
|
||||
import meetus # noqa: F401
|
||||
import meetus.workflow # noqa: F401 (frame_extractor → cv2 stub)
|
||||
import meetus.frame_extractor # noqa: F401
|
||||
import meetus.transcript_merger # noqa: F401
|
||||
import meetus.output_manager # noqa: F401
|
||||
import meetus.cache_manager # noqa: F401
|
||||
|
||||
def test_deprecated_importable_for_reference(self):
|
||||
import meetus.deprecated.ocr_processor # noqa: F401
|
||||
import meetus.deprecated.vision_processor # noqa: F401
|
||||
|
||||
def test_workflow_does_not_import_deprecated(self):
|
||||
src = (REPO_ROOT / "meetus" / "workflow.py").read_text()
|
||||
self.assertNotIn("from .deprecated", src)
|
||||
self.assertNotIn("import deprecated", src)
|
||||
|
||||
|
||||
class TestCliHelp(unittest.TestCase):
|
||||
def _help(self, rel):
|
||||
return subprocess.run(
|
||||
[sys.executable, str(REPO_ROOT / rel), "--help"],
|
||||
cwd=REPO_ROOT, env=subprocess_env(), capture_output=True, text=True,
|
||||
)
|
||||
|
||||
def test_process_meeting_help_is_clean(self):
|
||||
r = self._help("process_meeting.py")
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
out = r.stdout
|
||||
for kept in ("--embed-images", "--diarize", "--transcript-formats",
|
||||
"--scene-detection", "--skip-cache-frames", "--extract-only"):
|
||||
self.assertIn(kept, out, f"{kept} should still be offered")
|
||||
for gone in ("--use-vision", "--use-hybrid", "--ocr-engine",
|
||||
"--hybrid-llm", "--vision-model", "--vision-context",
|
||||
"--no-deduplicate", "--skip-cache-analysis"):
|
||||
self.assertNotIn(gone, out, f"{gone} should be removed from the CLI")
|
||||
|
||||
def test_summarizers_offer_output_dir(self):
|
||||
for rel in ("ctrl/summarize/summarize_simple.py",
|
||||
"ctrl/summarize/compile_meeting.py"):
|
||||
r = self._help(rel)
|
||||
self.assertEqual(r.returncode, 0, r.stderr)
|
||||
self.assertIn("--output-dir", r.stdout, rel)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
46
tests/test_summarize.py
Normal file
46
tests/test_summarize.py
Normal file
@@ -0,0 +1,46 @@
|
||||
"""Deeper test for the summarizer --output-dir work: compile_meeting.default_output
|
||||
(strips _enhanced, honors a base dir)."""
|
||||
import importlib.util
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from tests import REPO_ROOT
|
||||
|
||||
|
||||
def _load(rel, name):
|
||||
spec = importlib.util.spec_from_file_location(name, REPO_ROOT / rel)
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(mod)
|
||||
return mod
|
||||
|
||||
|
||||
class TestCompileDefaultOutput(unittest.TestCase):
|
||||
@classmethod
|
||||
def setUpClass(cls):
|
||||
cls.cm = _load("ctrl/summarize/compile_meeting.py", "compile_meeting")
|
||||
|
||||
def test_next_to_transcript_strips_enhanced(self):
|
||||
t = Path("/x/20260101-001-foo/foo_enhanced.txt")
|
||||
self.assertEqual(self.cm.default_output(t, "reference"),
|
||||
Path("/x/20260101-001-foo/foo_reference.md"))
|
||||
|
||||
def test_base_dir_override(self):
|
||||
t = Path("/x/20260101-001-foo/foo_enhanced.txt")
|
||||
self.assertEqual(self.cm.default_output(t, "reference", Path("/out")),
|
||||
Path("/out/foo_reference.md"))
|
||||
|
||||
def test_no_enhanced_suffix(self):
|
||||
t = Path("/x/run/foo.txt")
|
||||
self.assertEqual(self.cm.default_output(t, "reference"),
|
||||
Path("/x/run/foo_reference.md"))
|
||||
|
||||
def test_run_folder_mirroring(self):
|
||||
# How main() builds base_dir for --output-dir: <out>/<transcript's run dir>.
|
||||
t = Path("/runs/20260101-001-foo/foo_enhanced.txt")
|
||||
base = Path("/out") / t.parent.name
|
||||
self.assertEqual(self.cm.default_output(t, "reference", base),
|
||||
Path("/out/20260101-001-foo/foo_reference.md"))
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
Reference in New Issue
Block a user