diff --git a/README.md b/README.md index 504437a..12ed805 100644 --- a/README.md +++ b/README.md @@ -1,21 +1,3 @@ -**If you cloned this repository before August 25, 2025:** - -The commit history has been cleaned up for better readability. If you have a local clone: - -```bash -# Fetch latest changes -git fetch --all --prune - -# Switch to the new main branch -git switch main || git checkout -b main origin/main -git reset --hard origin/main - -# Optional: Clean up old tracking branches -git branch -d webcam # if you have it locally -``` -Original commit history: Check the webcam branch to see the original development history up to commit e790025. - - # Real-Time Video AI Analysis Platform ![Control Panel Overview](def/panel_capture.png) @@ -38,45 +20,13 @@ docker compose up ![System Architecture](def/architecture/architecture_diagram.svg) -**Key Design Patterns:** - -- **Source Adapters** (`streaming/source_adapters.py`) - Abstract webcam vs RTMP input -- **Execution Strategies** (`ai_processing/execution_strategies/`) - Local vs distributed processing -- **Analysis Adapters** (`ai_processing/adapters/`) - Pluggable AI models (CLIP, GCP Vision) -- **Queue Segregation** - Separate Celery workers for different analysis types - -## Code Organization - -``` -├── backend/ -│ ├── streaming/ # Video ingestion (RTMP/Webcam) -│ ├── ai_processing/ # AI analysis pipeline -│ │ ├── adapters/ # Pluggable AI models -│ │ ├── execution_strategies/ # Local/cloud/distributed -│ │ └── tasks.py # Celery workers -│ └── effects/ # Real-time video effects (future) -├── frontend/ # Angular 17+ SPA -├── k8s/ # Kubernetes manifests -└── logos/ # Test images (Apple, Nike, etc.) -``` - ## Tech Stack - **Backend**: Django + Channels, Celery, PostgreSQL, Redis -- **AI/ML**: PyTorch + CLIP, OpenCV, GCP Vision API +- **AI/ML**: PyTorch + CLIP, OpenCV - **Frontend**: Angular 17, WebSockets, HLS.js - **Infrastructure**: Docker, Kubernetes, NGINX -## Features Implemented - -✅ **Real-time logo detection** (CLIP + GCP Vision) -✅ **Live video streaming** (webcam/RTMP → HLS) -✅ **WebSocket overlays** (detection boxes, confidence scores) -✅ **Kubernetes deployment** (auto-scaling, health checks) -✅ **Modular architecture** (adapters, strategies, queues) - -🔄 **In progress**: Visual properties, audio transcription, distributed processing - --- -*This project demonstrates full-stack capabilities: AI/ML integration, real-time systems, cloud-native architecture, and modern web development.* \ No newline at end of file +*This project aims to demonstrate full-stack capabilities: AI/ML integration, real-time systems, cloud-native architecture, and modern web development.* \ No newline at end of file diff --git a/def/README.md b/def/README.md deleted file mode 100644 index d4f56db..0000000 --- a/def/README.md +++ /dev/null @@ -1,73 +0,0 @@ -# Media Analyzer - -Real-time video streaming and AI analysis platform that demonstrates modern cloud-native architecture and machine learning integration. The system ingests RTMP video streams (from sources like OBS), processes them with computer vision AI models, and provides live analysis results through a responsive web dashboard. - -## Features - -- **Video Ingestion**: Accept RTMP streams and convert to HLS for web playback -- **AI Processing**: Real-time object detection (YOLO) and scene analysis (CLIP) on video segments -- **Live Dashboard**: Angular frontend with WebSocket-powered real-time analysis overlays -- **Scalable Architecture**: Kubernetes-deployed microservices with configurable processing modes -- **Cloud Integration**: GCP services integration while maintaining platform agnostic design - -## Tech Stack - -- **Backend**: Django + Django Channels, PostgreSQL, Redis, Celery -- **AI/ML**: OpenCV, YOLO, CLIP, Whisper (Hugging Face Transformers) -- **Frontend**: Angular 17+ with HLS.js video player and Canvas overlays -- **Infrastructure**: Docker containers, Kubernetes, NGINX -- **Streaming**: FFmpeg for RTMPHLS conversion, WebSocket for real-time data - -## Quick Start - -### Option 1: Docker Compose (Development) - -```bash -# Start all services -docker compose up - -# Run migrations (in separate terminal) -docker compose --profile tools up migrate - -# Access the application -# Frontend: http://localhost:4200 -# Backend API: http://localhost:8000 -# RTMP Stream: rtmp://localhost:1935/live -# HLS Stream: http://localhost:8081/hls -``` - -### Option 2: Kubernetes (Production-ready) - -```bash -# Build and push images to local registry -./k8s/build-for-ctlptl.sh - -# Deploy to Kubernetes -kubectl apply -k k8s/overlays/development - -# Check deployment status -kubectl get pods -n media-analyzer - -# Access via port forwarding -kubectl port-forward service/frontend -n media-analyzer 4200:80 -``` - -## Architecture - -- **Django Backend**: Main API server with WebSocket support for real-time communication -- **Celery Workers**: Distributed task processing for AI analysis (logo detection, visual analysis) -- **PostgreSQL**: Primary database for application data and analysis results -- **Redis**: Cache and message broker for Celery tasks -- **Angular Frontend**: Single-page application with real-time video analysis overlays -- **NGINX RTMP**: Stream ingestion server for OBS and other RTMP sources - -## Development - -The system supports both local development with hot reload and production deployment: - -- **Development**: Uses Angular dev server and Django development server -- **Production**: Uses nginx for static files and optimized Docker images - -## Demo - -Stream video from OBS Studio to `rtmp://localhost:1935/live` and watch real-time AI analysis in the web dashboard with live object detection overlays. \ No newline at end of file diff --git a/def/architecture/architecture_diagram.svg b/def/architecture/architecture_diagram.svg index f6fcec8..c356e1d 100644 --- a/def/architecture/architecture_diagram.svg +++ b/def/architecture/architecture_diagram.svg @@ -4,642 +4,642 @@ - - + + MediaAnalyzer - + cluster_legend - -Legend + +Legend cluster_sources - -Video Sources + +Video Sources cluster_adapters - -Source Adapters Pattern + +Source Adapters Pattern cluster_core - -Core Platform + +Core Platform cluster_execution - -Execution Strategies Pattern + +Execution Strategies Pattern cluster_event_sources - -Event Source Pattern + +Event Source Pattern cluster_events - -Event-Driven Processing + +Event-Driven Processing cluster_workers - -Celery Workers + +Celery Workers cluster_ai_adapters - -Analysis Adapters Pattern + +Analysis Adapters Pattern cluster_storage - -Media Storage + +Media Storage cluster_frontend - -Frontend + +Frontend cluster_cloud - -GCP Services + +GCP Services implemented - -✅ Implemented + +✅ Implemented planned - -⏳ Planned + +⏳ Planned code_pattern - -📐 Code Pattern + +📐 Code Pattern service_arch - -⚡ Service/Architecture + +⚡ Service/Architecture webcam - -Webcam + +Webcam webcam_adapter - -WebcamAdapter + +WebcamAdapter webcam->webcam_adapter - - + + rtmp - -RTMP/OBS + +RTMP/OBS rtmp_adapter - -RtmpAdapter + +RtmpAdapter rtmp->rtmp_adapter - - + + files - -File Upload + +File Upload file_adapter - -FileAdapter + +FileAdapter files->file_adapter - - + + base_adapter - -BaseSourceAdapter -(Abstract) + +BaseSourceAdapter +(Abstract) base_adapter->webcam_adapter - - + + base_adapter->rtmp_adapter - - + + base_adapter->file_adapter - - + + django - -Django API -+ Channels -:8000 + +Django API ++ Channels +:8000 webcam_adapter->django - - + + rtmp_adapter->django - - + + file_adapter->django - - + + postgres - -PostgreSQL -Database + +PostgreSQL +Database django->postgres - - + + redis - -Redis -Cache & Broker + +Redis +Cache & Broker django->redis - - + + local_storage - -Local Files -(nginx-served) + +Local Files +(nginx-served) django->local_storage - - + + gcs_storage - -Google Cloud -Storage + +Google Cloud +Storage django->gcs_storage - - + + angular - -Angular 17 SPA -+ WebSocket -:4200 + +Angular 17 SPA ++ WebSocket +:4200 django->angular - - -WebSocket -API + + +WebSocket +API logo_worker - -Logo Detection -Worker -(logo_queue) + +Logo Detection +Worker +(logo_queue) redis->logo_worker - - + + visual_worker - -Visual Properties -Worker -(visual_queue) + +Visual Properties +Worker +(visual_queue) redis->visual_worker - - + + audio_worker - -Audio Transcript -Worker -(audio_queue) + +Audio Transcript +Worker +(audio_queue) redis->audio_worker - - + + text_worker - -Text Recognition -Worker -(text_queue) + +Text Recognition +Worker +(text_queue) redis->text_worker - - + + nginx - -NGINX -Reverse Proxy -:80 + +NGINX +Reverse Proxy +:80 nginx->angular - - + + base_strategy - -BaseExecutionStrategy -(Abstract) + +BaseExecutionStrategy +(Abstract) local_strategy - -LocalStrategy + +LocalStrategy base_strategy->local_strategy - - + + lan_strategy - -LANStrategy + +LANStrategy base_strategy->lan_strategy - - + + cloud_strategy - -CloudStrategy + +CloudStrategy base_strategy->cloud_strategy - - + + clip_adapter - -CLIPAdapter -(Local) + +CLIPAdapter +(Local) local_strategy->clip_adapter - - + + gcp_vision - -GCPVisionAdapter -(Cloud) + +GCPVisionAdapter +(Cloud) local_strategy->gcp_vision - - + + yolo_adapter - -YOLOAdapter -(Planned) + +YOLOAdapter +(Planned) lan_strategy->yolo_adapter - - + + speech_api - -Speech-to-Text API -(Audio Transcript) + +Speech-to-Text API +(Audio Transcript) cloud_strategy->speech_api - - + + base_event_source - -SegmentEventSource -(Abstract) + +SegmentEventSource +(Abstract) file_watcher_source - -FileWatcherEventSource -(Local/LAN) + +FileWatcherEventSource +(Local/LAN) base_event_source->file_watcher_source - - + + cloud_storage_source - -CloudStorageEventSource -(GCS Pub/Sub) + +CloudStorageEventSource +(GCS Pub/Sub) base_event_source->cloud_storage_source - - + + webhook_source - -WebhookEventSource -(External) + +WebhookEventSource +(External) base_event_source->webhook_source - - + + event_source_manager - -EventSourceManager -(Environment-based) + +EventSourceManager +(Environment-based) file_watcher_source->event_source_manager - - + + cloud_storage_source->event_source_manager - - + + webhook_source->event_source_manager - - + + redis_events - -Redis Event Queue -(segment_events) + +Redis Event Queue +(segment_events) event_source_manager->redis_events - - -publishes -events + + +publishes +events event_processor - -Event Processor -(triggers analysis) + +Event Processor +(triggers analysis) redis_events->event_processor - - -consumes -events + + +consumes +events event_processor->logo_worker - - -triggers -analysis + + +triggers +analysis logo_worker->django - - -analysis -results + + +analysis +results logo_worker->local_strategy - - + + visual_worker->lan_strategy - - + + audio_worker->cloud_strategy - - + + text_worker->cloud_strategy - - + + base_ai - -DetectionAdapter -(Abstract) + +DetectionAdapter +(Abstract) base_ai->clip_adapter - - + + base_ai->gcp_vision - - + + base_ai->yolo_adapter - - + + vision_api - -Cloud Vision API -(Logo Detection) + +Cloud Vision API +(Logo Detection) gcp_vision->vision_api - - + + local_storage->file_watcher_source - - -monitors -HLS segments + + +monitors +HLS segments gcs_storage->cloud_storage_source - - -storage -events + + +storage +events hls_player - -HLS.js Player -+ Canvas Overlays + +HLS.js Player ++ Canvas Overlays angular->hls_player - - + + diff --git a/def/architecture/system_architecture.dot b/def/architecture/system_architecture.dot index 06039fc..ae90c30 100644 --- a/def/architecture/system_architecture.dot +++ b/def/architecture/system_architecture.dot @@ -5,7 +5,7 @@ digraph MediaAnalyzer { fontname="Arial"; fontsize=12; ratio=fill; - size="9,10!"; + size="27.5,30!"; ranksep=0.3; nodesep=0.3;