mariano/mediaproc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updated docs

Browse Source
This commit is contained in:
buenosairesam
2026-02-12 19:46:12 -03:00
parent 013587d108
commit 9cead74fb3
4 changed files with 491 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files

150
docs/architecture/04-media-storage.md Normal file
View File

@@ -0,0 +1,150 @@
# Media Storage Architecture
## Overview
MPR separates media into **input** and **output** paths, each independently configurable. File paths are stored **relative to their respective root** to ensure portability between local development and cloud deployments (AWS S3, etc.).
## Storage Strategy
### Input / Output Separation
| Path | Env Var | Purpose |
|------|---------|---------|
| `MEDIA_IN` | `/app/media/in` | Source media files to process |
| `MEDIA_OUT` | `/app/media/out` | Transcoded/trimmed output files |
These can point to different locations or even different servers/buckets in production.
### File Path Storage
- **Database**: Stores only the relative path (e.g., `videos/sample.mp4`)
- **Input Root**: Configurable via `MEDIA_IN` env var
- **Output Root**: Configurable via `MEDIA_OUT` env var
- **Serving**: Base URL configurable via `MEDIA_BASE_URL` env var
### Why Relative Paths?
1. **Portability**: Same database works locally and in cloud
2. **Flexibility**: Easy to switch between storage backends
3. **Simplicity**: No need to update paths when migrating
## Local Development
### Configuration
```bash
MEDIA_IN=/app/media/in
MEDIA_OUT=/app/media/out
```
### File Structure
```
/app/media/
├── in/ # Source files
│ ├── video1.mp4
│ ├── video2.mp4
│ └── subfolder/
│ └── video3.mp4
└── out/ # Transcoded output
├── video1_h264.mp4
└── video2_trimmed.mp4
```
### Database Storage
```
# Source assets (scanned from media/in)
filename: video1.mp4
file_path: video1.mp4
filename: video3.mp4
file_path: subfolder/video3.mp4
```
### URL Serving
- Nginx serves input via `location /media/in { alias /app/media/in; }`
- Nginx serves output via `location /media/out { alias /app/media/out; }`
- Frontend accesses: `http://mpr.local.ar/media/in/video1.mp4`
- Video player: `<video src="/media/in/video1.mp4" />`
## AWS/Cloud Deployment
### S3 Configuration
```bash
# Input and output can be different buckets/paths
MEDIA_IN=s3://source-bucket/media/
MEDIA_OUT=s3://output-bucket/transcoded/
MEDIA_BASE_URL=https://source-bucket.s3.amazonaws.com/media/
```
### S3 Structure
```
s3://source-bucket/media/
├── video1.mp4
└── subfolder/
└── video3.mp4
s3://output-bucket/transcoded/
├── video1_h264.mp4
└── video2_trimmed.mp4
```
### Database Storage (Same!)
```
filename: video1.mp4
file_path: video1.mp4
filename: video3.mp4
file_path: subfolder/video3.mp4
```
## API Endpoints
### Scan Media Folder
```http
POST /api/assets/scan
```
**Behavior:**
1. Recursively scans `MEDIA_IN` directory
2. Finds all video/audio files (mp4, mkv, avi, mov, mp3, wav, etc.)
3. Stores paths **relative to MEDIA_IN**
4. Skips already-registered files (by filename)
5. Returns summary: `{ found, registered, skipped, files }`
### Create Job
```http
POST /api/jobs/
Content-Type: application/json
{
"source_asset_id": "uuid",
"preset_id": "uuid",
"trim_start": 10.0,
"trim_end": 30.0
}
```
**Behavior:**
- Server sets `output_path` using `MEDIA_OUT` + generated filename
- Output goes to the output directory, not alongside source files
## Migration Guide
### Moving from Local to S3
1. **Upload source files to S3:**
```bash
aws s3 sync /app/media/in/ s3://source-bucket/media/
aws s3 sync /app/media/out/ s3://output-bucket/transcoded/
```
2. **Update environment variables:**
```bash
MEDIA_IN=s3://source-bucket/media/
MEDIA_OUT=s3://output-bucket/transcoded/
MEDIA_BASE_URL=https://source-bucket.s3.amazonaws.com/media/
```
3. **Database paths remain unchanged** (already relative)
## Supported File Types
**Video:** `.mp4`, `.mkv`, `.avi`, `.mov`, `.webm`, `.flv`, `.wmv`, `.m4v`
**Audio:** `.mp3`, `.wav`, `.flac`, `.aac`, `.ogg`, `.m4a`

211
docs/architecture/index.html
View File

@@ -1,87 +1,144 @@
<!DOCTYPE html>
<!doctype html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>MPR - Architecture</title>
<link rel="stylesheet" href="styles.css">
</head>
<body>
<h1>MPR - Media Processor</h1>
<p>A web-based media transcoding tool with professional architecture.</p>
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>MPR - Architecture</title>
<link rel="stylesheet" href="styles.css" />
</head>
<body>
<h1>MPR - Media Processor</h1>
<p>
A web-based media transcoding tool with professional architecture.
</p>
<nav>
<a href="#overview">System Overview</a>
<a href="#data-model">Data Model</a>
<a href="#job-flow">Job Flow</a>
</nav>
<nav>
<a href="#overview">System Overview</a>
<a href="#data-model">Data Model</a>
<a href="#job-flow">Job Flow</a>
<a href="#media-storage">Media Storage</a>
</nav>
<h2 id="overview">System Overview</h2>
<div class="diagram-container">
<div class="diagram">
<h3>Architecture</h3>
<object type="image/svg+xml" data="01-system-overview.svg">
<img src="01-system-overview.svg" alt="System Overview">
</object>
<a href="01-system-overview.svg" target="_blank">Open full size</a>
<h2 id="overview">System Overview</h2>
<div class="diagram-container">
<div class="diagram">
<h3>Architecture</h3>
<object type="image/svg+xml" data="01-system-overview.svg">
<img src="01-system-overview.svg" alt="System Overview" />
</object>
<a href="01-system-overview.svg" target="_blank"
>Open full size</a
>
</div>
</div>
</div>
<div class="legend">
<h3>Components</h3>
<ul>
<li><span class="color-box" style="background: #e8f4f8"></span> Reverse Proxy (nginx)</li>
<li><span class="color-box" style="background: #f0f8e8"></span> Application Layer (Django, FastAPI, UI)</li>
<li><span class="color-box" style="background: #fff8e8"></span> Worker Layer (Celery, Lambda)</li>
<li><span class="color-box" style="background: #f8e8f0"></span> Data Layer (PostgreSQL, Redis, SQS)</li>
<li><span class="color-box" style="background: #f0f0f0"></span> Storage (Local FS, S3)</li>
</ul>
</div>
<h2 id="data-model">Data Model</h2>
<div class="diagram-container">
<div class="diagram">
<h3>Entity Relationships</h3>
<object type="image/svg+xml" data="02-data-model.svg">
<img src="02-data-model.svg" alt="Data Model">
</object>
<a href="02-data-model.svg" target="_blank">Open full size</a>
<div class="legend">
<h3>Components</h3>
<ul>
<li>
<span class="color-box" style="background: #e8f4f8"></span>
Reverse Proxy (nginx)
</li>
<li>
<span class="color-box" style="background: #f0f8e8"></span>
Application Layer (Django, FastAPI, UI)
</li>
<li>
<span class="color-box" style="background: #fff8e8"></span>
Worker Layer (Celery, Lambda)
</li>
<li>
<span class="color-box" style="background: #f8e8f0"></span>
Data Layer (PostgreSQL, Redis, SQS)
</li>
<li>
<span class="color-box" style="background: #f0f0f0"></span>
Storage (Local FS, S3)
</li>
</ul>
</div>
</div>
<div class="legend">
<h3>Entities</h3>
<ul>
<li><span class="color-box" style="background: #4a90d9"></span> MediaAsset - Video/audio files with metadata</li>
<li><span class="color-box" style="background: #50b050"></span> TranscodePreset - Encoding configurations</li>
<li><span class="color-box" style="background: #d9534f"></span> TranscodeJob - Processing queue items</li>
</ul>
</div>
<h2 id="job-flow">Job Flow</h2>
<div class="diagram-container">
<div class="diagram">
<h3>Job Lifecycle</h3>
<object type="image/svg+xml" data="03-job-flow.svg">
<img src="03-job-flow.svg" alt="Job Flow">
</object>
<a href="03-job-flow.svg" target="_blank">Open full size</a>
<h2 id="data-model">Data Model</h2>
<div class="diagram-container">
<div class="diagram">
<h3>Entity Relationships</h3>
<object type="image/svg+xml" data="02-data-model.svg">
<img src="02-data-model.svg" alt="Data Model" />
</object>
<a href="02-data-model.svg" target="_blank">Open full size</a>
</div>
</div>
</div>
<div class="legend">
<h3>Job States</h3>
<ul>
<li><span class="color-box" style="background: #ffc107"></span> PENDING - Waiting in queue</li>
<li><span class="color-box" style="background: #17a2b8"></span> PROCESSING - Worker executing</li>
<li><span class="color-box" style="background: #28a745"></span> COMPLETED - Success</li>
<li><span class="color-box" style="background: #dc3545"></span> FAILED - Error occurred</li>
<li><span class="color-box" style="background: #6c757d"></span> CANCELLED - User cancelled</li>
</ul>
</div>
<div class="legend">
<h3>Entities</h3>
<ul>
<li>
<span class="color-box" style="background: #4a90d9"></span>
MediaAsset - Video/audio files with metadata
</li>
<li>
<span class="color-box" style="background: #50b050"></span>
TranscodePreset - Encoding configurations
</li>
<li>
<span class="color-box" style="background: #d9534f"></span>
TranscodeJob - Processing queue items
</li>
</ul>
</div>
<h2>Quick Reference</h2>
<pre><code># Generate SVGs from DOT files
<h2 id="job-flow">Job Flow</h2>
<div class="diagram-container">
<div class="diagram">
<h3>Job Lifecycle</h3>
<object type="image/svg+xml" data="03-job-flow.svg">
<img src="03-job-flow.svg" alt="Job Flow" />
</object>
<a href="03-job-flow.svg" target="_blank">Open full size</a>
</div>
</div>
<div class="legend">
<h3>Job States</h3>
<ul>
<li>
<span class="color-box" style="background: #ffc107"></span>
PENDING - Waiting in queue
</li>
<li>
<span class="color-box" style="background: #17a2b8"></span>
PROCESSING - Worker executing
</li>
<li>
<span class="color-box" style="background: #28a745"></span>
COMPLETED - Success
</li>
<li>
<span class="color-box" style="background: #dc3545"></span>
FAILED - Error occurred
</li>
<li>
<span class="color-box" style="background: #6c757d"></span>
CANCELLED - User cancelled
</li>
</ul>
</div>
<h2 id="media-storage">Media Storage</h2>
<div class="diagram-container">
<p>
MPR separates media into input and output paths for flexible
storage configuration.
</p>
<p>
<a href="04-media-storage.md" target="_blank"
>View Media Storage Documentation →</a
>
</p>
</div>
<h2>Quick Reference</h2>
<pre><code># Generate SVGs from DOT files
dot -Tsvg 01-system-overview.dot -o 01-system-overview.svg
dot -Tsvg 02-data-model.dot -o 02-data-model.svg
dot -Tsvg 03-job-flow.dot -o 03-job-flow.svg
@@ -89,13 +146,13 @@ dot -Tsvg 03-job-flow.dot -o 03-job-flow.svg
# Or generate all at once
for f in *.dot; do dot -Tsvg "$f" -o "${f%.dot}.svg"; done</code></pre>
<h2>Access Points</h2>
<pre><code># Add to /etc/hosts
<h2>Access Points</h2>
<pre><code># Add to /etc/hosts
127.0.0.1 mpr.local.ar
# URLs
http://mpr.local.ar/admin - Django Admin
http://mpr.local.ar/api - FastAPI (docs at /api/docs)
http://mpr.local.ar/ui - Timeline UI</code></pre>
</body>
</body>
</html>