Compare commits
2 Commits
aws-int
...
da1ff62877
| Author | SHA1 | Date | |
|---|---|---|---|
| da1ff62877 | |||
| 9cead74fb3 |
@@ -17,6 +17,7 @@
|
||||
<a href="#overview">System Overview</a>
|
||||
<a href="#data-model">Data Model</a>
|
||||
<a href="#job-flow">Job Flow</a>
|
||||
<a href="#media-storage">Media Storage</a>
|
||||
</nav>
|
||||
|
||||
<h2 id="overview">System Overview</h2>
|
||||
@@ -141,6 +142,19 @@
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<h2 id="media-storage">Media Storage</h2>
|
||||
<div class="diagram-container">
|
||||
<p>
|
||||
MPR separates media into input and output paths for flexible
|
||||
storage configuration.
|
||||
</p>
|
||||
<p>
|
||||
<a href="04-media-storage.md" target="_blank"
|
||||
>View Media Storage Documentation →</a
|
||||
>
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h2>API Interfaces</h2>
|
||||
<pre><code># REST API
|
||||
http://mpr.local.ar/api/docs - Swagger UI
|
||||
|
||||
232
docs/index.html
Normal file
232
docs/index.html
Normal file
@@ -0,0 +1,232 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>MPR - Architecture</title>
|
||||
<link rel="stylesheet" href="architecture/styles.css" />
|
||||
</head>
|
||||
<body>
|
||||
<h1>MPR - Media Processor</h1>
|
||||
<p>
|
||||
A web-based media transcoding tool with professional architecture.
|
||||
</p>
|
||||
|
||||
<nav>
|
||||
<a href="#overview">System Overview</a>
|
||||
<a href="#data-model">Data Model</a>
|
||||
<a href="#job-flow">Job Flow</a>
|
||||
<a href="#media-storage">Media Storage</a>
|
||||
</nav>
|
||||
|
||||
<h2 id="overview">System Overview</h2>
|
||||
<div class="diagram-container">
|
||||
<div class="diagram">
|
||||
<h3>Architecture</h3>
|
||||
<object
|
||||
type="image/svg+xml"
|
||||
data="architecture/01-system-overview.svg"
|
||||
>
|
||||
<img
|
||||
src="architecture/01-system-overview.svg"
|
||||
alt="System Overview"
|
||||
/>
|
||||
</object>
|
||||
<a href="architecture/01-system-overview.svg" target="_blank"
|
||||
>Open full size</a
|
||||
>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<h3>Components</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<span class="color-box" style="background: #e8f4f8"></span>
|
||||
Reverse Proxy (nginx)
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #f0f8e8"></span>
|
||||
Application Layer (Django, FastAPI, UI)
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #fff8e8"></span>
|
||||
Worker Layer (Celery, Lambda)
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #f8e8f0"></span>
|
||||
Data Layer (PostgreSQL, Redis, SQS)
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #f0f0f0"></span>
|
||||
Storage (Local FS, S3)
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<h2 id="data-model">Data Model</h2>
|
||||
<div class="diagram-container">
|
||||
<div class="diagram">
|
||||
<h3>Entity Relationships</h3>
|
||||
<object
|
||||
type="image/svg+xml"
|
||||
data="architecture/02-data-model.svg"
|
||||
>
|
||||
<img
|
||||
src="architecture/02-data-model.svg"
|
||||
alt="Data Model"
|
||||
/>
|
||||
</object>
|
||||
<a href="architecture/02-data-model.svg" target="_blank"
|
||||
>Open full size</a
|
||||
>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<h3>Entities</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<span class="color-box" style="background: #4a90d9"></span>
|
||||
MediaAsset - Video/audio files with metadata
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #50b050"></span>
|
||||
TranscodePreset - Encoding configurations
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #d9534f"></span>
|
||||
TranscodeJob - Processing queue items
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<h2 id="job-flow">Job Flow</h2>
|
||||
<div class="diagram-container">
|
||||
<div class="diagram">
|
||||
<h3>Job Lifecycle</h3>
|
||||
<object
|
||||
type="image/svg+xml"
|
||||
data="architecture/03-job-flow.svg"
|
||||
>
|
||||
<img src="architecture/03-job-flow.svg" alt="Job Flow" />
|
||||
</object>
|
||||
<a href="architecture/03-job-flow.svg" target="_blank"
|
||||
>Open full size</a
|
||||
>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<h3>Job States</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<span class="color-box" style="background: #ffc107"></span>
|
||||
PENDING - Waiting in queue
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #17a2b8"></span>
|
||||
PROCESSING - Worker executing
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #28a745"></span>
|
||||
COMPLETED - Success
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #dc3545"></span>
|
||||
FAILED - Error occurred
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #6c757d"></span>
|
||||
CANCELLED - User cancelled
|
||||
</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<h2 id="media-storage">Media Storage</h2>
|
||||
<div class="diagram-container">
|
||||
<p>
|
||||
MPR separates media into <strong>input</strong> and
|
||||
<strong>output</strong> paths, each independently configurable.
|
||||
File paths are stored
|
||||
<strong>relative to their respective root</strong> to ensure
|
||||
portability between local development and cloud deployments (AWS
|
||||
S3, etc.).
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<h3>Input / Output Separation</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<span class="color-box" style="background: #4a90d9"></span>
|
||||
<code>MEDIA_IN</code> - Source media files to process
|
||||
</li>
|
||||
<li>
|
||||
<span class="color-box" style="background: #50b050"></span>
|
||||
<code>MEDIA_OUT</code> - Transcoded/trimmed output files
|
||||
</li>
|
||||
</ul>
|
||||
<p><strong>Why Relative Paths?</strong></p>
|
||||
<ul>
|
||||
<li>Portability: Same database works locally and in cloud</li>
|
||||
<li>Flexibility: Easy to switch between storage backends</li>
|
||||
<li>Simplicity: No need to update paths when migrating</li>
|
||||
</ul>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<h3>Local Development</h3>
|
||||
<pre><code>MEDIA_IN=/app/media/in
|
||||
MEDIA_OUT=/app/media/out
|
||||
|
||||
/app/media/
|
||||
├── in/ # Source files
|
||||
│ ├── video1.mp4
|
||||
│ └── subfolder/video3.mp4
|
||||
└── out/ # Transcoded output
|
||||
└── video1_h264.mp4</code></pre>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<h3>AWS/Cloud Deployment</h3>
|
||||
<pre><code>MEDIA_IN=s3://source-bucket/media/
|
||||
MEDIA_OUT=s3://output-bucket/transcoded/
|
||||
MEDIA_BASE_URL=https://source-bucket.s3.amazonaws.com/media/</code></pre>
|
||||
<p>
|
||||
Database paths remain unchanged (already relative). Just upload
|
||||
files to S3 and update environment variables.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div class="legend">
|
||||
<h3>API Endpoints</h3>
|
||||
<ul>
|
||||
<li>
|
||||
<code>POST /api/assets/scan</code> - Recursively scans
|
||||
MEDIA_IN and registers video/audio files
|
||||
</li>
|
||||
<li>
|
||||
<code>POST /api/jobs/</code> - Creates transcoding job with
|
||||
source asset, preset, and optional trim times
|
||||
</li>
|
||||
</ul>
|
||||
<p><strong>Supported File Types:</strong></p>
|
||||
<p>
|
||||
Video: mp4, mkv, avi, mov, webm, flv, wmv, m4v<br />
|
||||
Audio: mp3, wav, flac, aac, ogg, m4a
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<h2>Quick Reference</h2>
|
||||
|
||||
<h2>Access Points</h2>
|
||||
<pre><code># Add to /etc/hosts
|
||||
127.0.0.1 mpr.local.ar
|
||||
|
||||
# URLs
|
||||
http://mpr.local.ar/admin - Django Admin
|
||||
http://mpr.local.ar/api - FastAPI (docs at /api/docs)
|
||||
http://mpr.local.ar/ui - Timeline UI</code></pre>
|
||||
</body>
|
||||
</html>
|
||||
125
docs/media-storage.html
Normal file
125
docs/media-storage.html
Normal file
@@ -0,0 +1,125 @@
|
||||
<h1>Media Storage Architecture</h1>
|
||||
<h2>Overview</h2>
|
||||
<p>MPR separates media into <strong>input</strong> and <strong>output</strong> paths, each independently configurable. File paths are stored <strong>relative to their respective root</strong> to ensure portability between local development and cloud deployments (AWS S3, etc.).</p>
|
||||
<h2>Storage Strategy</h2>
|
||||
<h3>Input / Output Separation</h3>
|
||||
<p>| Path | Env Var | Purpose |
|
||||
|------|---------|---------|
|
||||
| <code>MEDIA_IN</code> | <code>/app/media/in</code> | Source media files to process |
|
||||
| <code>MEDIA_OUT</code> | <code>/app/media/out</code> | Transcoded/trimmed output files |</p>
|
||||
<p>These can point to different locations or even different servers/buckets in production.</p>
|
||||
<h3>File Path Storage</h3>
|
||||
<ul>
|
||||
<li><strong>Database</strong>: Stores only the relative path (e.g., <code>videos/sample.mp4</code>)</li>
|
||||
<li><strong>Input Root</strong>: Configurable via <code>MEDIA_IN</code> env var</li>
|
||||
<li><strong>Output Root</strong>: Configurable via <code>MEDIA_OUT</code> env var</li>
|
||||
<li><strong>Serving</strong>: Base URL configurable via <code>MEDIA_BASE_URL</code> env var</li>
|
||||
</ul>
|
||||
<h3>Why Relative Paths?</h3>
|
||||
<ol>
|
||||
<li><strong>Portability</strong>: Same database works locally and in cloud</li>
|
||||
<li><strong>Flexibility</strong>: Easy to switch between storage backends</li>
|
||||
<li><strong>Simplicity</strong>: No need to update paths when migrating</li>
|
||||
</ol>
|
||||
<h2>Local Development</h2>
|
||||
<h3>Configuration</h3>
|
||||
<p><code>bash
|
||||
MEDIA_IN=/app/media/in
|
||||
MEDIA_OUT=/app/media/out</code></p>
|
||||
<h3>File Structure</h3>
|
||||
<p><code>/app/media/
|
||||
├── in/ # Source files
|
||||
│ ├── video1.mp4
|
||||
│ ├── video2.mp4
|
||||
│ └── subfolder/
|
||||
│ └── video3.mp4
|
||||
└── out/ # Transcoded output
|
||||
├── video1_h264.mp4
|
||||
└── video2_trimmed.mp4</code></p>
|
||||
<h3>Database Storage</h3>
|
||||
<p>```</p>
|
||||
<h1>Source assets (scanned from media/in)</h1>
|
||||
<p>filename: video1.mp4
|
||||
file_path: video1.mp4</p>
|
||||
<p>filename: video3.mp4
|
||||
file_path: subfolder/video3.mp4
|
||||
```</p>
|
||||
<h3>URL Serving</h3>
|
||||
<ul>
|
||||
<li>Nginx serves input via <code>location /media/in { alias /app/media/in; }</code></li>
|
||||
<li>Nginx serves output via <code>location /media/out { alias /app/media/out; }</code></li>
|
||||
<li>Frontend accesses: <code>http://mpr.local.ar/media/in/video1.mp4</code></li>
|
||||
<li>Video player: <code><video src="/media/in/video1.mp4" /></code></li>
|
||||
</ul>
|
||||
<h2>AWS/Cloud Deployment</h2>
|
||||
<h3>S3 Configuration</h3>
|
||||
<p>```bash</p>
|
||||
<h1>Input and output can be different buckets/paths</h1>
|
||||
<p>MEDIA_IN=s3://source-bucket/media/
|
||||
MEDIA_OUT=s3://output-bucket/transcoded/
|
||||
MEDIA_BASE_URL=https://source-bucket.s3.amazonaws.com/media/
|
||||
```</p>
|
||||
<h3>S3 Structure</h3>
|
||||
<p>```
|
||||
s3://source-bucket/media/
|
||||
├── video1.mp4
|
||||
└── subfolder/
|
||||
└── video3.mp4</p>
|
||||
<p>s3://output-bucket/transcoded/
|
||||
├── video1_h264.mp4
|
||||
└── video2_trimmed.mp4
|
||||
```</p>
|
||||
<h3>Database Storage (Same!)</h3>
|
||||
<p>```
|
||||
filename: video1.mp4
|
||||
file_path: video1.mp4</p>
|
||||
<p>filename: video3.mp4
|
||||
file_path: subfolder/video3.mp4
|
||||
```</p>
|
||||
<h2>API Endpoints</h2>
|
||||
<h3>Scan Media Folder</h3>
|
||||
<p><code>http
|
||||
POST /api/assets/scan</code></p>
|
||||
<p><strong>Behavior:</strong>
|
||||
1. Recursively scans <code>MEDIA_IN</code> directory
|
||||
2. Finds all video/audio files (mp4, mkv, avi, mov, mp3, wav, etc.)
|
||||
3. Stores paths <strong>relative to MEDIA_IN</strong>
|
||||
4. Skips already-registered files (by filename)
|
||||
5. Returns summary: <code>{ found, registered, skipped, files }</code></p>
|
||||
<h3>Create Job</h3>
|
||||
<p>```http
|
||||
POST /api/jobs/
|
||||
Content-Type: application/json</p>
|
||||
<p>{
|
||||
"source_asset_id": "uuid",
|
||||
"preset_id": "uuid",
|
||||
"trim_start": 10.0,
|
||||
"trim_end": 30.0
|
||||
}
|
||||
```</p>
|
||||
<p><strong>Behavior:</strong>
|
||||
- Server sets <code>output_path</code> using <code>MEDIA_OUT</code> + generated filename
|
||||
- Output goes to the output directory, not alongside source files</p>
|
||||
<h2>Migration Guide</h2>
|
||||
<h3>Moving from Local to S3</h3>
|
||||
<ol>
|
||||
<li>
|
||||
<p><strong>Upload source files to S3:</strong>
|
||||
<code>bash
|
||||
aws s3 sync /app/media/in/ s3://source-bucket/media/
|
||||
aws s3 sync /app/media/out/ s3://output-bucket/transcoded/</code></p>
|
||||
</li>
|
||||
<li>
|
||||
<p><strong>Update environment variables:</strong>
|
||||
<code>bash
|
||||
MEDIA_IN=s3://source-bucket/media/
|
||||
MEDIA_OUT=s3://output-bucket/transcoded/
|
||||
MEDIA_BASE_URL=https://source-bucket.s3.amazonaws.com/media/</code></p>
|
||||
</li>
|
||||
<li>
|
||||
<p><strong>Database paths remain unchanged</strong> (already relative)</p>
|
||||
</li>
|
||||
</ol>
|
||||
<h2>Supported File Types</h2>
|
||||
<p><strong>Video:</strong> <code>.mp4</code>, <code>.mkv</code>, <code>.avi</code>, <code>.mov</code>, <code>.webm</code>, <code>.flv</code>, <code>.wmv</code>, <code>.m4v</code>
|
||||
<strong>Audio:</strong> <code>.mp3</code>, <code>.wav</code>, <code>.flac</code>, <code>.aac</code>, <code>.ogg</code>, <code>.m4a</code></p>
|
||||
Reference in New Issue
Block a user