mariano/mediaproc
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

plug task enqueing properly

Browse Source
This commit is contained in:
buenosairesam
2026-02-06 10:49:05 -03:00
parent 2cf6c89fbb
commit 013587d108
20 changed files with 413 additions and 356 deletions
Download Patch File Download Diff File Expand all files Collapse all files

167
docs/media-storage.md
View File

@@ -2,13 +2,23 @@
## Overview
MPR stores media file paths **relative to the media root** to ensure portability between local development and cloud deployments (AWS S3, etc.).
MPR separates media into **input** and **output** paths, each independently configurable. File paths are stored **relative to their respective root** to ensure portability between local development and cloud deployments (AWS S3, etc.).
## Storage Strategy
### Input / Output Separation
| Path | Env Var | Purpose |
|------|---------|---------|
| `MEDIA_IN` | `/app/media/in` | Source media files to process |
| `MEDIA_OUT` | `/app/media/out` | Transcoded/trimmed output files |
These can point to different locations or even different servers/buckets in production.
### File Path Storage
- **Database**: Stores only the relative path (e.g., `videos/sample.mp4`)
- **Media Root**: Configurable base directory via `MEDIA_ROOT` env var
- **Input Root**: Configurable via `MEDIA_IN` env var
- **Output Root**: Configurable via `MEDIA_OUT` env var
- **Serving**: Base URL configurable via `MEDIA_BASE_URL` env var
### Why Relative Paths?
@@ -20,20 +30,26 @@ MPR stores media file paths **relative to the media root** to ensure portability
### Configuration
```bash
MEDIA_ROOT=/app/media
MEDIA_IN=/app/media/in
MEDIA_OUT=/app/media/out
```
### File Structure
```
/app/media/
├── video1.mp4
├── video2.mp4
└── subfolder/
└── video3.mp4
├── in/ # Source files
│ ├── video1.mp4
│ ├── video2.mp4
│ └── subfolder/
│ └── video3.mp4
└── out/ # Transcoded output
├── video1_h264.mp4
└── video2_trimmed.mp4
```
### Database Storage
```
# Source assets (scanned from media/in)
filename: video1.mp4
file_path: video1.mp4
@@ -42,25 +58,31 @@ file_path: subfolder/video3.mp4
```
### URL Serving
- Nginx serves via `location /media { alias /app/media; }`
- Frontend accesses: `http://mpr.local.ar/media/video1.mp4`
- Video player: `<video src="/media/video1.mp4" />`
- Nginx serves input via `location /media/in { alias /app/media/in; }`
- Nginx serves output via `location /media/out { alias /app/media/out; }`
- Frontend accesses: `http://mpr.local.ar/media/in/video1.mp4`
- Video player: `<video src="/media/in/video1.mp4" />`
## AWS/Cloud Deployment
### S3 Configuration
```bash
MEDIA_ROOT=s3://my-bucket/media/
MEDIA_BASE_URL=https://my-bucket.s3.amazonaws.com/media/
# Input and output can be different buckets/paths
MEDIA_IN=s3://source-bucket/media/
MEDIA_OUT=s3://output-bucket/transcoded/
MEDIA_BASE_URL=https://source-bucket.s3.amazonaws.com/media/
```
### S3 Structure
```
s3://my-bucket/media/
s3://source-bucket/media/
├── video1.mp4
├── video2.mp4
└── subfolder/
└── video3.mp4
s3://output-bucket/transcoded/
├── video1_h264.mp4
└── video2_trimmed.mp4
```
### Database Storage (Same!)
@@ -72,10 +94,6 @@ filename: video3.mp4
file_path: subfolder/video3.mp4
```
### URL Serving
- Frontend prepends `MEDIA_BASE_URL`: `https://my-bucket.s3.amazonaws.com/media/video1.mp4`
- Video player: `<video src="https://my-bucket.s3.amazonaws.com/media/video1.mp4" />`
## API Endpoints
### Scan Media Folder
@@ -84,126 +102,49 @@ POST /api/assets/scan
```
**Behavior:**
1. Recursively scans `MEDIA_ROOT` directory
1. Recursively scans `MEDIA_IN` directory
2. Finds all video/audio files (mp4, mkv, avi, mov, mp3, wav, etc.)
3. Stores paths **relative to MEDIA_ROOT**
3. Stores paths **relative to MEDIA_IN**
4. Skips already-registered files (by filename)
5. Returns summary: `{ found, registered, skipped, files }`
**Example:**
```bash
curl -X POST http://mpr.local.ar/api/assets/scan
```
```json
{
"found": 15,
"registered": 12,
"skipped": 3,
"files": ["video1.mp4", "video2.mp4", ...]
}
```
### Create Asset
### Create Job
```http
POST /api/assets/
POST /api/jobs/
Content-Type: application/json
{
"file_path": "/app/media/video.mp4",
"filename": "video.mp4"
"source_asset_id": "uuid",
"preset_id": "uuid",
"trim_start": 10.0,
"trim_end": 30.0
}
```
**Behavior:**
- Validates file exists
- Converts absolute path to relative (relative to `MEDIA_ROOT`)
- Stores relative path in database
- Server sets `output_path` using `MEDIA_OUT` + generated filename
- Output goes to the output directory, not alongside source files
## Migration Guide
### Moving from Local to S3
1. **Upload files to S3:**
1. **Upload source files to S3:**
```bash
aws s3 sync /app/media/ s3://my-bucket/media/
aws s3 sync /app/media/in/ s3://source-bucket/media/
aws s3 sync /app/media/out/ s3://output-bucket/transcoded/
```
2. **Update environment variables:**
```bash
MEDIA_ROOT=s3://my-bucket/media/
MEDIA_BASE_URL=https://my-bucket.s3.amazonaws.com/media/
MEDIA_IN=s3://source-bucket/media/
MEDIA_OUT=s3://output-bucket/transcoded/
MEDIA_BASE_URL=https://source-bucket.s3.amazonaws.com/media/
```
3. **Database paths remain unchanged** (already relative)
4. **Update frontend** to use `MEDIA_BASE_URL` from config
### Moving from S3 to Local
1. **Download files from S3:**
```bash
aws s3 sync s3://my-bucket/media/ /app/media/
```
2. **Update environment variables:**
```bash
MEDIA_ROOT=/app/media
# Remove MEDIA_BASE_URL or set to /media/
```
3. **Database paths remain unchanged** (already relative)
## Implementation Details
### Backend (FastAPI)
**File path normalization** (`api/routes/assets.py`):
```python
import os
from pathlib import Path
media_root = Path(os.environ.get("MEDIA_ROOT", "/app/media"))
# Convert absolute to relative
file_path = Path("/app/media/subfolder/video.mp4")
rel_path = str(file_path.relative_to(media_root))
# Result: "subfolder/video.mp4"
```
### Frontend (React)
**Current implementation:**
```typescript
<video src={`/media/${asset.file_path}`} />
```
**Future cloud implementation:**
```typescript
const MEDIA_BASE_URL = import.meta.env.VITE_MEDIA_BASE_URL || '/media/';
<video src={`${MEDIA_BASE_URL}${asset.file_path}`} />
```
## Supported File Types
**Video formats:**
- `.mp4`, `.mkv`, `.avi`, `.mov`, `.webm`, `.flv`, `.wmv`, `.m4v`
**Audio formats:**
- `.mp3`, `.wav`, `.flac`, `.aac`, `.ogg`, `.m4a`
## Best Practices
1. **Always use relative paths** when storing file references
2. **Use environment variables** for media root and base URL
3. **Validate file existence** before creating assets
4. **Scan periodically** to discover new files
5. **Use presigned URLs** for S3 private buckets (TODO: implement)
## Future Enhancements
- [ ] S3 presigned URL generation for private buckets
- [ ] CloudFront CDN integration
- [ ] Multi-region S3 replication
- [ ] Automatic metadata extraction on upload
- [ ] Thumbnail generation and storage
**Video:** `.mp4`, `.mkv`, `.avi`, `.mov`, `.webm`, `.flv`, `.wmv`, `.m4v`
**Audio:** `.mp3`, `.wav`, `.flac`, `.aac`, `.ogg`, `.m4a`