update docs
This commit is contained in:
38
README.md
38
README.md
@@ -93,6 +93,27 @@ python schema/generate.py --typescript
|
|||||||
python schema/generate.py --proto
|
python schema/generate.py --proto
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Media Storage
|
||||||
|
|
||||||
|
MPR stores media file paths **relative to the media root** for cloud portability.
|
||||||
|
|
||||||
|
### Local Development
|
||||||
|
- Files: `/app/media/video.mp4`
|
||||||
|
- Stored path: `video.mp4`
|
||||||
|
- Served via: `http://mpr.local.ar/media/video.mp4` (nginx alias)
|
||||||
|
|
||||||
|
### AWS/Cloud Deployment
|
||||||
|
For S3 or cloud storage, set `MEDIA_BASE_URL`:
|
||||||
|
```bash
|
||||||
|
MEDIA_BASE_URL=https://bucket.s3.amazonaws.com/
|
||||||
|
```
|
||||||
|
|
||||||
|
- Files: S3 bucket
|
||||||
|
- Stored path: `video.mp4` (same relative path)
|
||||||
|
- Served via: `https://bucket.s3.amazonaws.com/video.mp4`
|
||||||
|
|
||||||
|
**Scan Endpoint**: `POST /api/assets/scan` recursively scans the media folder and registers new files with relative paths.
|
||||||
|
|
||||||
## Project Structure
|
## Project Structure
|
||||||
|
|
||||||
```
|
```
|
||||||
@@ -105,18 +126,18 @@ mpr/
|
|||||||
├── ctrl/ # Docker & deployment
|
├── ctrl/ # Docker & deployment
|
||||||
│ ├── docker-compose.yml
|
│ ├── docker-compose.yml
|
||||||
│ └── nginx.conf
|
│ └── nginx.conf
|
||||||
├── docs/ # Architecture diagrams
|
├── media/ # Media files (local storage)
|
||||||
├── grpc/ # gRPC server & client
|
├── rpc/ # gRPC server & client
|
||||||
│ └── protos/ # Protobuf definitions (generated)
|
│ └── protos/ # Protobuf definitions (generated)
|
||||||
├── mpr/ # Django project
|
├── mpr/ # Django project
|
||||||
│ └── media_assets/ # Django app
|
│ └── media_assets/ # Django app
|
||||||
├── schema/ # Source of truth
|
├── schema/ # Source of truth
|
||||||
│ └── models/ # Dataclass definitions
|
│ └── models/ # Dataclass definitions
|
||||||
├── ui/ # Frontend
|
├── task/ # Celery job execution
|
||||||
│ └── timeline/ # React app
|
│ ├── executor.py # Executor abstraction
|
||||||
└── worker/ # Job execution
|
│ └── tasks.py # Celery tasks
|
||||||
├── executor.py # Executor abstraction
|
└── ui/ # Frontend
|
||||||
└── tasks.py # Celery tasks
|
└── timeline/ # React app
|
||||||
```
|
```
|
||||||
|
|
||||||
## Environment Variables
|
## Environment Variables
|
||||||
@@ -130,6 +151,9 @@ See `ctrl/.env.template` for all configuration options.
|
|||||||
| `GRPC_HOST` | grpc | gRPC server hostname |
|
| `GRPC_HOST` | grpc | gRPC server hostname |
|
||||||
| `GRPC_PORT` | 50051 | gRPC server port |
|
| `GRPC_PORT` | 50051 | gRPC server port |
|
||||||
| `MPR_EXECUTOR` | local | Executor type (local/lambda) |
|
| `MPR_EXECUTOR` | local | Executor type (local/lambda) |
|
||||||
|
| `MEDIA_ROOT` | /app/media | Media files directory |
|
||||||
|
| `MEDIA_BASE_URL` | /media/ | Base URL for serving media (use S3 URL for cloud) |
|
||||||
|
| `VITE_ALLOWED_HOSTS` | - | Comma-separated allowed hosts for Vite dev server |
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
209
docs/media-storage.md
Normal file
209
docs/media-storage.md
Normal file
@@ -0,0 +1,209 @@
|
|||||||
|
# Media Storage Architecture
|
||||||
|
|
||||||
|
## Overview
|
||||||
|
|
||||||
|
MPR stores media file paths **relative to the media root** to ensure portability between local development and cloud deployments (AWS S3, etc.).
|
||||||
|
|
||||||
|
## Storage Strategy
|
||||||
|
|
||||||
|
### File Path Storage
|
||||||
|
- **Database**: Stores only the relative path (e.g., `videos/sample.mp4`)
|
||||||
|
- **Media Root**: Configurable base directory via `MEDIA_ROOT` env var
|
||||||
|
- **Serving**: Base URL configurable via `MEDIA_BASE_URL` env var
|
||||||
|
|
||||||
|
### Why Relative Paths?
|
||||||
|
1. **Portability**: Same database works locally and in cloud
|
||||||
|
2. **Flexibility**: Easy to switch between storage backends
|
||||||
|
3. **Simplicity**: No need to update paths when migrating
|
||||||
|
|
||||||
|
## Local Development
|
||||||
|
|
||||||
|
### Configuration
|
||||||
|
```bash
|
||||||
|
MEDIA_ROOT=/app/media
|
||||||
|
```
|
||||||
|
|
||||||
|
### File Structure
|
||||||
|
```
|
||||||
|
/app/media/
|
||||||
|
├── video1.mp4
|
||||||
|
├── video2.mp4
|
||||||
|
└── subfolder/
|
||||||
|
└── video3.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Storage
|
||||||
|
```
|
||||||
|
filename: video1.mp4
|
||||||
|
file_path: video1.mp4
|
||||||
|
|
||||||
|
filename: video3.mp4
|
||||||
|
file_path: subfolder/video3.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
### URL Serving
|
||||||
|
- Nginx serves via `location /media { alias /app/media; }`
|
||||||
|
- Frontend accesses: `http://mpr.local.ar/media/video1.mp4`
|
||||||
|
- Video player: `<video src="/media/video1.mp4" />`
|
||||||
|
|
||||||
|
## AWS/Cloud Deployment
|
||||||
|
|
||||||
|
### S3 Configuration
|
||||||
|
```bash
|
||||||
|
MEDIA_ROOT=s3://my-bucket/media/
|
||||||
|
MEDIA_BASE_URL=https://my-bucket.s3.amazonaws.com/media/
|
||||||
|
```
|
||||||
|
|
||||||
|
### S3 Structure
|
||||||
|
```
|
||||||
|
s3://my-bucket/media/
|
||||||
|
├── video1.mp4
|
||||||
|
├── video2.mp4
|
||||||
|
└── subfolder/
|
||||||
|
└── video3.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
### Database Storage (Same!)
|
||||||
|
```
|
||||||
|
filename: video1.mp4
|
||||||
|
file_path: video1.mp4
|
||||||
|
|
||||||
|
filename: video3.mp4
|
||||||
|
file_path: subfolder/video3.mp4
|
||||||
|
```
|
||||||
|
|
||||||
|
### URL Serving
|
||||||
|
- Frontend prepends `MEDIA_BASE_URL`: `https://my-bucket.s3.amazonaws.com/media/video1.mp4`
|
||||||
|
- Video player: `<video src="https://my-bucket.s3.amazonaws.com/media/video1.mp4" />`
|
||||||
|
|
||||||
|
## API Endpoints
|
||||||
|
|
||||||
|
### Scan Media Folder
|
||||||
|
```http
|
||||||
|
POST /api/assets/scan
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior:**
|
||||||
|
1. Recursively scans `MEDIA_ROOT` directory
|
||||||
|
2. Finds all video/audio files (mp4, mkv, avi, mov, mp3, wav, etc.)
|
||||||
|
3. Stores paths **relative to MEDIA_ROOT**
|
||||||
|
4. Skips already-registered files (by filename)
|
||||||
|
5. Returns summary: `{ found, registered, skipped, files }`
|
||||||
|
|
||||||
|
**Example:**
|
||||||
|
```bash
|
||||||
|
curl -X POST http://mpr.local.ar/api/assets/scan
|
||||||
|
```
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"found": 15,
|
||||||
|
"registered": 12,
|
||||||
|
"skipped": 3,
|
||||||
|
"files": ["video1.mp4", "video2.mp4", ...]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
### Create Asset
|
||||||
|
```http
|
||||||
|
POST /api/assets/
|
||||||
|
Content-Type: application/json
|
||||||
|
|
||||||
|
{
|
||||||
|
"file_path": "/app/media/video.mp4",
|
||||||
|
"filename": "video.mp4"
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
**Behavior:**
|
||||||
|
- Validates file exists
|
||||||
|
- Converts absolute path to relative (relative to `MEDIA_ROOT`)
|
||||||
|
- Stores relative path in database
|
||||||
|
|
||||||
|
## Migration Guide
|
||||||
|
|
||||||
|
### Moving from Local to S3
|
||||||
|
|
||||||
|
1. **Upload files to S3:**
|
||||||
|
```bash
|
||||||
|
aws s3 sync /app/media/ s3://my-bucket/media/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Update environment variables:**
|
||||||
|
```bash
|
||||||
|
MEDIA_ROOT=s3://my-bucket/media/
|
||||||
|
MEDIA_BASE_URL=https://my-bucket.s3.amazonaws.com/media/
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Database paths remain unchanged** (already relative)
|
||||||
|
|
||||||
|
4. **Update frontend** to use `MEDIA_BASE_URL` from config
|
||||||
|
|
||||||
|
### Moving from S3 to Local
|
||||||
|
|
||||||
|
1. **Download files from S3:**
|
||||||
|
```bash
|
||||||
|
aws s3 sync s3://my-bucket/media/ /app/media/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Update environment variables:**
|
||||||
|
```bash
|
||||||
|
MEDIA_ROOT=/app/media
|
||||||
|
# Remove MEDIA_BASE_URL or set to /media/
|
||||||
|
```
|
||||||
|
|
||||||
|
3. **Database paths remain unchanged** (already relative)
|
||||||
|
|
||||||
|
## Implementation Details
|
||||||
|
|
||||||
|
### Backend (FastAPI)
|
||||||
|
|
||||||
|
**File path normalization** (`api/routes/assets.py`):
|
||||||
|
```python
|
||||||
|
import os
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
media_root = Path(os.environ.get("MEDIA_ROOT", "/app/media"))
|
||||||
|
|
||||||
|
# Convert absolute to relative
|
||||||
|
file_path = Path("/app/media/subfolder/video.mp4")
|
||||||
|
rel_path = str(file_path.relative_to(media_root))
|
||||||
|
# Result: "subfolder/video.mp4"
|
||||||
|
```
|
||||||
|
|
||||||
|
### Frontend (React)
|
||||||
|
|
||||||
|
**Current implementation:**
|
||||||
|
```typescript
|
||||||
|
<video src={`/media/${asset.file_path}`} />
|
||||||
|
```
|
||||||
|
|
||||||
|
**Future cloud implementation:**
|
||||||
|
```typescript
|
||||||
|
const MEDIA_BASE_URL = import.meta.env.VITE_MEDIA_BASE_URL || '/media/';
|
||||||
|
<video src={`${MEDIA_BASE_URL}${asset.file_path}`} />
|
||||||
|
```
|
||||||
|
|
||||||
|
## Supported File Types
|
||||||
|
|
||||||
|
**Video formats:**
|
||||||
|
- `.mp4`, `.mkv`, `.avi`, `.mov`, `.webm`, `.flv`, `.wmv`, `.m4v`
|
||||||
|
|
||||||
|
**Audio formats:**
|
||||||
|
- `.mp3`, `.wav`, `.flac`, `.aac`, `.ogg`, `.m4a`
|
||||||
|
|
||||||
|
## Best Practices
|
||||||
|
|
||||||
|
1. **Always use relative paths** when storing file references
|
||||||
|
2. **Use environment variables** for media root and base URL
|
||||||
|
3. **Validate file existence** before creating assets
|
||||||
|
4. **Scan periodically** to discover new files
|
||||||
|
5. **Use presigned URLs** for S3 private buckets (TODO: implement)
|
||||||
|
|
||||||
|
## Future Enhancements
|
||||||
|
|
||||||
|
- [ ] S3 presigned URL generation for private buckets
|
||||||
|
- [ ] CloudFront CDN integration
|
||||||
|
- [ ] Multi-region S3 replication
|
||||||
|
- [ ] Automatic metadata extraction on upload
|
||||||
|
- [ ] Thumbnail generation and storage
|
||||||
Reference in New Issue
Block a user