92 lines
3.8 KiB
Markdown
92 lines
3.8 KiB
Markdown
# Distributed System Monitoring Platform
|
|
|
|
## Project Overview
|
|
|
|
A real-time system monitoring platform that streams metrics from multiple machines to a central hub with live web dashboard. Built to demonstrate production microservices patterns (gRPC, FastAPI, streaming, event-driven architecture).
|
|
|
|
**Primary Goal:** Portfolio project demonstrating real-time streaming with gRPC
|
|
**Status:** Working, deployed at sysmonstm.mcrn.ar
|
|
|
|
## Architecture
|
|
|
|
```
|
|
┌─────────────┐ ┌─────────────────────────────────────┐ ┌─────────────┐
|
|
│ Collector │────▶│ Aggregator + Gateway + Redis + TS │────▶│ Edge │────▶ Browser
|
|
│ (mcrn) │gRPC │ (LOCAL) │ WS │ (AWS) │ WS
|
|
└─────────────┘ └─────────────────────────────────────┘ └─────────────┘
|
|
┌─────────────┐ │
|
|
│ Collector │────────────────────┘
|
|
│ (nfrt) │gRPC
|
|
└─────────────┘
|
|
```
|
|
|
|
- **Collectors** (`services/collector/`) - gRPC clients on each monitored machine
|
|
- **Aggregator** (`services/aggregator/`) - gRPC server, stores in Redis/TimescaleDB
|
|
- **Gateway** (`services/gateway/`) - FastAPI, bridges gRPC to WebSocket, forwards to edge
|
|
- **Edge** (`ctrl/edge/`) - Simple WebSocket relay for AWS, serves public dashboard
|
|
|
|
## Directory Structure
|
|
|
|
```
|
|
sms/
|
|
├── services/ # gRPC-based microservices
|
|
│ ├── collector/ # gRPC client, streams to aggregator
|
|
│ ├── aggregator/ # gRPC server, stores in Redis/TimescaleDB
|
|
│ ├── gateway/ # FastAPI, WebSocket, forwards to edge
|
|
│ └── alerts/ # Event subscriber for threshold alerts
|
|
│
|
|
├── ctrl/ # Deployment configurations
|
|
│ ├── dev/ # Full stack docker-compose
|
|
│ └── edge/ # Cloud dashboard (AWS)
|
|
│
|
|
├── proto/ # Protocol Buffer definitions
|
|
├── shared/ # Shared Python modules (config, logging, events)
|
|
└── web/ # Dashboard templates and static files
|
|
```
|
|
|
|
## Running
|
|
|
|
### Local Development
|
|
```bash
|
|
docker compose up
|
|
```
|
|
|
|
### With Edge Forwarding (to AWS)
|
|
```bash
|
|
EDGE_URL=wss://sysmonstm.mcrn.ar/ws docker compose up
|
|
```
|
|
|
|
### Collector on Remote Machine
|
|
```bash
|
|
docker run -d --network host \
|
|
-e AGGREGATOR_URL=<local-ip>:50051 \
|
|
-e MACHINE_ID=$(hostname) \
|
|
registry.mcrn.ar/sysmonstm/collector:latest
|
|
```
|
|
|
|
## Technical Stack
|
|
|
|
- **Python 3.11+**
|
|
- **gRPC** - Collector to aggregator communication (showcased)
|
|
- **FastAPI** - Gateway REST/WebSocket
|
|
- **Redis** - Pub/Sub events, current state cache
|
|
- **TimescaleDB** - Historical metrics storage
|
|
- **WebSocket** - Gateway to edge, edge to browser
|
|
|
|
## Key Files
|
|
|
|
| File | Purpose |
|
|
|------|---------|
|
|
| `proto/metrics.proto` | gRPC service and message definitions |
|
|
| `services/collector/main.py` | gRPC streaming client |
|
|
| `services/aggregator/main.py` | gRPC server, metric processing |
|
|
| `services/gateway/main.py` | WebSocket bridge, edge forwarding |
|
|
| `ctrl/edge/edge.py` | Simple WebSocket relay for AWS |
|
|
|
|
## Portfolio Talking Points
|
|
|
|
- **gRPC streaming** - Efficient binary protocol for real-time metrics
|
|
- **Event-driven** - Redis Pub/Sub decouples processing from delivery
|
|
- **Edge pattern** - Heavy processing local, lightweight relay in cloud
|
|
- **Cost optimization** - ~$10/mo for public dashboard (data transfer, not requests)
|