# Distributed System Monitoring Platform

## Project Overview

A real-time system monitoring platform that streams metrics from multiple machines to a central hub with live web dashboard. Built to demonstrate production microservices patterns (gRPC, FastAPI, streaming, event-driven architecture).

**Primary Goal:** Portfolio project demonstrating real-time streaming with gRPC  
**Status:** Working, deployed at sysmonstm.mcrn.ar

## Architecture

```
┌─────────────┐     ┌─────────────────────────────────────┐     ┌─────────────┐
│  Collector  │────▶│  Aggregator + Gateway + Redis + TS  │────▶│    Edge     │────▶ Browser
│   (mcrn)    │gRPC │            (LOCAL)                  │ WS  │   (AWS)     │ WS
└─────────────┘     └─────────────────────────────────────┘     └─────────────┘
┌─────────────┐                    │
│  Collector  │────────────────────┘
│   (nfrt)    │gRPC
└─────────────┘
```

- **Collectors** (`services/collector/`) - gRPC clients on each monitored machine
- **Aggregator** (`services/aggregator/`) - gRPC server, stores in Redis/TimescaleDB
- **Gateway** (`services/gateway/`) - FastAPI, bridges gRPC to WebSocket, forwards to edge
- **Edge** (`ctrl/edge/`) - Simple WebSocket relay for AWS, serves public dashboard

## Directory Structure

```
sms/
├── services/                   # gRPC-based microservices
│   ├── collector/              # gRPC client, streams to aggregator
│   ├── aggregator/             # gRPC server, stores in Redis/TimescaleDB
│   ├── gateway/                # FastAPI, WebSocket, forwards to edge
│   └── alerts/                 # Event subscriber for threshold alerts
│
├── ctrl/                       # Deployment configurations
│   ├── dev/                    # Full stack docker-compose
│   └── edge/                   # Cloud dashboard (AWS)
│
├── proto/                      # Protocol Buffer definitions
├── shared/                     # Shared Python modules (config, logging, events)
└── web/                        # Dashboard templates and static files
```

## Running

### Local Development
```bash
docker compose up
```

### With Edge Forwarding (to AWS)
```bash
EDGE_URL=wss://sysmonstm.mcrn.ar/ws docker compose up
```

### Collector on Remote Machine
```bash
docker run -d --network host \
  -e AGGREGATOR_URL=<local-ip>:50051 \
  -e MACHINE_ID=$(hostname) \
  registry.mcrn.ar/sysmonstm/collector:latest
```

## Technical Stack

- **Python 3.11+**
- **gRPC** - Collector to aggregator communication (showcased)
- **FastAPI** - Gateway REST/WebSocket
- **Redis** - Pub/Sub events, current state cache
- **TimescaleDB** - Historical metrics storage
- **WebSocket** - Gateway to edge, edge to browser

## Key Files

| File | Purpose |
|------|---------|
| `proto/metrics.proto` | gRPC service and message definitions |
| `services/collector/main.py` | gRPC streaming client |
| `services/aggregator/main.py` | gRPC server, metric processing |
| `services/gateway/main.py` | WebSocket bridge, edge forwarding |
| `ctrl/edge/edge.py` | Simple WebSocket relay for AWS |

## Portfolio Talking Points

- **gRPC streaming** - Efficient binary protocol for real-time metrics
- **Event-driven** - Redis Pub/Sub decouples processing from delivery
- **Edge pattern** - Heavy processing local, lightweight relay in cloud
- **Cost optimization** - ~$10/mo for public dashboard (data transfer, not requests)