migrated all pawprint work
This commit is contained in:
171
station/monitors/databrowse/README.md
Normal file
171
station/monitors/databrowse/README.md
Normal file
@@ -0,0 +1,171 @@
|
||||
# Data Browse Monitor
|
||||
|
||||
**Test-oriented data navigation for AMAR** - quickly find which users to log in as for different test scenarios.
|
||||
|
||||
## Purpose
|
||||
|
||||
When working on multiple tickets simultaneously, you need to quickly navigate to users in specific data states:
|
||||
- New user with no data
|
||||
- User with pets but no requests
|
||||
- User with pending payment
|
||||
- Vet with active requests
|
||||
- Admin account
|
||||
|
||||
This monitor provides at-a-glance views of the database grouped by test-relevant states.
|
||||
|
||||
## Architecture
|
||||
|
||||
Follows pawprint **book/larder pattern**:
|
||||
- **larder/** contains all data files (schema, views, scenarios)
|
||||
- **main.py** generates SQL queries from view definitions
|
||||
- Two modes: **SQL** (direct queries) and **API** (Django backend, placeholder)
|
||||
|
||||
### Key Concepts
|
||||
|
||||
**Schema** (`larder/schema.json`)
|
||||
- AMAR data model with SQL table mappings
|
||||
- Regular fields (from database columns)
|
||||
- Computed fields (SQL expressions)
|
||||
- Support for multiple graph generators
|
||||
|
||||
**Views** (`larder/views.json`)
|
||||
- Define what to display and how to group it
|
||||
- Each view targets an entity (User, PetOwner, Veterinarian, etc.)
|
||||
- Can group results (e.g., by role, by data state, by availability)
|
||||
- SQL is generated automatically from view configuration
|
||||
|
||||
**Scenarios** (`larder/scenarios.json`)
|
||||
- Test scenarios emerge from actual usage
|
||||
- Format defined, real scenarios added as needed
|
||||
- Links scenarios to specific views with filters
|
||||
|
||||
## Available Views
|
||||
|
||||
1. **users_by_role** - All users grouped by USER/VET/ADMIN for quick login selection
|
||||
2. **petowners_by_state** - Pet owners grouped by data state (has_pets, has_coverage, has_requests, has_turnos)
|
||||
3. **vets_by_availability** - Vets grouped by availability status (available, busy, very busy, no availability)
|
||||
4. **requests_pipeline** - Active service requests grouped by state (similar to turnos monitor)
|
||||
|
||||
## Running Locally
|
||||
|
||||
```bash
|
||||
cd /home/mariano/wdir/ama/pawprint/ward/monitor/data_browse
|
||||
python main.py
|
||||
# Opens on http://localhost:12020
|
||||
```
|
||||
|
||||
Or with uvicorn:
|
||||
```bash
|
||||
uvicorn ward.monitor.data_browse.main:app --port 12020 --reload
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
```bash
|
||||
# Database connection (defaults to local dev)
|
||||
export NEST_NAME=local
|
||||
export DB_HOST=localhost
|
||||
export DB_PORT=5433
|
||||
export DB_NAME=amarback
|
||||
export DB_USER=mariano
|
||||
export DB_PASSWORD=""
|
||||
```
|
||||
|
||||
## API Endpoints
|
||||
|
||||
```
|
||||
GET / # Landing page
|
||||
GET /view/{view_slug} # View display (HTML)
|
||||
GET /health # Health check
|
||||
GET /api/views # List all views (JSON)
|
||||
GET /api/view/{view_slug} # View data (JSON)
|
||||
GET /api/schema # Data model schema (JSON)
|
||||
GET /api/scenarios # Test scenarios (JSON)
|
||||
```
|
||||
|
||||
## Adding New Views
|
||||
|
||||
Edit `larder/views.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "my_new_view",
|
||||
"title": "My New View",
|
||||
"slug": "my-new-view",
|
||||
"description": "Description of what this shows",
|
||||
"mode": "sql",
|
||||
"entity": "PetOwner",
|
||||
"group_by": "some_field",
|
||||
"fields": ["id", "first_name", "last_name", "email"],
|
||||
"display_fields": {
|
||||
"id": {"label": "ID", "width": "60px"},
|
||||
"first_name": {"label": "Name", "width": "120px"}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The SQL query is automatically generated from:
|
||||
- Entity definition in `schema.json` (table name, columns)
|
||||
- Fields list (regular + computed fields)
|
||||
- Group by configuration (if specified)
|
||||
- Filters (if specified)
|
||||
|
||||
## Adding Computed Fields
|
||||
|
||||
Edit `larder/schema.json` in the entity definition:
|
||||
|
||||
```json
|
||||
"computed": {
|
||||
"has_pets": {
|
||||
"description": "Has at least one pet",
|
||||
"sql": "EXISTS (SELECT 1 FROM mascotas_pet WHERE petowner_id = mascotas_petowner.id AND deleted = false)"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Computed fields can be used in views just like regular fields.
|
||||
|
||||
## Adding Test Scenarios
|
||||
|
||||
As you identify test patterns, add them to `larder/scenarios.json`:
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "User with Pending Payment",
|
||||
"slug": "user-pending-payment",
|
||||
"description": "User with accepted request awaiting payment",
|
||||
"role": "USER",
|
||||
"entity": "ServiceRequest",
|
||||
"view": "requests_pipeline",
|
||||
"filters": {
|
||||
"state": ["vet_accepted", "in_progress_pay"],
|
||||
"has_payment": false
|
||||
},
|
||||
"test_cases": [
|
||||
"Payment flow (MercadoPago)",
|
||||
"Payment reminders",
|
||||
"Payment timeout"
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
data_browse/
|
||||
├── larder/
|
||||
│ ├── .larder # Larder marker (book pattern)
|
||||
│ ├── schema.json # AMAR data model with SQL mappings
|
||||
│ ├── views.json # View configurations
|
||||
│ └── scenarios.json # Test scenarios
|
||||
├── main.py # FastAPI app
|
||||
├── index.html # Landing page
|
||||
├── view.html # View display template
|
||||
└── README.md # This file
|
||||
```
|
||||
|
||||
## Status
|
||||
|
||||
**Current:** SQL mode fully implemented, ready for local testing
|
||||
**Next:** Test with local database, refine views based on usage
|
||||
**Future:** See `workbench/data_browse_roadmap.md`
|
||||
0
station/monitors/databrowse/depot/.depot
Normal file
0
station/monitors/databrowse/depot/.depot
Normal file
13
station/monitors/databrowse/depot/scenarios.json
Normal file
13
station/monitors/databrowse/depot/scenarios.json
Normal file
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"scenarios": [
|
||||
{
|
||||
"name": "Example Scenario",
|
||||
"description": "Example test scenario for databrowse",
|
||||
"steps": [
|
||||
"Navigate to users view",
|
||||
"Verify user list is displayed",
|
||||
"Filter by active status"
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
27
station/monitors/databrowse/depot/schema.json
Normal file
27
station/monitors/databrowse/depot/schema.json
Normal file
@@ -0,0 +1,27 @@
|
||||
{
|
||||
"name": "example",
|
||||
"description": "Example schema for databrowse. Replace with room-specific schema.",
|
||||
"tables": {
|
||||
"users": {
|
||||
"description": "Example users table",
|
||||
"columns": {
|
||||
"id": { "type": "integer", "primary_key": true },
|
||||
"username": { "type": "string" },
|
||||
"email": { "type": "string" },
|
||||
"is_active": { "type": "boolean" },
|
||||
"created_at": { "type": "datetime" }
|
||||
}
|
||||
},
|
||||
"items": {
|
||||
"description": "Example items table",
|
||||
"columns": {
|
||||
"id": { "type": "integer", "primary_key": true },
|
||||
"name": { "type": "string" },
|
||||
"description": { "type": "text" },
|
||||
"user_id": { "type": "integer", "foreign_key": "users.id" },
|
||||
"status": { "type": "string" },
|
||||
"created_at": { "type": "datetime" }
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
38
station/monitors/databrowse/depot/views.json
Normal file
38
station/monitors/databrowse/depot/views.json
Normal file
@@ -0,0 +1,38 @@
|
||||
{
|
||||
"views": [
|
||||
{
|
||||
"slug": "users",
|
||||
"title": "All Users",
|
||||
"description": "List of all users",
|
||||
"table": "users",
|
||||
"fields": ["id", "username", "email", "is_active", "created_at"],
|
||||
"order_by": "-created_at"
|
||||
},
|
||||
{
|
||||
"slug": "active-users",
|
||||
"title": "Active Users",
|
||||
"description": "Users with active status",
|
||||
"table": "users",
|
||||
"fields": ["id", "username", "email", "created_at"],
|
||||
"where": "is_active = true",
|
||||
"order_by": "username"
|
||||
},
|
||||
{
|
||||
"slug": "items",
|
||||
"title": "All Items",
|
||||
"description": "List of all items",
|
||||
"table": "items",
|
||||
"fields": ["id", "name", "status", "user_id", "created_at"],
|
||||
"order_by": "-created_at"
|
||||
},
|
||||
{
|
||||
"slug": "items-by-status",
|
||||
"title": "Items by Status",
|
||||
"description": "Items grouped by status",
|
||||
"table": "items",
|
||||
"fields": ["id", "name", "user_id", "created_at"],
|
||||
"group_by": "status",
|
||||
"order_by": "-created_at"
|
||||
}
|
||||
]
|
||||
}
|
||||
345
station/monitors/databrowse/index.html
Normal file
345
station/monitors/databrowse/index.html
Normal file
@@ -0,0 +1,345 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>Databrowse · {{ room_name }}</title>
|
||||
<style>
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family:
|
||||
system-ui,
|
||||
-apple-system,
|
||||
sans-serif;
|
||||
background: #111827;
|
||||
color: #f3f4f6;
|
||||
min-height: 100vh;
|
||||
padding: 1.5rem;
|
||||
}
|
||||
|
||||
header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 0 0 1.5rem;
|
||||
border-bottom: 1px solid #374151;
|
||||
margin-bottom: 2rem;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: 1.75rem;
|
||||
font-weight: 600;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.75rem;
|
||||
}
|
||||
|
||||
.room-badge {
|
||||
font-size: 0.75rem;
|
||||
background: #374151;
|
||||
padding: 0.25rem 0.6rem;
|
||||
border-radius: 4px;
|
||||
color: #9ca3af;
|
||||
}
|
||||
|
||||
.subtitle {
|
||||
font-size: 0.9rem;
|
||||
color: #9ca3af;
|
||||
margin-top: 0.5rem;
|
||||
}
|
||||
|
||||
main {
|
||||
max-width: 1400px;
|
||||
margin: 0 auto;
|
||||
}
|
||||
|
||||
.section {
|
||||
margin-bottom: 2.5rem;
|
||||
}
|
||||
|
||||
.section-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
h2 {
|
||||
font-size: 1.25rem;
|
||||
font-weight: 600;
|
||||
color: #60a5fa;
|
||||
}
|
||||
|
||||
.count {
|
||||
font-size: 0.85rem;
|
||||
color: #6b7280;
|
||||
}
|
||||
|
||||
/* Card Grid */
|
||||
.card-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(320px, 1fr));
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.card {
|
||||
background: #1f2937;
|
||||
border-radius: 8px;
|
||||
padding: 1.25rem;
|
||||
border: 1px solid #374151;
|
||||
transition: all 0.2s;
|
||||
cursor: pointer;
|
||||
}
|
||||
|
||||
.card:hover {
|
||||
background: #252f3f;
|
||||
border-color: #60a5fa;
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
|
||||
.card-title {
|
||||
font-size: 1.1rem;
|
||||
font-weight: 600;
|
||||
color: #f3f4f6;
|
||||
margin-bottom: 0.5rem;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.card-mode {
|
||||
font-size: 0.65rem;
|
||||
background: #374151;
|
||||
padding: 0.15rem 0.4rem;
|
||||
border-radius: 3px;
|
||||
color: #9ca3af;
|
||||
text-transform: uppercase;
|
||||
}
|
||||
|
||||
.card-mode.sql {
|
||||
background: #065f46;
|
||||
color: #10b981;
|
||||
}
|
||||
|
||||
.card-mode.api {
|
||||
background: #1e3a8a;
|
||||
color: #60a5fa;
|
||||
}
|
||||
|
||||
.card-mode.graph {
|
||||
background: #581c87;
|
||||
color: #c084fc;
|
||||
}
|
||||
|
||||
.card-description {
|
||||
font-size: 0.85rem;
|
||||
color: #d1d5db;
|
||||
margin-bottom: 1rem;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.card-meta {
|
||||
display: flex;
|
||||
gap: 1rem;
|
||||
font-size: 0.75rem;
|
||||
color: #6b7280;
|
||||
}
|
||||
|
||||
.card-meta-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.3rem;
|
||||
}
|
||||
|
||||
.card-meta-label {
|
||||
color: #9ca3af;
|
||||
}
|
||||
|
||||
/* Empty State */
|
||||
.empty {
|
||||
text-align: center;
|
||||
color: #6b7280;
|
||||
padding: 3rem;
|
||||
font-size: 0.9rem;
|
||||
background: #1f2937;
|
||||
border-radius: 8px;
|
||||
border: 1px dashed #374151;
|
||||
}
|
||||
|
||||
/* Footer */
|
||||
footer {
|
||||
margin-top: 3rem;
|
||||
padding-top: 1.5rem;
|
||||
border-top: 1px solid #374151;
|
||||
font-size: 0.75rem;
|
||||
color: #6b7280;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
}
|
||||
|
||||
footer a {
|
||||
color: #60a5fa;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
footer a:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
/* Links */
|
||||
a {
|
||||
color: inherit;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
/* Scenario specific styles */
|
||||
.scenario-card {
|
||||
border-left: 3px solid #60a5fa;
|
||||
}
|
||||
|
||||
.scenario-card .card-title {
|
||||
color: #60a5fa;
|
||||
}
|
||||
|
||||
.card-filters {
|
||||
margin-top: 0.75rem;
|
||||
font-size: 0.75rem;
|
||||
color: #9ca3af;
|
||||
background: #111827;
|
||||
padding: 0.5rem;
|
||||
border-radius: 4px;
|
||||
}
|
||||
|
||||
.filter-item {
|
||||
display: inline-block;
|
||||
margin-right: 0.75rem;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<div>
|
||||
<h1>
|
||||
Databrowse
|
||||
<span class="room-badge">{{ room_name }}</span>
|
||||
</h1>
|
||||
<p class="subtitle">
|
||||
Test-oriented data navigation for AMAR - find the right
|
||||
user/scenario
|
||||
</p>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
<!-- Views Section -->
|
||||
<section class="section">
|
||||
<div class="section-header">
|
||||
<h2>Views</h2>
|
||||
<span class="count">{{ views|length }} available</span>
|
||||
</div>
|
||||
|
||||
{% if views|length == 0 %}
|
||||
<div class="empty">
|
||||
No views configured. Add views to larder/views.json
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="card-grid">
|
||||
{% for view in views %}
|
||||
<a href="/view/{{ view.slug }}" class="card">
|
||||
<div class="card-title">
|
||||
{{ view.title }}
|
||||
<span class="card-mode {{ view.mode }}"
|
||||
>{{ view.mode }}</span
|
||||
>
|
||||
</div>
|
||||
<div class="card-description">
|
||||
{{ view.description }}
|
||||
</div>
|
||||
<div class="card-meta">
|
||||
<span class="card-meta-item">
|
||||
<span class="card-meta-label">Entity:</span>
|
||||
{{ view.entity }}
|
||||
</span>
|
||||
{% if view.group_by %}
|
||||
<span class="card-meta-item">
|
||||
<span class="card-meta-label">Group:</span>
|
||||
{{ view.group_by }}
|
||||
</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
</a>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</section>
|
||||
|
||||
<!-- Scenarios Section -->
|
||||
<section class="section">
|
||||
<div class="section-header">
|
||||
<h2>Test Scenarios</h2>
|
||||
<span class="count">{{ scenarios|length }} defined</span>
|
||||
</div>
|
||||
|
||||
{% if scenarios|length == 0 %}
|
||||
<div class="empty">
|
||||
No scenarios defined yet. Scenarios emerge from usage and
|
||||
conversations.
|
||||
<br />Add them to larder/scenarios.json as you identify test
|
||||
patterns.
|
||||
</div>
|
||||
{% else %}
|
||||
<div class="card-grid">
|
||||
{% for scenario in scenarios %} {% if not scenario._example
|
||||
%}
|
||||
<a
|
||||
href="/view/{{ scenario.view }}?scenario={{ scenario.slug }}"
|
||||
class="card scenario-card"
|
||||
>
|
||||
<div class="card-title">{{ scenario.name }}</div>
|
||||
<div class="card-description">
|
||||
{{ scenario.description }}
|
||||
</div>
|
||||
<div class="card-meta">
|
||||
<span class="card-meta-item">
|
||||
<span class="card-meta-label">Role:</span>
|
||||
{{ scenario.role }}
|
||||
</span>
|
||||
{% if scenario.priority %}
|
||||
<span class="card-meta-item">
|
||||
<span class="card-meta-label">Priority:</span>
|
||||
{{ scenario.priority }}
|
||||
</span>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% if scenario.filters %}
|
||||
<div class="card-filters">
|
||||
{% for key, value in scenario.filters.items() %}
|
||||
<span class="filter-item"
|
||||
>{{ key }}: {{ value }}</span
|
||||
>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</a>
|
||||
{% endif %} {% endfor %}
|
||||
</div>
|
||||
{% endif %}
|
||||
</section>
|
||||
</main>
|
||||
|
||||
<footer>
|
||||
<span>Data Browse Monitor v0.1.0</span>
|
||||
<div>
|
||||
<a href="/health">/health</a> ·
|
||||
<a href="/api/schema">/api/schema</a> ·
|
||||
<a href="/api/views">/api/views</a> ·
|
||||
<a href="/api/scenarios">/api/scenarios</a>
|
||||
</div>
|
||||
</footer>
|
||||
</body>
|
||||
</html>
|
||||
406
station/monitors/databrowse/main.py
Normal file
406
station/monitors/databrowse/main.py
Normal file
@@ -0,0 +1,406 @@
|
||||
"""
|
||||
Databrowse Monitor - Test-oriented data navigation.
|
||||
|
||||
Navigate users, entities, and data to find the right test scenario.
|
||||
Room-agnostic: configure via environment or cfg/<room>/databrowse/depot/
|
||||
|
||||
Run standalone:
|
||||
python main.py
|
||||
|
||||
Or use uvicorn:
|
||||
uvicorn main:app --port 12020 --reload
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.responses import HTMLResponse, JSONResponse
|
||||
from fastapi.templating import Jinja2Templates
|
||||
from sqlalchemy import create_engine, text
|
||||
from sqlalchemy.engine import Engine
|
||||
|
||||
app = FastAPI(title="Databrowse Monitor", version="0.1.0")
|
||||
|
||||
# Paths
|
||||
BASE_DIR = Path(__file__).parent
|
||||
DEPOT_DIR = Path(os.getenv("DATABROWSE_DEPOT", str(BASE_DIR / "depot")))
|
||||
templates = Jinja2Templates(directory=str(BASE_DIR))
|
||||
|
||||
# =============================================================================
|
||||
# ROOM CONFIG - Pluggable environment targeting
|
||||
# =============================================================================
|
||||
# Default room: local development database
|
||||
# Override with env vars or future room selector UI
|
||||
|
||||
ROOM_CONFIG = {
|
||||
"name": os.getenv("ROOM_NAME", "local"),
|
||||
"db": {
|
||||
"host": os.getenv("DB_HOST", "localhost"),
|
||||
"port": os.getenv("DB_PORT", "5432"),
|
||||
"name": os.getenv("DB_NAME", "database"),
|
||||
"user": os.getenv("DB_USER", "user"),
|
||||
"password": os.getenv("DB_PASSWORD", ""),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def get_db_url() -> str:
|
||||
"""Build database URL from room config."""
|
||||
db = ROOM_CONFIG["db"]
|
||||
return f"postgresql://{db['user']}:{db['password']}@{db['host']}:{db['port']}/{db['name']}"
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# DEPOT DATA - Load from JSON files
|
||||
# =============================================================================
|
||||
|
||||
_schema: Optional[Dict[str, Any]] = None
|
||||
_views: Optional[Dict[str, Any]] = None
|
||||
_scenarios: Optional[Dict[str, Any]] = None
|
||||
|
||||
|
||||
def load_depot_file(filename: str) -> Dict[str, Any]:
|
||||
"""Load JSON file from depot directory."""
|
||||
path = DEPOT_DIR / filename
|
||||
if not path.exists():
|
||||
return {}
|
||||
with open(path) as f:
|
||||
return json.load(f)
|
||||
|
||||
|
||||
def get_schema() -> Dict[str, Any]:
|
||||
"""Get data model schema."""
|
||||
global _schema
|
||||
if _schema is None:
|
||||
_schema = load_depot_file("schema.json")
|
||||
return _schema
|
||||
|
||||
|
||||
def get_views() -> Dict[str, Any]:
|
||||
"""Get view configurations."""
|
||||
global _views
|
||||
if _views is None:
|
||||
_views = load_depot_file("views.json")
|
||||
return _views
|
||||
|
||||
|
||||
def get_scenarios() -> Dict[str, Any]:
|
||||
"""Get test scenario definitions."""
|
||||
global _scenarios
|
||||
if _scenarios is None:
|
||||
_scenarios = load_depot_file("scenarios.json")
|
||||
return _scenarios
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# DATABASE
|
||||
# =============================================================================
|
||||
|
||||
_engine: Optional[Engine] = None
|
||||
|
||||
|
||||
def get_engine() -> Optional[Engine]:
|
||||
"""Get or create database engine (lazy singleton)."""
|
||||
global _engine
|
||||
if _engine is None:
|
||||
try:
|
||||
_engine = create_engine(get_db_url(), pool_pre_ping=True)
|
||||
except Exception as e:
|
||||
print(f"[databrowse] DB engine error: {e}")
|
||||
return _engine
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# SQL MODE - Direct database queries
|
||||
# =============================================================================
|
||||
|
||||
|
||||
def build_view_query(view_name: str) -> Optional[str]:
|
||||
"""Build SQL query from view configuration.
|
||||
|
||||
This is the SQL mode - generates queries directly from view definitions.
|
||||
"""
|
||||
views = get_views()
|
||||
view = None
|
||||
for v in views.get("views", []):
|
||||
if v.get("slug") == view_name or v.get("name") == view_name:
|
||||
view = v
|
||||
break
|
||||
|
||||
if not view:
|
||||
return None
|
||||
|
||||
if view.get("mode") != "sql":
|
||||
return None # Only SQL mode supported for now
|
||||
|
||||
schema = get_schema()
|
||||
entity_def = schema.get("definitions", {}).get(view["entity"], {})
|
||||
table = entity_def.get("table")
|
||||
|
||||
if not table:
|
||||
return None
|
||||
|
||||
# Build field list
|
||||
fields = view.get("fields", [])
|
||||
field_list = []
|
||||
|
||||
for field_name in fields:
|
||||
# Check if it's a regular field or computed field
|
||||
if field_name in entity_def.get("properties", {}):
|
||||
col = entity_def["properties"][field_name].get("column", field_name)
|
||||
field_list.append(f"{table}.{col} as {field_name}")
|
||||
elif field_name in entity_def.get("computed", {}):
|
||||
computed_def = entity_def["computed"][field_name]
|
||||
sql_expr = computed_def.get("sql", "NULL")
|
||||
field_list.append(f"({sql_expr}) as {field_name}")
|
||||
|
||||
# Add computed group field if specified
|
||||
group_by = view.get("group_by")
|
||||
computed_group = view.get("computed_group", {}).get(group_by)
|
||||
if computed_group:
|
||||
group_sql = computed_group.get("sql")
|
||||
field_list.append(f"({group_sql}) as {group_by}")
|
||||
|
||||
# Build query
|
||||
query = f"SELECT {', '.join(field_list)} FROM {table}"
|
||||
|
||||
# Add filters
|
||||
filters = view.get("filter", {})
|
||||
where_clauses = []
|
||||
for field, value in filters.items():
|
||||
if isinstance(value, list):
|
||||
# IN clause
|
||||
values_str = ", ".join([f"'{v}'" for v in value])
|
||||
where_clauses.append(f"{table}.{field} IN ({values_str})")
|
||||
else:
|
||||
where_clauses.append(f"{table}.{field} = '{value}'")
|
||||
|
||||
if where_clauses:
|
||||
query += " WHERE " + " AND ".join(where_clauses)
|
||||
|
||||
# Add ordering
|
||||
order_by = view.get("order_by")
|
||||
if order_by:
|
||||
query += f" ORDER BY {order_by}"
|
||||
|
||||
return query
|
||||
|
||||
|
||||
def execute_view_query(view_name: str) -> List[Dict[str, Any]]:
|
||||
"""Execute view query and return results."""
|
||||
engine = get_engine()
|
||||
if not engine:
|
||||
return []
|
||||
|
||||
query_sql = build_view_query(view_name)
|
||||
if not query_sql:
|
||||
return []
|
||||
|
||||
try:
|
||||
with engine.connect() as conn:
|
||||
result = conn.execute(text(query_sql))
|
||||
rows = result.fetchall()
|
||||
columns = result.keys()
|
||||
return [dict(zip(columns, row)) for row in rows]
|
||||
except Exception as e:
|
||||
print(f"[databrowse] Query error for view '{view_name}': {e}")
|
||||
print(f"[databrowse] Query was: {query_sql}")
|
||||
return []
|
||||
|
||||
|
||||
def group_results_by(
|
||||
results: List[Dict[str, Any]], group_field: str, labels: Dict[str, str] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""Group results by a field value."""
|
||||
grouped = {}
|
||||
for row in results:
|
||||
key = row.get(group_field, "unknown")
|
||||
if key not in grouped:
|
||||
grouped[key] = []
|
||||
grouped[key].append(row)
|
||||
|
||||
# Add labels if provided
|
||||
if labels:
|
||||
return {
|
||||
key: {"label": labels.get(key, key), "items": items}
|
||||
for key, items in grouped.items()
|
||||
}
|
||||
return {key: {"label": key, "items": items} for key, items in grouped.items()}
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# API MODE - Placeholder for backend integration
|
||||
# =============================================================================
|
||||
|
||||
|
||||
def fetch_via_api(view_name: str) -> List[Dict[str, Any]]:
|
||||
"""Fetch data via backend API.
|
||||
|
||||
This is the API mode - will call backend endpoints.
|
||||
Currently placeholder.
|
||||
"""
|
||||
# TODO: Implement API mode
|
||||
# Will call endpoints like /api/v1/databrowse/{view_name}/
|
||||
# with auth token from room config
|
||||
return []
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# ROUTES
|
||||
# =============================================================================
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
def health():
|
||||
"""Health check."""
|
||||
engine = get_engine()
|
||||
db_ok = False
|
||||
if engine:
|
||||
try:
|
||||
with engine.connect() as conn:
|
||||
conn.execute(text("SELECT 1"))
|
||||
db_ok = True
|
||||
except:
|
||||
pass
|
||||
|
||||
return {
|
||||
"status": "ok" if db_ok else "degraded",
|
||||
"service": "databrowse-monitor",
|
||||
"room": ROOM_CONFIG["name"],
|
||||
"database": "connected" if db_ok else "disconnected",
|
||||
"depot": {
|
||||
"path": str(DEPOT_DIR),
|
||||
"schema": "loaded" if get_schema() else "missing",
|
||||
"views": len(get_views().get("views", [])),
|
||||
"scenarios": len(get_scenarios().get("scenarios", [])),
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@app.get("/", response_class=HTMLResponse)
|
||||
def index(request: Request):
|
||||
"""Main data browser landing page."""
|
||||
views = get_views().get("views", [])
|
||||
scenarios = get_scenarios().get("scenarios", [])
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"index.html",
|
||||
{
|
||||
"request": request,
|
||||
"views": views,
|
||||
"scenarios": scenarios,
|
||||
"room_name": ROOM_CONFIG["name"],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@app.get("/view/{view_slug}", response_class=HTMLResponse)
|
||||
def view_data(request: Request, view_slug: str):
|
||||
"""Display data for a specific view."""
|
||||
views_data = get_views()
|
||||
view_config = None
|
||||
|
||||
for v in views_data.get("views", []):
|
||||
if v.get("slug") == view_slug:
|
||||
view_config = v
|
||||
break
|
||||
|
||||
if not view_config:
|
||||
return HTMLResponse("View not found", status_code=404)
|
||||
|
||||
# Execute query based on mode
|
||||
mode = view_config.get("mode", "sql")
|
||||
if mode == "sql":
|
||||
results = execute_view_query(view_slug)
|
||||
elif mode == "api":
|
||||
results = fetch_via_api(view_slug)
|
||||
else:
|
||||
results = []
|
||||
|
||||
# Group results if specified
|
||||
group_by = view_config.get("group_by")
|
||||
if group_by:
|
||||
computed_group = view_config.get("computed_group", {}).get(group_by, {})
|
||||
labels = computed_group.get("labels", {})
|
||||
grouped_results = group_results_by(results, group_by, labels)
|
||||
else:
|
||||
grouped_results = {"all": {"label": "All Results", "items": results}}
|
||||
|
||||
return templates.TemplateResponse(
|
||||
"view.html",
|
||||
{
|
||||
"request": request,
|
||||
"view": view_config,
|
||||
"results": results,
|
||||
"grouped_results": grouped_results,
|
||||
"total": len(results),
|
||||
"room_name": ROOM_CONFIG["name"],
|
||||
},
|
||||
)
|
||||
|
||||
|
||||
@app.get("/api/views")
|
||||
def api_views():
|
||||
"""List all available views."""
|
||||
return get_views()
|
||||
|
||||
|
||||
@app.get("/api/view/{view_slug}")
|
||||
def api_view_data(view_slug: str):
|
||||
"""Get data for a specific view as JSON."""
|
||||
views_data = get_views()
|
||||
view_config = None
|
||||
|
||||
for v in views_data.get("views", []):
|
||||
if v.get("slug") == view_slug:
|
||||
view_config = v
|
||||
break
|
||||
|
||||
if not view_config:
|
||||
return JSONResponse({"error": "View not found"}, status_code=404)
|
||||
|
||||
# Execute query
|
||||
mode = view_config.get("mode", "sql")
|
||||
if mode == "sql":
|
||||
results = execute_view_query(view_slug)
|
||||
elif mode == "api":
|
||||
results = fetch_via_api(view_slug)
|
||||
else:
|
||||
results = []
|
||||
|
||||
return {
|
||||
"view": view_config,
|
||||
"results": results,
|
||||
"total": len(results),
|
||||
"room": ROOM_CONFIG["name"],
|
||||
}
|
||||
|
||||
|
||||
@app.get("/api/schema")
|
||||
def api_schema():
|
||||
"""Get data model schema."""
|
||||
return get_schema()
|
||||
|
||||
|
||||
@app.get("/api/scenarios")
|
||||
def api_scenarios():
|
||||
"""Get test scenario definitions."""
|
||||
return get_scenarios()
|
||||
|
||||
|
||||
# =============================================================================
|
||||
# MAIN
|
||||
# =============================================================================
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
|
||||
uvicorn.run(
|
||||
"main:app",
|
||||
host="0.0.0.0",
|
||||
port=int(os.getenv("PORT", "12020")),
|
||||
reload=True,
|
||||
)
|
||||
418
station/monitors/databrowse/view.html
Normal file
418
station/monitors/databrowse/view.html
Normal file
@@ -0,0 +1,418 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<title>{{ view.title }} · {{ room_name }}</title>
|
||||
<style>
|
||||
* {
|
||||
box-sizing: border-box;
|
||||
margin: 0;
|
||||
padding: 0;
|
||||
}
|
||||
|
||||
body {
|
||||
font-family:
|
||||
system-ui,
|
||||
-apple-system,
|
||||
sans-serif;
|
||||
background: #111827;
|
||||
color: #f3f4f6;
|
||||
min-height: 100vh;
|
||||
padding: 1rem;
|
||||
}
|
||||
|
||||
header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
padding: 0.5rem 0 1rem;
|
||||
border-bottom: 1px solid #374151;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
h1 {
|
||||
font-size: 1.25rem;
|
||||
font-weight: 600;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.5rem;
|
||||
}
|
||||
|
||||
.room-badge {
|
||||
font-size: 0.7rem;
|
||||
background: #374151;
|
||||
padding: 0.2rem 0.5rem;
|
||||
border-radius: 4px;
|
||||
color: #9ca3af;
|
||||
}
|
||||
|
||||
.header-right {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 1.5rem;
|
||||
}
|
||||
|
||||
.total {
|
||||
font-size: 1.5rem;
|
||||
font-weight: 700;
|
||||
color: #60a5fa;
|
||||
}
|
||||
|
||||
.total small {
|
||||
font-size: 0.75rem;
|
||||
color: #9ca3af;
|
||||
font-weight: 400;
|
||||
}
|
||||
|
||||
.back-link {
|
||||
color: #60a5fa;
|
||||
text-decoration: none;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
.back-link:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
|
||||
/* Pipeline/Column Layout (for grouped views) */
|
||||
.pipeline {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
|
||||
gap: 0.75rem;
|
||||
margin-bottom: 1rem;
|
||||
}
|
||||
|
||||
.column {
|
||||
background: #1f2937;
|
||||
border-radius: 8px;
|
||||
overflow: hidden;
|
||||
max-height: 80vh;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
}
|
||||
|
||||
.column-header {
|
||||
padding: 0.6rem 0.75rem;
|
||||
font-size: 0.8rem;
|
||||
font-weight: 600;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
border-bottom: 2px solid #60a5fa;
|
||||
color: #60a5fa;
|
||||
background: #111827;
|
||||
}
|
||||
|
||||
.column-count {
|
||||
font-size: 1rem;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.column-items {
|
||||
padding: 0.5rem;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
gap: 0.4rem;
|
||||
overflow-y: auto;
|
||||
flex: 1;
|
||||
}
|
||||
|
||||
/* Individual item card */
|
||||
.item-card {
|
||||
background: #111827;
|
||||
border-radius: 6px;
|
||||
padding: 0.6rem 0.7rem;
|
||||
font-size: 0.75rem;
|
||||
border-left: 3px solid #60a5fa;
|
||||
cursor: pointer;
|
||||
transition: background 0.15s;
|
||||
}
|
||||
|
||||
.item-card:hover {
|
||||
background: #1f2937;
|
||||
}
|
||||
|
||||
.item-id {
|
||||
font-weight: 600;
|
||||
color: #9ca3af;
|
||||
font-size: 0.7rem;
|
||||
margin-bottom: 0.25rem;
|
||||
}
|
||||
|
||||
.item-primary {
|
||||
color: #d1d5db;
|
||||
font-weight: 500;
|
||||
margin-bottom: 0.25rem;
|
||||
}
|
||||
|
||||
.item-meta {
|
||||
font-size: 0.7rem;
|
||||
color: #6b7280;
|
||||
margin-top: 0.3rem;
|
||||
display: flex;
|
||||
gap: 0.5rem;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
|
||||
.meta-item {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
gap: 0.2rem;
|
||||
}
|
||||
|
||||
/* Icon indicators */
|
||||
.icon {
|
||||
display: inline-block;
|
||||
width: 6px;
|
||||
height: 6px;
|
||||
border-radius: 50%;
|
||||
background: #4ade80;
|
||||
}
|
||||
|
||||
.icon.false {
|
||||
background: #6b7280;
|
||||
}
|
||||
|
||||
/* Table Layout (for non-grouped views) */
|
||||
.table-container {
|
||||
background: #1f2937;
|
||||
border-radius: 8px;
|
||||
overflow-x: auto;
|
||||
}
|
||||
|
||||
table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
font-size: 0.85rem;
|
||||
}
|
||||
|
||||
thead {
|
||||
background: #111827;
|
||||
position: sticky;
|
||||
top: 0;
|
||||
}
|
||||
|
||||
th {
|
||||
text-align: left;
|
||||
padding: 0.75rem;
|
||||
font-weight: 600;
|
||||
color: #60a5fa;
|
||||
border-bottom: 2px solid #374151;
|
||||
}
|
||||
|
||||
td {
|
||||
padding: 0.6rem 0.75rem;
|
||||
border-bottom: 1px solid #374151;
|
||||
}
|
||||
|
||||
tr:hover {
|
||||
background: #252f3f;
|
||||
}
|
||||
|
||||
.text-primary {
|
||||
color: #d1d5db;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.text-secondary {
|
||||
color: #9ca3af;
|
||||
}
|
||||
|
||||
.text-muted {
|
||||
color: #6b7280;
|
||||
font-size: 0.8em;
|
||||
}
|
||||
|
||||
/* Badge */
|
||||
.badge {
|
||||
display: inline-block;
|
||||
padding: 0.2rem 0.5rem;
|
||||
border-radius: 4px;
|
||||
font-size: 0.75rem;
|
||||
font-weight: 500;
|
||||
}
|
||||
|
||||
.badge.pending {
|
||||
background: #7c2d12;
|
||||
color: #fbbf24;
|
||||
}
|
||||
|
||||
.badge.active {
|
||||
background: #065f46;
|
||||
color: #4ade80;
|
||||
}
|
||||
|
||||
/* Empty state */
|
||||
.empty {
|
||||
text-align: center;
|
||||
color: #6b7280;
|
||||
padding: 3rem;
|
||||
font-size: 0.9rem;
|
||||
}
|
||||
|
||||
/* Footer */
|
||||
footer {
|
||||
margin-top: 1rem;
|
||||
padding-top: 1rem;
|
||||
border-top: 1px solid #374151;
|
||||
font-size: 0.7rem;
|
||||
color: #6b7280;
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
}
|
||||
|
||||
footer a {
|
||||
color: #60a5fa;
|
||||
text-decoration: none;
|
||||
}
|
||||
|
||||
footer a:hover {
|
||||
text-decoration: underline;
|
||||
}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<header>
|
||||
<h1>
|
||||
{{ view.title }}
|
||||
<span class="room-badge">{{ room_name }}</span>
|
||||
</h1>
|
||||
<div class="header-right">
|
||||
<a href="/" class="back-link">← Back</a>
|
||||
<div class="total">
|
||||
{{ total }}
|
||||
<small>items</small>
|
||||
</div>
|
||||
</div>
|
||||
</header>
|
||||
|
||||
<main>
|
||||
{% if total == 0 %}
|
||||
<div class="empty">No data found</div>
|
||||
{% elif view.group_by %}
|
||||
<!-- Grouped/Pipeline View -->
|
||||
<div class="pipeline">
|
||||
{% for group_key, group_data in grouped_results.items() %}
|
||||
<div class="column">
|
||||
<div class="column-header">
|
||||
<span>{{ group_data.label }}</span>
|
||||
<span class="column-count"
|
||||
>{{ group_data.items|length }}</span
|
||||
>
|
||||
</div>
|
||||
<div class="column-items">
|
||||
{% for item in group_data.items %}
|
||||
<div class="item-card">
|
||||
{% if item.id %}
|
||||
<div class="item-id">#{{ item.id }}</div>
|
||||
{% endif %} {% if item.username %}
|
||||
<div class="item-primary">{{ item.username }}</div>
|
||||
{% elif item.first_name %}
|
||||
<div class="item-primary">
|
||||
{{ item.first_name }} {{ item.last_name or '' }}
|
||||
</div>
|
||||
{% endif %}
|
||||
|
||||
<div class="item-meta">
|
||||
{% if item.email %}
|
||||
<span class="meta-item text-muted"
|
||||
>{{ item.email }}</span
|
||||
>
|
||||
{% endif %} {% if item.phone %}
|
||||
<span class="meta-item text-muted"
|
||||
>{{ item.phone }}</span
|
||||
>
|
||||
{% endif %} {% if item.has_pets is defined %}
|
||||
<span class="meta-item">
|
||||
<span
|
||||
class="icon {{ 'true' if item.has_pets else 'false' }}"
|
||||
></span>
|
||||
Pets
|
||||
</span>
|
||||
{% endif %} {% if item.has_coverage is defined
|
||||
%}
|
||||
<span class="meta-item">
|
||||
<span
|
||||
class="icon {{ 'true' if item.has_coverage else 'false' }}"
|
||||
></span>
|
||||
Coverage
|
||||
</span>
|
||||
{% endif %} {% if item.has_requests is defined
|
||||
%}
|
||||
<span class="meta-item">
|
||||
<span
|
||||
class="icon {{ 'true' if item.has_requests else 'false' }}"
|
||||
></span>
|
||||
Requests
|
||||
</span>
|
||||
{% endif %} {% if item.has_turnos is defined %}
|
||||
<span class="meta-item">
|
||||
<span
|
||||
class="icon {{ 'true' if item.has_turnos else 'false' }}"
|
||||
></span>
|
||||
Turnos
|
||||
</span>
|
||||
{% endif %} {% if item.active_requests is
|
||||
defined %}
|
||||
<span class="meta-item"
|
||||
>{{ item.active_requests }} active</span
|
||||
>
|
||||
{% endif %}
|
||||
</div>
|
||||
|
||||
{% if item.user_id %}
|
||||
<div class="item-meta">
|
||||
<span class="text-muted"
|
||||
>User ID: {{ item.user_id }}</span
|
||||
>
|
||||
</div>
|
||||
{% endif %}
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
</div>
|
||||
{% endfor %}
|
||||
</div>
|
||||
{% else %}
|
||||
<!-- Table View (non-grouped) -->
|
||||
<div class="table-container">
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
{% for field in view.fields[:8] %}
|
||||
<th>{{ field|replace('_', ' ')|title }}</th>
|
||||
{% endfor %}
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
{% for item in results %}
|
||||
<tr>
|
||||
{% for field in view.fields[:8] %}
|
||||
<td>
|
||||
{% if item[field] is boolean %}
|
||||
<span
|
||||
class="icon {{ 'true' if item[field] else 'false' }}"
|
||||
></span>
|
||||
{% elif item[field] is number %} {{ item[field]
|
||||
}} {% else %} {{ item[field] or '-' }} {% endif
|
||||
%}
|
||||
</td>
|
||||
{% endfor %}
|
||||
</tr>
|
||||
{% endfor %}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
{% endif %}
|
||||
</main>
|
||||
|
||||
<footer>
|
||||
<span>{{ view.description }}</span>
|
||||
<div>
|
||||
<a href="/api/view/{{ view.slug }}">JSON</a> ·
|
||||
<a href="/">Home</a>
|
||||
</div>
|
||||
</footer>
|
||||
</body>
|
||||
</html>
|
||||
Reference in New Issue
Block a user