Skip to content

VIEW_LOGS_IMPLEMENTATION.md

Latest commit

 

History

History
345 lines (258 loc) · 9.4 KB

VIEW_LOGS_IMPLEMENTATION.md

File metadata and controls

345 lines (258 loc) · 9.4 KB
Raw

View Logs Implementation - Complete Summary

Overview

Implemented a complete "View Logs" functionality for the GraphWiz-XR Admin Dashboard with:

  • In-memory log storage backend
  • REST API endpoints for log querying
  • Full-featured React component with filtering, search, and pagination

Implementation Details

1. Backend Changes

A. Core Library - Log Storage Module

File: packages/services/reticulum/core/src/log_store.rs

Features:

  • LogEntry struct with id, timestamp, level, service, message, context
  • LogLevel enum: Debug, Info, Warn, Error
  • LogStore - Thread-safe in-memory log store with max entries (10,000 per service)
  • LogsQuery - Supports filtering by service, level, time range, search
  • LogsResponse - Paginated response with entries, total, page, per_page
  • Global log stores for all services: auth, hub, presence, sfu, storage
  • add_log() - Async function to add log entries
  • Tracing layer integration for automatic log capture

Added to: packages/services/reticulum/core/src/lib.rs

pub mod log_store;
pub use log_store::{add_log, get_log_store, LogEntry, LogLevel, LogStore, LogsQuery, LogsResponse};

B. Auth Service - Admin Log Endpoints

File: packages/services/reticulum/auth/src/admin_handlers.rs

Endpoints:

  1. GET /auth/admin/logs/{service_name} - Fetch logs for specific service
  2. GET /auth/admin/logs - Fetch logs from all services (with filtering)
  3. GET /auth/admin/logs/export - Export logs as JSON file download
  4. POST /auth/admin/logs/clear - Clear logs for a service

Query Parameters:

  • service - Filter by service name
  • level - Filter by log level (debug/info/warn/error)
  • start_time - ISO 8601 timestamp (default: last 1 hour)
  • end_time - ISO 8601 timestamp
  • search - Text search in messages
  • page - Page number (default: 1)
  • per_page - Items per page (default: 50, max: 1000)

Added to: packages/services/reticulum/auth/src/lib.rs

pub mod admin_handlers;

Updated: packages/services/reticulum/auth/src/routes.rs

.route("/admin/logs", web::get().to(admin_handlers::fetch_all_logs))
.route("/admin/logs/export", web::get().to(admin_handlers::export_logs))
.route("/admin/logs/clear", web::post().to(admin_handlers::clear_logs))
.service(web::resource("/admin/logs/{service_name}").route(web::get().to(admin_handlers::fetch_logs)))

2. Frontend Changes

A. API Client - Log Fetching Functions

File: packages/clients/admin-client/src/api-client.ts

New Types:

export type LogLevel = 'debug' | 'info' | 'warn' | 'error';
export interface LogEntry {
  id: string;
  timestamp: string;
  level: LogLevel;
  service: string;
  message: string;
  context?: Record<string, unknown>;
}
export interface LogsResponse {
  entries: LogEntry[];
  total: number;
  page: number;
  perPage: number;
}
export interface LogsQuery {
  service?: string;
  level?: LogLevel;
  startTime?: string;
  endTime?: string;
  search?: string;
  page?: number;
  perPage?: number;
}

New Functions:

  1. fetchLogsForService(serviceName, query) - Fetch logs from specific service
  2. fetchLogs(query) - Fetch logs from all services with client-side filtering
  3. exportLogs(query) - Export logs as JSON string
  4. applyFilters(entries, query) - Apply filters and sorting to log entries

B. LogsViewer Component

File: packages/clients/admin-client/src/LogsViewer.tsx

Features:

  • Filtering:

    • Service selector dropdown (All, Auth, Hub, Presence, SFU, Storage)
    • Log level dropdown (Debug, Info, Warn, Error)
    • Time range picker (1h, 24h, 7d, 30d, 90d)
    • Search input with real-time filtering

  • Display:

    • Log entries with color-coded level indicators
    • Icons: ○ Debug, ● Info, ⚠ Warn, ✕ Error
    • Timestamp in local format
    • Service badge
    • Expandable entries for context details
    • Auto-scrolling with max height

  • Pagination:

    • Previous/Next buttons
    • Page number display
    • Total entries count
    • Page selector (max 5 pages visible)

  • Auto-Refresh:

    • Toggle button (30-second interval)
    • Visual indicator (green when enabled, gray when disabled)
    • Automatic data reload

  • Export:

    • Download logs as JSON file
    • Filename: logs-YYYY-MM-DD.json
    • Includes metadata (export timestamp, total count)

  • Loading States:

    • Spinner during initial load
    • Empty state when no logs match
    • Button disabled during operations

C. Admin Dashboard Integration

File: packages/clients/admin-client/src/App.tsx

Changes:

  • Added showLogsViewer state for modal visibility
  • Updated "View Logs" button to open modal instead of alert
  • Added full-screen modal overlay with LogsViewer component
  • Close button (×) to dismiss modal
  • Responsive design (max-width: 6xl)

Testing Instructions

1. Start Backend Services

cd packages/services/reticulum/auth
cargo run

The auth service will start on port 8011 with admin log endpoints enabled.

2. Generate Sample Logs (Optional)

cd packages/services/reticulum/auth
cargo run --bin generate_sample_logs

This will add 100 sample log entries across all services.

3. Start Admin Dashboard

cd packages/clients/admin-client
npm run dev

Admin dashboard will start on port 5174.

4. Test Logs Feature

  1. Open http://localhost:5174
  2. Click "View Logs" button
  3. Logs viewer modal will open
  4. Test features:
    • Change service filter (e.g., "Auth")
    • Change log level (e.g., "Error")
    • Adjust time range (e.g., "Last 24 hours")
    • Search for text (e.g., "connection")
    • Click pagination buttons
    • Toggle auto-refresh
    • Click "Export Logs" to download JSON
    • Click log entry to expand context
    • Click "×" to close modal

API Endpoints Reference

Fetch Logs from All Services

GET /auth/admin/logs?service=auth&level=error&search=failed&page=1&per_page=50

Fetch Logs from Specific Service

GET /auth/admin/logs/auth?level=warn&start_time=2026-01-01T00:00:00Z

Export Logs

GET /auth/admin/logs/export?service=hub&level=error

Clear Logs

POST /auth/admin/logs/clear

Response Format

{
  "entries": [
    {
      "id": "550e8400-e29b-41d4-a716-446655440000",
      "timestamp": "2026-01-06T12:00:00Z",
      "level": "error",
      "service": "auth",
      "message": "Database connection failed",
      "context": {
        "error": "Connection refused"
      }
    }
  ],
  "total": 42,
  "page": 1,
  "per_page": 50
}

Technical Architecture

Backend Flow

  1. Log entries stored in thread-safe RwLock<Vec<LogEntry>>
  2. Each service gets dedicated log store (10,000 max entries)
  3. Log capture via tracing layer (automatic from all log::info/warn/error calls)
  4. Query filters applied before sorting and pagination
  5. Sorted by timestamp descending (newest first)

Frontend Flow

  1. Fetch logs from /auth/admin/logs endpoint
  2. Apply additional client-side filters (if needed)
  3. Display paginated results
  4. Auto-refresh every 30s (if enabled)
  5. Export via JSON blob download

Port Configuration

  • Auth: 8011
  • Hub: 8012
  • Presence: 8013
  • SFU: 8014
  • Storage: 8015
  • Admin Dashboard: 5174

Limitations & Future Enhancements

Current Limitations

  • In-memory log storage (logs lost on service restart)
  • No log rotation or archival
  • No log shipping to external services (Loki, CloudWatch)
  • No advanced filtering (regex, multiple levels)
  • No real-time streaming (Server-Sent Events, WebSocket)
  • Only auth service has admin endpoints (needs replication to other services)

Recommended Enhancements

  1. Persistent Storage: Add PostgreSQL table for log persistence
  2. Log Rotation: Auto-archive old logs to S3/file system
  3. Real-time Streaming: Implement SSE/WebSocket for live log updates
  4. Advanced Search: Regex search, multiple level selection, time range picker
  5. Metrics Integration: Correlate logs with metrics from presence service
  6. Alerting: Email/webhook on error-level logs
  7. Multi-service Deployment: Add admin endpoints to all 5 services
  8. Authentication: Add admin role verification to log endpoints

Files Modified/Created

Backend (Rust)

  • ✅ packages/services/reticulum/core/src/log_store.rs (NEW, 220 lines)
  • ✅ packages/services/reticulum/core/src/lib.rs (MODIFIED, 2 lines added)
  • ✅ packages/services/reticulum/auth/src/admin_handlers.rs (NEW, 150 lines)
  • ✅ packages/services/reticulum/auth/src/lib.rs (MODIFIED, 1 line added)
  • ✅ packages/services/reticulum/auth/src/routes.rs (MODIFIED, 6 lines added)
  • ✅ packages/services/reticulum/auth/src/generate_sample_logs.rs (NEW, 30 lines)

Frontend (TypeScript/React)

  • ✅ packages/clients/admin-client/src/api-client.ts (MODIFIED, 120 lines added)
  • ✅ packages/clients/admin-client/src/LogsViewer.tsx (NEW, 310 lines)
  • ✅ packages/clients/admin-client/src/App.tsx (MODIFIED, 25 lines added)

Total Lines Added

  • Backend: ~400 lines
  • Frontend: ~450 lines
  • Total: ~850 lines

Status

✅ COMPLETE - All core functionality implemented and ready for testing

Next Steps for Production

  1. Add admin log endpoints to hub, presence, sfu, storage services
  2. Implement PostgreSQL log persistence
  3. Add admin authentication to log endpoints
  4. Add log rotation and archival
  5. Implement real-time log streaming
  6. Add advanced filtering and search capabilities
  7. Integrate with monitoring/alerting systems