Skip to content

AGENT.md

Latest commit

 

History

History
245 lines (192 loc) · 9.38 KB

AGENT.md

File metadata and controls

245 lines (192 loc) · 9.38 KB
Raw

Ollana Agent Specification

Overview

Ollana is an intelligent agent designed to facilitate seamless access to Ollama servers across local networks. It automatically discovers available Ollama instances and provides transparent proxying capabilities, allowing clients to interact with remote Ollama servers without complex configuration.

Agent Capabilities

Core Functionality

  • Auto-discovery: Automatically detects Ollama servers on the local network using UDP broadcast discovery
  • Transparent Proxying: Acts as an HTTP proxy that forwards requests between clients and Ollama servers
  • Multi-server Management: Manages connections to multiple Ollama servers with automatic failover
  • Liveness Monitoring: Continuously monitors server availability and removes unreachable servers from rotation

Technical Capabilities

  • Cross-platform Support: Runs on Linux, macOS, and Windows systems
  • Zero Configuration: Requires no manual setup of client applications
  • Dynamic Discovery: Adapts to network changes automatically
  • High Availability: Maintains connection to the most responsive server available
  • Secure Communication: Uses standard HTTP/HTTPS protocols for communication

Agent Modes

  1. Server Mode:

    • Runs an HTTP proxy that forwards requests to a local Ollama instance
    • Broadcasts its presence on the network for client discovery
    • Handles incoming requests from clients and forwards them to the local Ollama server

  2. Client Mode:

    • Discovers available Ollama servers on the network
    • Manages connections to multiple servers
    • Provides transparent proxying to the currently active server
    • Performs liveness checks to ensure server availability

Interaction Protocols

Discovery Protocol

  • UDP Broadcast: Uses UDP broadcast messages with magic number 0x4C414E41 (LANA) for discovery
  • Port: Default discovery port 11436
  • Frequency: Broadcasts every 5 seconds in client mode
  • Response: Servers respond to discovery messages with their own presence

HTTP Proxy Protocol

  • HTTP/HTTPS: Supports standard HTTP protocols for proxying requests
  • Request Forwarding: Forwards all HTTP methods (GET, POST, PUT, DELETE) transparently
  • Streaming Support: Handles streaming responses for long-running operations
  • Path Preservation: Maintains original request paths and query parameters

Communication Patterns

  1. Client-Server Discovery:

    • Client broadcasts discovery message
    • Server responds with its own presence
    • Client registers server in its connection pool

  2. Proxy Request Flow:

    • Client sends HTTP request to Ollana proxy
    • Proxy forwards request to active Ollama server
    • Response is forwarded back to client

Data Handling Procedures

Data Flow Architecture

Client Application → [Ollana Client Proxy] → [Ollama Server]
                    ↑
            [Network Discovery]
                    ↓
            [Ollana Server Proxy] ← [Ollama Instance]

Data Processing

  • Request Forwarding: All HTTP requests are forwarded without modification
  • Response Handling: All responses are forwarded transparently to clients
  • Stream Management: Supports streaming data for long-running operations
  • Error Propagation: Errors from Ollama servers are propagated back to clients

Security Considerations

  • Local Network Only: Designed for local network use only
  • No Data Encryption: Uses standard HTTP (not HTTPS) by default
  • Authentication: No built-in authentication mechanisms
  • Access Control: Relies on network-level access controls

Safety Measures

System Safety

  • Graceful Shutdown: Proper cleanup of resources during shutdown
  • Resource Management: Automatic cleanup of unused connections and proxies
  • Error Handling: Comprehensive error handling to prevent crashes
  • Memory Management: Efficient memory usage with proper resource cleanup

Network Safety

  • Broadcast Limitations: Uses standard broadcast addresses only
  • Port Isolation: Uses dedicated ports for different functions (discovery, proxy)
  • Timeout Handling: Configurable timeouts for network operations
  • Connection Limits: Prevents excessive connection attempts

Operational Safety

  • Daemon Mode Support: Can run as a background service
  • Process Management: Proper process isolation and management
  • Logging: Comprehensive logging for debugging and monitoring
  • Health Checks: Automatic liveness checking of servers

Performance Metrics

System Performance

  • Response Time: Typically under 10ms for local network requests
  • Throughput: Supports concurrent connections up to system limits
  • Resource Usage: Minimal CPU and memory footprint during operation
  • Latency: Network latency dependent on local network conditions

Monitoring Metrics

  • Connection Count: Number of active server connections
  • Request Rate: Requests per second processed
  • Error Rate: Percentage of failed requests
  • Proxy Latency: Time taken to forward requests

Performance Optimization

  • Connection Pooling: Reuses connections where possible
  • Asynchronous Processing: Uses async/await for non-blocking operations
  • Efficient Streaming: Handles large payloads efficiently
  • Load Balancing: Automatically selects the best available server

Integration Guidelines

Client Integration

  1. Configuration-Free Setup:

    • No client-side configuration required
    • Automatically discovers and connects to servers

  2. API Compatibility:

    • Fully compatible with standard Ollama API endpoints
    • Supports all Ollama HTTP methods and parameters

  3. Environment Variables:

    • RUST_LOG for logging control
    • Standard environment variables for network configuration

Server Integration

  1. Deployment:

    • Run ollana serve to start in server mode
    • Use ollana serve -d for daemon mode

  2. Network Configuration:

    • Ensure UDP broadcast is enabled on the network
    • Verify firewall allows traffic on configured ports

  3. Monitoring:

    • Monitor logs for system status and errors
    • Use standard process monitoring tools

Development Integration

  1. Build Requirements:

    • Rust 1.70+ toolchain required
    • Cargo package manager

  2. Testing:

    • Unit tests for core components
    • Integration tests for proxy functionality
    • Network simulation tests

Compliance Requirements

Security Standards

  • Local Network Only: Designed for secure local network environments
  • No Data Encryption: Uses standard HTTP (not HTTPS) - suitable only for trusted networks
  • Access Control: Relies on network-level security controls

Privacy Considerations

  • Data Minimization: Only forwards necessary data between clients and servers
  • No Logging: Does not log request content by default
  • Network Visibility: Broadcast discovery messages are visible to local network devices

Regulatory Compliance

  • GDPR: No personal data processing, so no GDPR implications
  • HIPAA: Not designed for healthcare environments
  • PCI DSS: Not designed for payment card processing environments

Operational Compliance

  • System Requirements: Meets standard system requirements for Rust applications
  • Resource Usage: Complies with typical resource usage expectations
  • Network Standards: Follows standard network protocols and practices

Technical Specifications

System Requirements

  • Operating Systems: Linux, macOS, Windows (x86_64)
  • Memory: Minimum 128MB RAM
  • Storage: Minimal disk space required
  • Network: Standard TCP/IP networking support

Port Configuration

  • Discovery Port: 11436 (UDP) - for server discovery
  • Client Proxy Port: 11435 (TCP) - for client connections
  • Ollama Default Port: 11434 (TCP) - standard Ollama port

Dependencies

  • Rust 1.70+ toolchain
  • Tokio async runtime
  • Actix Web framework
  • Reqwest HTTP client
  • Serde JSON serialization

Deployment Considerations

Production Deployment

  1. Daemon Mode: Use ollana serve -d for production deployments
  2. User Management: Run as dedicated user with minimal privileges
  3. Log Management: Configure appropriate log rotation
  4. Monitoring: Integrate with existing monitoring systems

Development Deployment

  1. Local Testing: Run directly with ollana serve
  2. Debug Logging: Use RUST_LOG=debug for detailed logging
  3. Hot Reloading: Supports development with watchexec

Troubleshooting

Common Issues

  • Discovery Failure: Verify UDP broadcast is enabled on network
  • Connection Problems: Check firewall settings and port accessibility
  • Performance Issues: Monitor system resources and network latency

Diagnostic Commands

# Enable debug logging
RUST_LOG=debug ollana serve

# Run in daemon mode
ollana serve -d

# Check current configuration
ollana --help

Future Enhancements

Planned Features

  • Authentication Support: Built-in authentication mechanisms
  • HTTPS Proxying: Secure communication support
  • Advanced Load Balancing: More sophisticated server selection algorithms
  • Configuration Management: Runtime configuration changes

Architecture Improvements

  • Plugin System: Extensible architecture for additional protocols
  • Metrics Export: Prometheus-compatible metrics endpoint
  • Enhanced Logging: Structured logging with more detailed information