Yantrik ML

Pluggable AI inference library for Yantrik OS. Provides LLM backends, embeddings, vision, text-to-speech, and model-adaptive intelligence — all in Rust.

Features

• Multiple LLM backends — Ollama, OpenAI-compatible APIs, Claude CLI, llama.cpp (via C bindings)
• Automatic fallback — Primary backend fails? Seamlessly switches to a local fallback model
• Model capability profiling — Auto-detects model tier (Tiny/Small/Medium/Large) from model name, adjusts prompting strategy, tool usage, and context budgets accordingly
• Embeddings — Candle-based local sentence embeddings for semantic search
• Text-to-speech — Platform-native TTS via system speech APIs
• Speech-to-text — Whisper-based transcription via Candle
• Multi-provider support — Anthropic, Google Gemini, OpenAI, and generic OpenAI-compatible endpoints with automatic routing and health checks
• Hardware detection — GPU/CPU capability detection for optimal model selection

Architecture

yantrik-ml/
├── src/
│   ├── llm/
│   │   ├── api.rs          # Ollama + OpenAI-compatible backend
│   │   ├── claude_cli.rs   # Claude Code CLI backend
│   │   ├── llamacpp.rs     # llama.cpp C bindings (optional)
│   │   ├── fallback.rs     # Auto-fallback wrapper (primary → fallback)
│   │   └── candle.rs       # Candle-based local inference
│   ├── provider/
│   │   ├── anthropic.rs    # Claude API
│   │   ├── gemini.rs       # Google Gemini API
│   │   ├── generic_openai.rs # Any OpenAI-compatible endpoint
│   │   ├── routing.rs      # Multi-provider request routing
│   │   └── health.rs       # Provider health monitoring
│   ├── stt/                # Speech-to-text (Whisper)
│   ├── capability.rs       # Model tier detection + adaptive config
│   ├── embedder.rs         # Sentence embeddings
│   ├── hardware.rs         # GPU/CPU detection
│   ├── tts.rs              # Text-to-speech
│   └── traits.rs           # LLMBackend trait

Model-Adaptive Intelligence

Yantrik ML automatically profiles models and adjusts behavior:

Tier	Size	Tools per prompt	Tool call mode	Native tools
Tiny	0.5–1.5B	3	MCQ	No
Small	1.5–4B	5	Structured JSON	No
Medium	4–14B	8	Structured JSON	No
Large	14B+	15+	Native function calling	Yes

Usage

This crate is used as a dependency by yantrik-companion and yantrik-os. It is not intended to be used standalone.

[dependencies]
yantrik-ml = { git = "https://github.com/yantrikos/yantrik-ml.git" }

License

AGPL-3.0. See LICENSE for details.