📚 A curated list of papers & technical articles on AI Quality & Safety

AI model health monitor for LLM apps – runtime checks for drift, hallucination risk, latency, and JSON/format quality on any OpenAI, Anthropic, or local client.

Ship evals before you ship features.

Eval framework. Define correct, test against it, get results.

Open-source AI model evaluation and benchmarking framework for LLMs (OpenAI, Ollama, Claude, Gemini)

Find what your AI agent gets wrong — before you have a rubric. Qualitative eval for PMs.

Diagnose your AI agents in production. Extract policies from prompts, evaluate traces, generate diagnostic reports.

A framework-agnostic metric for measuring AI code generation quality. Sealed-envelope testing protocol + reference validators.

朱雀 Suzaku — AI 生成品質模組。諂媚抑制、建設性挑戰、輸出適配、上下文錨定、一致性守護。基於 LDRIT 設計。

Open-source AI agent security testing framework. Test for prompt injection, data leakage, and privilege escalation before production.

Provider-agnostic LLM evaluation harness: golden dataset, deterministic + LLM-as-judge scoring, RAG failure attribution, red-team suite, severity-weighted CI gate.

Python SDK for IvyCheck

Universal skill enhancement layer for Claude Code. Sees what your skill was trying to do, grades the gap, drives the rewrite.

Evaluate your LLM apps with one function call. Hallucination detection, RAG scoring, and agent evals for OpenAI, Anthropic, and more. 14 evaluators, pytest plugin, composite trust scores.

AI Agent Ops framework for Claude Code — independent evaluator, adversarial review, and pre-commit quality gate for AI-generated code.

A 5-layer adversarial quality gate for Claude Code. Catches factual errors, score inflation, and buried conclusions before your AI output ships.

The definitive CI/CD platform for AI Quality.

🚀 Professional-grade AI Agent Evaluation Platform. Multi-provider LLM-as-a-Judge (OpenAI, Anthropic, Gemini), automated testing, A/B benchmarking, and safety auditing.

Self-improving AI quality system - auto-score, auto-regenerate, auto-learn.

Evidence-backed quality control for LLM outputs: rubrics, claim grounding, safety gates, human review, and reports.