Best Evaluation AI tools
Measuring whether models, agents and prompts actually work — scores, traces and test harnesses. The 13 highest-ranked of the 13 apps tagged Evaluation in the Onei AI catalogue.
Everything tagged Evaluation: apps, skills and MCP servers
-
Run agent evals and private benchmarks in sandboxed cloud environments — compare Claude Code, Codex, Cursor and Copilot on the same real tasks.
Also: Developers, Testing · Editor's pick
-
Microsoft
Microsoft's open-source optimizer that trains an agent's SKILL.md like a model parameter — epochs, learning-rate budgets and a held-out validation gate, with frozen weights.
Also: Developers, Open Source · Editor's pick
-
Open-source testing and evals platform for MCP servers — trace tool calls, debug OAuth, and catch cross-client regressions in CI.
Also: Open Source, Testing · Editor's pick
-
Tatva Labs Inc.
Continuous automated QA for voice and chat AI agents, from pre-launch simulation to live call monitoring.
Also: Developers, Monitoring, Testing · Editor's pick
-
Arize AI
AI engineering platform for tracing, evaluating and improving LLM apps and agents — with Phoenix, its open-source local-first counterpart, for teams that want to start without a vendor.
Also: Observability, Open Source
-
The standard experiment tracker for machine learning, now with tracing and evaluation for LLM applications.
Also: Observability, Python
-
Lenz IO
Audit-grade fact-checking API for AI products — eight models across five stages, adversarial debate, and a full citation trail behind every verdict.
Also: Developers
-
Reads your production chat and voice agent conversations to surface the silent failures, frustration loops and policy breaches that offline evals never catch.
Also: Monitoring, Observability
-
Evaluation-first AI development — measure whether a prompt or model change actually improved anything.
Also: Developers, Observability, Testing
-
Open-source LLM observability — trace, evaluate, and improve AI applications with production data.
Also: Developers, Observability, Open Source
-
The framework and observability platform most teams use to build, debug, and ship LLM agents.
Also: Developers, Observability, Open Source
-
LangGenius
An open-source platform for building production LLM applications — visual workflows, RAG, and agents in one place.
Also: Developers, Open Source
-
Prefactor Pty Ltd
Agent observability and evaluation that scores every production run for quality, drift and risk — and can pause or block a risky action before it executes.
Also: Developers, Monitoring, Observability
How this list is ranked
Ranked by editorial weight (0–100), plus 5 points per bookmark up to 10, plus or minus up to 20 for the average review rating; ties go to featured apps, then the most recently updated. Discontinued apps are never ranked.