Skip to content

Evaluation

Measuring whether models, agents and prompts actually work — scores, traces and test harnesses.

16 apps, 12 skills and 6 MCP servers tagged Evaluation.

Best Evaluation AI tools, ranked

Best Evaluation agent skills, ranked

Apps

All apps
Featured

Run agent evals and private benchmarks in sandboxed cloud environments — compare Claude Code, Codex, Cursor and Copilot on the same real tasks.

Coding & DevelopmentFreemium

App: SkillOpt

Microsoft

Featured

Microsoft's open-source optimizer that trains an agent's SKILL.md like a model parameter — epochs, learning-rate budgets and a held-out validation gate, with frozen weights.

Coding & DevelopmentFree
Featured

Open-source testing and evals platform for MCP servers — trace tool calls, debug OAuth, and catch cross-client regressions in CI.

Coding & DevelopmentFreemium

App: Cekura

Tatva Labs Inc.

Featured

Continuous automated QA for voice and chat AI agents, from pre-launch simulation to live call monitoring.

Coding & DevelopmentFreemium

App: Promptfoo

Promptfoo, Inc.

New

Open-source CLI for evaluating and red-teaming LLM apps: compare models side by side, assert on outputs in CI, and scan agents and RAG pipelines for jailbreaks and data leaks.

Coding & DevelopmentFreemium

App: Entry Point AI

Entry Point AI Inc.

New

No-code workspace to run prompts over your data, generate synthetic training examples and fine-tune task-specific LLMs on your own provider accounts.

Data & AnalyticsFreemium

An open-source TypeScript framework for building AI agents, paired with VoltOps, a console for tracing, evals, prompt management and deployment.

Coding & DevelopmentFreemium

App: Arize AX

Arize AI

AI engineering platform for tracing, evaluating and improving LLM apps and agents — with Phoenix, its open-source local-first counterpart, for teams that want to start without a vendor.

Data & AnalyticsFreemium

App: Lenz

Lenz IO

Audit-grade fact-checking API for AI products — eight models across five stages, adversarial debate, and a full citation trail behind every verdict.

Coding & DevelopmentFreemium

Reads your production chat and voice agent conversations to surface the silent failures, frustration loops and policy breaches that offline evals never catch.

Data & AnalyticsFreemium

App: Prefactor

Prefactor Pty Ltd

Agent observability and evaluation that scores every production run for quality, drift and risk — and can pause or block a risky action before it executes.

Coding & DevelopmentFreemium

The standard experiment tracker for machine learning, now with tracing and evaluation for LLM applications.

Data & AnalyticsFreemium

Evaluation-first AI development — measure whether a prompt or model change actually improved anything.

Data & AnalyticsFreemium

Open-source LLM observability — trace, evaluate, and improve AI applications with production data.

Data & AnalyticsFreemium

The framework and observability platform most teams use to build, debug, and ship LLM agents.

Coding & DevelopmentFreemium

App: Dify

LangGenius

An open-source platform for building production LLM applications — visual workflows, RAG, and agents in one place.

Automation & WorkflowsFreemium

Skills

All skills

Diagnoses wrong gradients in differentiable NVIDIA Warp programs by measuring first — comparing autodiff against finite differences on a shrunk reproduction before proposing any fix.

Skill: SkillHone

by Tencent

Featured

Tencent's skill-evolution harness: it rewrites a whole skill folder — SKILL.md, scripts and references together — and lands every decision as a real Git issue, PR and wiki entry you can review.

92 views
New

Microsoft's official skill for building, deploying, evaluating, fine-tuning and operating agents and models on Microsoft Foundry with azd and the Foundry MCP tools, from project setup and RBAC to quota, tracing and CI/CD.

A protocol for measuring whether agents can discover and use your tool without being told about it — WITH/WITHOUT trials, binary scoring, and lift as the only metric. The subject under test is your docs, not the model.

Anthropic's official skill that appends 2-3 targeted follow-up questions to a substantive answer, prompting the reader to check facts, probe assumptions and spot missing context.

NVIDIA's official skill for building synthetic datasets with NeMo Data Designer — describe the data you want and the agent writes a declarative generation pipeline, column by column.

Skill: MCP Builder

by Anthropic

Anthropic's guide for building high-quality MCP servers end to end — research, tool design, implementation in TypeScript or Python, testing, and a 10-question evaluation suite.

Create, edit, and optimize Agent Skills — scaffold new skills, refine existing ones, run evals, and tune descriptions for reliable triggering.

MCP servers

All MCP servers
Featured

The official W&B MCP server: query experiment runs, Weave LLM traces, artifacts and registries in natural language, and write findings back as a W&B report.

Testing and evaluation workbench for MCP server authors — inspect tools and prompts, debug OAuth step by step, and score behaviour across 16 client configurations before shipping.

MCP: Langfuse

by Langfuse

Langfuse's own MCP server for LLM observability — read traces and observations, manage prompt versions, run datasets and evaluators, and query cost and latency metrics from inside your agent.

MCP: Everything

by Model Context Protocol

A reference/test server that exercises every MCP feature — prompts, resources, tools, and sampling — for building and debugging clients.

Related tags

Tags that appear alongside this one, ranked by how often.

All tags