20 Promptfoo alternatives
Promptfoo: Open-source CLI for evaluating and red-teaming LLM apps: compare models side by side, assert on outputs in CI, and scan agents and RAG pipelines for jailbreaks and data leaks. 20 apps do similar work, most sharing Open Source, CLI and Evaluation, ranked by how much they have in common with Promptfoo.
Why look for a Promptfoo alternative? Promptfoo's free plan comes with paid upgrades; 12 alternatives below are free outright. Several also cover Developers, Coding Agent and Terminal, which Promptfoo's listing does not mention.
Promptfoo vs. the top 5 alternatives
| App | Pricing | Free plan | Category | Compared with Promptfoo |
|---|---|---|---|---|
| Promptfoo | Freemium | Yes | Coding & Development | The app these are compared with |
| MCPJam | Freemium | Yes | Coding & Development | Also covers MCP and OAuth |
| Oqoqo | Freemium | Yes | Coding & Development | Also covers Coding Agent and Developers |
| Cekura | Freemium | Yes | Coding & Development | Also covers Developers and Monitoring |
| Microsoft Execution Containers (MXC) | Free | Yes | Coding & Development | Free to use; Also covers Agents and Sandbox |
| Kane CLI | Freemium | Yes | Coding & Development | Also covers Browser Automation and Developers |
All 20 Promptfoo alternatives
-
Open-source testing and evals platform for MCP servers — trace tool calls, debug OAuth, and catch cross-client regressions in CI.
vs Promptfoo: Also covers MCP and OAuth
Shared with Promptfoo: Evaluation, Open Source, Testing
-
Run agent evals and private benchmarks in sandboxed cloud environments — compare Claude Code, Codex, Cursor and Copilot on the same real tasks.
vs Promptfoo: Also covers Coding Agent and Developers
Shared with Promptfoo: Evaluation, Testing
-
Tatva Labs Inc.
Continuous automated QA for voice and chat AI agents, from pre-launch simulation to live call monitoring.
vs Promptfoo: Also covers Developers and Monitoring
Shared with Promptfoo: Evaluation, Testing
-
Microsoft
Microsoft's open-source, policy-driven sandbox SDK for running agents, model-generated code, tools and plugins with only the files and network access you grant.
vs Promptfoo: Free to use · Also covers Agents and Sandbox
Shared with Promptfoo: Open Source, Security
-
TestMu AI
Describe a browser or mobile test in plain English and run it from your terminal — Kane CLI drives real Chrome, no selectors or test scripts required.
vs Promptfoo: Also covers Browser Automation and Developers
-
Microsoft
Microsoft's open-source optimizer that trains an agent's SKILL.md like a model parameter — epochs, learning-rate budgets and a held-out validation gate, with frozen weights.
vs Promptfoo: Free to use · Also covers Developers and Microsoft
Shared with Promptfoo: Evaluation, Open Source
-
An open-source TypeScript framework for building AI agents, paired with VoltOps, a console for tracing, evals, prompt management and deployment.
vs Promptfoo: Also covers Agents and Observability
Shared with Promptfoo: Evaluation, Open Source
-
The framework and observability platform most teams use to build, debug, and ship LLM agents.
vs Promptfoo: Also covers Developers and Observability
Shared with Promptfoo: Evaluation, Open Source
-
Open-source AI coding agent for VS Code, JetBrains and the terminal — 500+ models at provider cost, task-specific modes, parallel agents in isolated git worktrees.
vs Promptfoo: Also covers Code Generation and Coding Agent
Shared with Promptfoo: CLI, Open Source
-
Agentic AI Foundation
Open-source AI agent from the Agentic AI Foundation — a native desktop app, CLI and API that runs tasks locally with 70+ MCP extensions.
vs Promptfoo: Free to use · Also covers Coding Agent and Desktop App
Shared with Promptfoo: CLI, Open Source
-
Abhishek Iyer
A local, keyboard-driven diff viewer built for the huge changes coding agents produce - and it remembers what you have already seen.
vs Promptfoo: Free to use · Also covers Code Review and Coding Agent
Shared with Promptfoo: CLI, Open Source
-
Mario Zechner
Minimal, MIT-licensed terminal coding agent that adapts to your workflow via TypeScript extensions.
vs Promptfoo: Free to use · Also covers Coding Agent and Developers
Shared with Promptfoo: CLI, Open Source
-
Vercel Labs
Vercel Labs' tiny open-source coding agent: a ~6MB Zig binary that cold-starts in microseconds and embeds anywhere.
vs Promptfoo: Free to use · Also covers Coding Agent and Developers
Shared with Promptfoo: CLI, Open Source
-
RTK AI
Open-source Rust CLI that rewrites the shell commands AI coding agents run and strips noise from their output, cutting the tokens Claude Code, Cursor or Codex send to the model.
vs Promptfoo: Free to use · Also covers Coding Agent and Cost Optimization
Shared with Promptfoo: CLI, Open Source
-
Qwen Team, Alibaba Cloud
Open-source AI coding agent from Alibaba's Qwen team for the terminal, IDEs, desktop and chat, with skills, subagents, memory and MCP, and any OpenAI, Anthropic, Gemini or local model.
vs Promptfoo: Free to use · Also covers Coding Agent and IDE
Shared with Promptfoo: CLI, Open Source
-
Prompt One, Inc.
Open-source CLI framework that gives AI agents one command grammar across Salesforce, ServiceNow, DocuSign and Google Workspace, loading each schema only when asked.
vs Promptfoo: Free to use · Also covers Agents and Developers
Shared with Promptfoo: CLI, Open Source
-
Continue
Open-source AI coding agent for VS Code, JetBrains and the terminal; acquired by Cursor in 2026 and no longer maintained.
vs Promptfoo: Free to use · Also covers Autocomplete and JetBrains
Shared with Promptfoo: CLI, Open Source
-
Open-source local inference server that profiles your hardware, picks models that fit, and points your coding agent at them — free, private and offline.
vs Promptfoo: Free to use · Also covers Coding Agent and Inference
Shared with Promptfoo: CLI, Open Source
-
SST
An open-source terminal coding agent that works with any model and shares sessions across devices.
vs Promptfoo: Free to use · Also covers Coding Agent and Developers
Shared with Promptfoo: CLI, Open Source
-
Evaluation-first AI development — measure whether a prompt or model change actually improved anything.
vs Promptfoo: Also covers Developers and Observability
Shared with Promptfoo: Evaluation, Testing