Skip to content

20 Promptfoo alternatives

Promptfoo: Open-source CLI for evaluating and red-teaming LLM apps: compare models side by side, assert on outputs in CI, and scan agents and RAG pipelines for jailbreaks and data leaks. 20 apps do similar work, most sharing Open Source, CLI and Evaluation, ranked by how much they have in common with Promptfoo.

Why look for a Promptfoo alternative? Promptfoo's free plan comes with paid upgrades; 12 alternatives below are free outright. Several also cover Developers, Coding Agent and Terminal, which Promptfoo's listing does not mention.

Back to Promptfoo

Promptfoo vs. the top 5 alternatives

AppPricingFree planCategoryCompared with Promptfoo
PromptfooFreemiumYesCoding & DevelopmentThe app these are compared with
MCPJamFreemiumYesCoding & DevelopmentAlso covers MCP and OAuth
OqoqoFreemiumYesCoding & DevelopmentAlso covers Coding Agent and Developers
CekuraFreemiumYesCoding & DevelopmentAlso covers Developers and Monitoring
Microsoft Execution Containers (MXC)FreeYesCoding & DevelopmentFree to use; Also covers Agents and Sandbox
Kane CLIFreemiumYesCoding & DevelopmentAlso covers Browser Automation and Developers

All 20 Promptfoo alternatives

  1. Open-source testing and evals platform for MCP servers — trace tool calls, debug OAuth, and catch cross-client regressions in CI.

    vs Promptfoo: Also covers MCP and OAuth

    Shared with Promptfoo: Evaluation, Open Source, Testing

  2. Run agent evals and private benchmarks in sandboxed cloud environments — compare Claude Code, Codex, Cursor and Copilot on the same real tasks.

    vs Promptfoo: Also covers Coding Agent and Developers

    Shared with Promptfoo: Evaluation, Testing

  3. Tatva Labs Inc.

    Continuous automated QA for voice and chat AI agents, from pre-launch simulation to live call monitoring.

    vs Promptfoo: Also covers Developers and Monitoring

    Shared with Promptfoo: Evaluation, Testing

  4. Microsoft

    Microsoft's open-source, policy-driven sandbox SDK for running agents, model-generated code, tools and plugins with only the files and network access you grant.

    vs Promptfoo: Free to use · Also covers Agents and Sandbox

    Shared with Promptfoo: Open Source, Security

  5. TestMu AI

    Describe a browser or mobile test in plain English and run it from your terminal — Kane CLI drives real Chrome, no selectors or test scripts required.

    vs Promptfoo: Also covers Browser Automation and Developers

    Shared with Promptfoo: CLI, Testing

  6. Microsoft

    Microsoft's open-source optimizer that trains an agent's SKILL.md like a model parameter — epochs, learning-rate budgets and a held-out validation gate, with frozen weights.

    vs Promptfoo: Free to use · Also covers Developers and Microsoft

    Shared with Promptfoo: Evaluation, Open Source

  7. An open-source TypeScript framework for building AI agents, paired with VoltOps, a console for tracing, evals, prompt management and deployment.

    vs Promptfoo: Also covers Agents and Observability

    Shared with Promptfoo: Evaluation, Open Source

  8. The framework and observability platform most teams use to build, debug, and ship LLM agents.

    vs Promptfoo: Also covers Developers and Observability

    Shared with Promptfoo: Evaluation, Open Source

  9. Open-source AI coding agent for VS Code, JetBrains and the terminal — 500+ models at provider cost, task-specific modes, parallel agents in isolated git worktrees.

    vs Promptfoo: Also covers Code Generation and Coding Agent

    Shared with Promptfoo: CLI, Open Source

  10. Agentic AI Foundation

    Open-source AI agent from the Agentic AI Foundation — a native desktop app, CLI and API that runs tasks locally with 70+ MCP extensions.

    vs Promptfoo: Free to use · Also covers Coding Agent and Desktop App

    Shared with Promptfoo: CLI, Open Source

  11. Abhishek Iyer

    A local, keyboard-driven diff viewer built for the huge changes coding agents produce - and it remembers what you have already seen.

    vs Promptfoo: Free to use · Also covers Code Review and Coding Agent

    Shared with Promptfoo: CLI, Open Source

  12. Mario Zechner

    Minimal, MIT-licensed terminal coding agent that adapts to your workflow via TypeScript extensions.

    vs Promptfoo: Free to use · Also covers Coding Agent and Developers

    Shared with Promptfoo: CLI, Open Source

  13. Vercel Labs

    Vercel Labs' tiny open-source coding agent: a ~6MB Zig binary that cold-starts in microseconds and embeds anywhere.

    vs Promptfoo: Free to use · Also covers Coding Agent and Developers

    Shared with Promptfoo: CLI, Open Source

  14. RTK AI

    Open-source Rust CLI that rewrites the shell commands AI coding agents run and strips noise from their output, cutting the tokens Claude Code, Cursor or Codex send to the model.

    vs Promptfoo: Free to use · Also covers Coding Agent and Cost Optimization

    Shared with Promptfoo: CLI, Open Source

  15. Qwen Team, Alibaba Cloud

    Open-source AI coding agent from Alibaba's Qwen team for the terminal, IDEs, desktop and chat, with skills, subagents, memory and MCP, and any OpenAI, Anthropic, Gemini or local model.

    vs Promptfoo: Free to use · Also covers Coding Agent and IDE

    Shared with Promptfoo: CLI, Open Source

  16. Prompt One, Inc.

    Open-source CLI framework that gives AI agents one command grammar across Salesforce, ServiceNow, DocuSign and Google Workspace, loading each schema only when asked.

    vs Promptfoo: Free to use · Also covers Agents and Developers

    Shared with Promptfoo: CLI, Open Source

  17. Continue

    Open-source AI coding agent for VS Code, JetBrains and the terminal; acquired by Cursor in 2026 and no longer maintained.

    vs Promptfoo: Free to use · Also covers Autocomplete and JetBrains

    Shared with Promptfoo: CLI, Open Source

  18. Open-source local inference server that profiles your hardware, picks models that fit, and points your coding agent at them — free, private and offline.

    vs Promptfoo: Free to use · Also covers Coding Agent and Inference

    Shared with Promptfoo: CLI, Open Source

  19. SST

    An open-source terminal coding agent that works with any model and shares sessions across devices.

    vs Promptfoo: Free to use · Also covers Coding Agent and Developers

    Shared with Promptfoo: CLI, Open Source

  20. Evaluation-first AI development — measure whether a prompt or model change actually improved anything.

    vs Promptfoo: Also covers Developers and Observability

    Shared with Promptfoo: Evaluation, Testing