Arize AX

Shipping an agent is the easy part. Knowing why it answered badly on Tuesday, and whether last week's prompt change actually helped, is the part that needs tooling. Arize AX is a managed AI engineering platform built around that loop: observe what production is doing, evaluate it, and change something with evidence that the change worked.

Observe

AX ingests traces from LLM applications and agents and renders them as a span tree — the router call, the tool calls beneath it, each nested sub-agent, each with its own latency. For multi-step agents that is the difference between "the answer was wrong" and "the orchestrator called the wrong tool at step two". Traces, evals and datasets land in the Arize database, which can also read from BigQuery, Snowflake and Databricks rather than forcing a separate copy of your data.

Evaluate

Evaluations run over traces and datasets, with templates aimed specifically at agents — including trajectory analysis, which scores whether the agent took the right path to the answer rather than only whether the final string matched. Projects are tracked with pass rates you can watch move, and evals are unlimited on every tier, which matters because eval volume is what makes the practice work.

Improve

A side-by-side prompt playground runs two prompt-and-model combinations against the same dataset and shows the diff between them, with an optimised-prompt suggestion path. Because you can change the model as well as the prompt, it doubles as the tool for answering "would a cheaper model still pass?" before a migration rather than after.

Phoenix, and why it matters here

Arize also maintains Phoenix, an open-source, local-first tool covering tracing, evaluation, experimentation and prompt iteration. The intended path is explicit: stay local and open while that is enough, and move to AX when you need the managed side. For teams wary of putting production traces in a vendor on day one, that is a genuinely different adoption story from platforms that only sell the hosted product.

Pricing

AX Free is $0 and real rather than a trial — 25,000 trace spans and 1 GB of ingestion a month, 10 issues, 15-day retention, unlimited users and unlimited evals. AX Pro is $50/month for 25 issues, 50,000 spans, 10 GB and 30-day retention. AX Enterprise is custom, with unlimited signals, negotiated volume and retention, and SaaS or self-hosted deployment. Startup pricing is available on application. The free tier's unlimited-users policy is unusual in this category and worth noting if you want the whole team looking at traces rather than one licensed owner.

Who it is for

AI engineering teams running LLM applications or agents in production who have outgrown log-grepping — and who would like the evaluation half of the loop to be as well-instrumented as the observability half.