LiteLLM

LiteLLM started as a Python library that calls any LLM in the OpenAI request format, and grew into the gateway many platform teams put in front of every model their company uses. It comes in two pieces: the litellm SDK for calling models from code, and the LiteLLM proxy, a self-hosted server with an admin UI.

One API for everything. The proxy exposes a single OpenAI-compatible endpoint that reaches 140+ providers and 1,800+ models — OpenAI, Anthropic, Vertex AI, Bedrock, Azure, self-hosted and fine-tuned models — so switching a backend model is a configuration change, not a code change. Agents and MCP servers can be put behind the same gateway.

Spend control is the core of it. Every request is costed. Virtual keys are issued per team, project or app, each with its own model allow-list, TPM/RPM rate limits and maximum budget, and spend can be broken down by key, team, model or tag (for example by which coding agent sent the request). Budgets are enforced before a request runs, not reported after the bill arrives. Routers handle load balancing and fallbacks to cheaper or healthier deployments.

Governance. SSO, team and member roles, audit logs, and guardrails that plug in PII masking, prompt-injection and secret detection from providers such as Presidio, Lakera or Bedrock Guardrails, run before, during or after the call.

Who it is for. Platform and infrastructure teams that want to give a whole organisation model access through one key while keeping control of cost and data. You run it yourself — a quick-start script, Docker, or one-click deploys to Render, Railway, AWS and GCP.

Pricing. The SDK and proxy are open source and free to self-host; LiteLLM sells an Enterprise tier with additional features and support through its sales team.