Skip to content

What is LiteLLM?

Open-source, self-hosted AI gateway and Python SDK: one OpenAI-compatible API for 100+ providers, with spend tracking, budgets and guardrails.

Pricing
Freemium
Free plan
Yes
Made by
BerriAI
Released
Updated

Compare LiteLLM alternatives

LiteLLM screenshots

LiteLLM features & details

LiteLLM started as a Python library that calls any LLM in the OpenAI request format, and grew into the gateway many platform teams put in front of every model their company uses. It comes in two pieces: the litellm SDK for calling models from code, and the LiteLLM proxy, a self-hosted server with an admin UI.

One API for everything. The proxy exposes a single OpenAI-compatible endpoint that reaches 140+ providers and 1,800+ models — OpenAI, Anthropic, Vertex AI, Bedrock, Azure, self-hosted and fine-tuned models — so switching a backend model is a configuration change, not a code change. Agents and MCP servers can be put behind the same gateway.

Spend control is the core of it. Every request is costed. Virtual keys are issued per team, project or app, each with its own model allow-list, TPM/RPM rate limits and maximum budget, and spend can be broken down by key, team, model or tag (for example by which coding agent sent the request). Budgets are enforced before a request runs, not reported after the bill arrives. Routers handle load balancing and fallbacks to cheaper or healthier deployments.

Governance. SSO, team and member roles, audit logs, and guardrails that plug in PII masking, prompt-injection and secret detection from providers such as Presidio, Lakera or Bedrock Guardrails, run before, during or after the call.

Who it is for. Platform and infrastructure teams that want to give a whole organisation model access through one key while keeping control of cost and data. You run it yourself — a quick-start script, Docker, or one-click deploys to Render, Railway, AWS and GCP.

Pricing. The SDK and proxy are open source and free to self-host; LiteLLM sells an Enterprise tier with additional features and support through its sales team.

No reviews yet — sign in to write the first one

Apps similar to LiteLLM

The closest matches are Helicone and Portkey. See every LiteLLM alternative compared by price and free plan.

Open-source AI gateway and LLM observability: one OpenAI-compatible endpoint for 100+ models, with fallbacks, caching, cost tracking and request tracing.

Coding & DevelopmentFreemium

App: Portkey

Portkey (Palo Alto Networks)

AI gateway and LLMOps platform: one API to 1,600+ models with fallbacks, caching, guardrails, cost tracking and prompt management.

Coding & DevelopmentFreemium

An open-source proxy that sits between your coding agent and the model, compressing tool schemas, file reads and stale history to cut token bills.

Coding & DevelopmentFreemium

Secure cloud sandboxes where AI agents can run untrusted code safely, starting in milliseconds.

Coding & DevelopmentFreemium
Featured

One API and one bill for hundreds of language models, with automatic failover between providers.

Coding & DevelopmentFreemium