Open-source AI gateway and LLM observability: one OpenAI-compatible endpoint for 100+ models, with fallbacks, caching, cost tracking and request tracing.
LiteLLM
What is LiteLLM?
Open-source, self-hosted AI gateway and Python SDK: one OpenAI-compatible API for 100+ providers, with spend tracking, budgets and guardrails.
- Pricing
- Freemium
- Free plan
- Yes
- Category
- Coding & Development
- Made by
- BerriAI
- Released
- Updated
LiteLLM screenshots
LiteLLM features & details
LiteLLM started as a Python library that calls any LLM in the OpenAI request format, and grew into the gateway many platform teams put in front of every model their company uses. It comes in two pieces: the litellm SDK for calling models from code, and the LiteLLM proxy, a self-hosted server with an admin UI.
One API for everything. The proxy exposes a single OpenAI-compatible endpoint that reaches 140+ providers and 1,800+ models — OpenAI, Anthropic, Vertex AI, Bedrock, Azure, self-hosted and fine-tuned models — so switching a backend model is a configuration change, not a code change. Agents and MCP servers can be put behind the same gateway.
Spend control is the core of it. Every request is costed. Virtual keys are issued per team, project or app, each with its own model allow-list, TPM/RPM rate limits and maximum budget, and spend can be broken down by key, team, model or tag (for example by which coding agent sent the request). Budgets are enforced before a request runs, not reported after the bill arrives. Routers handle load balancing and fallbacks to cheaper or healthier deployments.
Governance. SSO, team and member roles, audit logs, and guardrails that plug in PII masking, prompt-injection and secret detection from providers such as Presidio, Lakera or Bedrock Guardrails, run before, during or after the call.
Who it is for. Platform and infrastructure teams that want to give a whole organisation model access through one key while keeping control of cost and data. You run it yourself — a quick-start script, Docker, or one-click deploys to Render, Railway, AWS and GCP.
Pricing. The SDK and proxy are open source and free to self-host; LiteLLM sells an Enterprise tier with additional features and support through its sales team.
No reviews yet — sign in to write the first one
Apps similar to LiteLLM
The closest matches are Helicone and Portkey. See every LiteLLM alternative compared by price and free plan.
AI gateway and LLMOps platform: one API to 1,600+ models with fallbacks, caching, guardrails, cost tracking and prompt management.
An open-source proxy that sits between your coding agent and the model, compressing tool schemas, file reads and stale history to cut token bills.
Secure cloud sandboxes where AI agents can run untrusted code safely, starting in milliseconds.
One API and one bill for hundreds of language models, with automatic failover between providers.