What is liteLLM?

LiteLLM is an open‑source gateway that standardizes access to over 100 large language models, including OpenAI, Azure, Bedrock, Gemini, and GCP. It routes requests through an OpenAI‑compatible API, enabling seamless fallback to alternate providers when a model is unavailable.

Built-in cost tracking logs spend per key, user, team, or organization, and supports tag‑based attribution for budgeting and rate limiting. The platform offers guardrails, prompt management, and pass‑through endpoints, while automatically logging usage to S3, GCS, or other storage backends.

Users can deploy LiteLLM on‑prem or in the cloud, with Docker images and a lightweight Python SDK for integration into existing infrastructure. Logging and observability are supported via OpenTelemetry, Prometheus, and third‑party services such as Langfuse, Arize Phoenix, and Langsmith.

Enterprise deployments add JWT authentication, SSO, audit logs, custom SLAs, and virtual key management for secure, scalable LLM usage. LiteLLM provides rate limits, RPM/TPM controls, and load balancing to manage traffic and maintain consistent performance.

liteLLM pricing Freemium

Open source $0

Enterprise get quote

Verify on the official pricing page.

View plans

liteLLM user reviews

Would you recommend liteLLM?

Recommend this tool?

liteLLM's key features

OpenAI-compatible API gateway
Spend tracking with budgets
Rate limiting and guardrails
Multi-provider fallback support
Logging to S3, GCS, Prometheus
Virtual keys and JWT auth
Self-hosted or cloud deployment

liteLLM use cases

Build an internal chatbot that automatically switches between GPT‑4 and open‑source models based on cost thresholds and compliance tags, ensuring seamless user experience without manual model selection.
Integrate LiteLLM into your data pipeline to centrally log and analyze LLM usage, generating real‑time cost dashboards and trigger alerts when budgets are exceeded, all while maintaining privacy through on‑prem deployment.
Create a multi‑tenant SaaS application that routes user prompts through different LLM providers according to region and latency, applying guardrails to filter harmful content and providing end‑to‑end observability via built‑in monitoring hooks.