Scribble Profiles
Browse the directory
Scribble ProfilesLiving profile dossier
Product profile Living profile

LiteLLM

litellm.ai ↗

By Berrie AI Incorporated (LiteLLM)

LiteLLM is an open-source, self-hosted AI gateway and Python SDK that connects applications to 140+ LLM providers and 1,892 models through one OpenAI-compatible API.

AI & AutomationPublic web research
Visit product ↗AI Profile ↗
Source-backed public recordPrimary source ↗

Snapshot

Continue to About ↓
What it does
  • LiteLLM is an open-source, self-hosted AI gateway and Python SDK that connects applications to 140+ LLM providers and 1,892 models through one OpenAI-compatible API.
More detail
  • It centralizes virtual keys, budgets, rate limits, spend tracking, routing, and guardrails, and can also route to MCP servers and agents.
  • The MIT-licensed core runs in your own cloud, on-prem, or air-gapped; LiteLLM Enterprise adds governance and support.
Best for
  • Platform and infrastructure teams running AI in production that need centralized key management, hard spend caps, model routing, and self-hosted or air-gapped deployment.
More detail
  • Python developers who want one-call access to many providers via the SDK.
Pricing
  • The open-source core is MIT-licensed and free to use in production; the enterprise/ directory requires a paid license.
More detail
  • LiteLLM adds no per-request, per-token, or percentage fee on model cost.
  • LiteLLM Enterprise is an annual license priced on request capacity, installation architecture, and support (
Main limitation
  • Teams wanting a fully managed SaaS with zero operational burden, or organizations that cannot absorb frequent security upgrades and a self-hosted Postgres/Redis stack.
Record
Living and monitored
Reviewed
Evidence
Public web research

About

  • LiteLLM is an open-source, self-hosted AI gateway and Python SDK that connects applications to 140+ LLM providers and 1,892 models through one OpenAI-compatible API.
  • It centralizes virtual keys, budgets, rate limits, spend tracking, routing, and guardrails, and can also route to MCP servers and agents.
  • The MIT-licensed core runs in your own cloud, on-prem, or air-gapped; LiteLLM Enterprise adds governance and support.

What it helps with

Capabilities

A practical view of what is available and where limitations apply.

OfferingAvailableDetail
OpenAI-compatible unified API YesOne API to 140+ providers and 1,892 models; swap models without changing app code.
Spend tracking and hard budgets YesUsage and spend tracked per key, user, team, org, tool, agent, and MCP; requests stop at the cap.
Rate limits and leaked-key protection YesPer-team TPM/RPM limits and protection so runaway jobs or compromised keys cannot run up bills.
Routing and cost optimization YesLoad balancing, lowest-cost routing, Auto Routing, response and semantic caching (Redis, S3, GCS), and prompt compression.
Guardrails and audit logs YesPII masking, prompt-injection detection, secret detection, moderation, and an audit log on every request.
MCP and agent gateway YesMCP servers behind one fixed endpoint with per-tool access control; A2A agents, Vertex AI Agent Engine, LangGraph, Azure AI Foundry, Bedrock AgentCore, and Pydantic AI.
Python SDK Yeslitellm package calls 100+ LLM APIs in OpenAI format with retries, fallbacks, and cost tracking.
Self-hosted and air-gapped deployment YesOfficial Helm chart, Terraform module, Docker images; runs on your Postgres and Redis and sends no telemetry to LiteLLM (first-party claim).
SSO, SCIM, and Audit Logs (open source) LimitedSSO, SCIM, OIDC/JWT auth, audit logs, secret managers with key rotation, and multi-region control plane are Enterprise add-ons, not the MIT core.
Managed SaaS option NoLiteLLM is self-hosted software; buyers operate the gateway and its Postgres/Redis dependencies.

Bot trust

How confidently AI systems can verify this profile from public evidence.

—of 100
ChatGPT—/100
Claude—/100

This profile has not completed a Bot Score assessment yet.

Evidence

Where this record came from, when it was reviewed, and how confidently it can be used.

Verification
  • Scribble researched these facts from the linked public sources.
  • The owner has not manually confirmed every field.
Last reviewed
Primary source
litellm.ai
Technical profile details
Record IDaa1c3274-a330-44b4-a707-a65c8562ecc8Canonical URLhttps://profiles.scribble.network/listing/litellm-aa1c3274