What This Repo Claims
A Python SDK and Proxy Server (AI Gateway) that gives you one unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure and more — using the OpenAI format.
The core promise: swap providers without rewriting your code. Change only the model string. Everything else stays the same.
45.4k stars. 7.7k forks. Two install paths:
SDK (direct library integration):
pip install litellm
Proxy Server (AI Gateway):
pip install 'litellm[proxy]'
litellm --model gpt-4o
This review tests the SDK path — the simplest and most commonly used install.
What I Tested
Environment:
- macOS, MacBook Pro 14-inch M4 Pro, 24GB RAM
- Python 3.13.5, pip 25.3
- litellm 1.83.14
- Fresh macOS user — no prior Python environment
Test 1: Install
pip3 install litellm
No errors. Clean install.
Test 2: Core claim — same function call, two providers
import litellm
# Anthropic
response = litellm.completion(
model="anthropic/claude-haiku-4-5-20251001",
messages=[{"role": "user", "content": "Say exactly: Hello from Anthropic"}]
)
print(response.choices[0].message.content)
# Output: Hello from Anthropic
# OpenAI — only the model string changes
response = litellm.completion(
model="openai/gpt-4o-mini",
messages=[{"role": "user", "content": "Say exactly: Hello from OpenAI"}]
)
print(response.choices[0].message.content)
# Output: Hello from OpenAI!
Same function. Same parameters. Same response access pattern.
Different providers. Both worked first time.
Test 3: Response object consistency
Both providers return a
ModelResponse object with identical structure:
response.choices[0].message.role→ "assistant"
response.choices[0].message.content→ response text
response.usage.prompt_tokens→ integer
response.usage.completion_tokens→ integer
response.usage.total_tokens→ integer
type(anthropic_response) == type(openai_response)→ True
The unified interface holds under inspection — not just at the surface level.
Finding: Response Object is Pydantic, Not a Dict
During testing, calling
.keys() on the response object threw an error:
AttributeError: 'ModelResponse' object has no attribute 'keys'
The response is a typed Pydantic
ModelResponse, not a plain Python dict. Dict-like access patterns (response.keys(),
response["choices"]) do not work.
The README shows OpenAI-style attribute access throughout and does not explicitly document that the object is a Pydantic model. Developers coming from the raw OpenAI SDK — which returns a similar object — will likely not hit this. Developers who assume JSON dict access will.
This is a documentation gap, not a functional failure. The correct access pattern (
response.choices[0].message.content)
works identically across all providers tested.
What I Did Not Test
- The Proxy Server (AI Gateway) path — requires Docker or a separate service
- Cost tracking and spend management features
- Guardrails and load balancing
- All 100+ claimed providers — only Anthropic and OpenAI tested
- Streaming responses
- Function calling / tool use across providers
This review covers the Python SDK install path and the core provider-switching claim. The proxy server and enterprise
features require a separate, longer test.
Verdict: Solid
Install works. Core claim is real and immediately verifiable. Two providers, same function call, same response structure —
exactly as documented.
The Pydantic ModelResponse finding is worth knowing before you build on top of litellm. Use attribute access, not dict
access. The docs don't make this explicit enough.
45.4k stars. For a library that removes multi-provider friction in 3 lines of Python, the attention is deserved.
Worth installing if you're building anything that touches more than one LLM provider.