BerriAI / litellm

What This Repo Claims

A Python SDK and Proxy Server (AI Gateway) that gives you one unified interface to call 100+ LLM providers — OpenAI, Anthropic, Gemini, Bedrock, Azure and more — using the OpenAI format.

The core promise: swap providers without rewriting your code. Change only the model string. Everything else stays the same.

45.4k stars. 7.7k forks. Two install paths:

SDK (direct library integration):

pip install litellm

Proxy Server (AI Gateway):

pip install 'litellm[proxy]'
litellm --model gpt-4o

This review tests the SDK path — the simplest and most commonly used install.

What I Tested

Environment:

• macOS, MacBook Pro 14-inch M4 Pro, 24GB RAM
• Python 3.13.5, pip 25.3
• litellm 1.83.14
• Fresh macOS user — no prior Python environment

Test 1: Install

pip3 install litellm

No errors. Clean install.

Test 2: Core claim — same function call, two providers

import litellm

# Anthropic
response = litellm.completion(
    model="anthropic/claude-haiku-4-5-20251001",
    messages=[{"role": "user", "content": "Say exactly: Hello from Anthropic"}]
)
print(response.choices[0].message.content)
# Output: Hello from Anthropic

# OpenAI — only the model string changes
response = litellm.completion(
    model="openai/gpt-4o-mini",
    messages=[{"role": "user", "content": "Say exactly: Hello from OpenAI"}]
)
print(response.choices[0].message.content)
# Output: Hello from OpenAI!

Same function. Same parameters. Same response access pattern. Different providers. Both worked first time.

Test 3: Response object consistency

Both providers return a ModelResponse object with identical structure:

• response.choices[0].message.role → "assistant"
• response.choices[0].message.content → response text
• response.usage.prompt_tokens → integer
• response.usage.completion_tokens → integer
• response.usage.total_tokens → integer
• type(anthropic_response) == type(openai_response) → True

The unified interface holds under inspection — not just at the surface level.

Finding: Response Object is Pydantic, Not a Dict

During testing, calling .keys() on the response object threw an error:

AttributeError: 'ModelResponse' object has no attribute 'keys'

The response is a typed Pydantic ModelResponse, not a plain Python dict. Dict-like access patterns (response.keys(), response["choices"]) do not work.

The README shows OpenAI-style attribute access throughout and does not explicitly document that the object is a Pydantic model. Developers coming from the raw OpenAI SDK — which returns a similar object — will likely not hit this. Developers who assume JSON dict access will.

This is a documentation gap, not a functional failure. The correct access pattern (response.choices[0].message.content) works identically across all providers tested.

What I Did Not Test

• The Proxy Server (AI Gateway) path — requires Docker or a separate service
• Cost tracking and spend management features
• Guardrails and load balancing
• All 100+ claimed providers — only Anthropic and OpenAI tested
• Streaming responses
• Function calling / tool use across providers

This review covers the Python SDK install path and the core provider-switching claim. The proxy server and enterprise features require a separate, longer test.

Verdict: Solid

Install works. Core claim is real and immediately verifiable. Two providers, same function call, same response structure — exactly as documented.

The Pydantic ModelResponse finding is worth knowing before you build on top of litellm. Use attribute access, not dict access. The docs don't make this explicit enough.

45.4k stars. For a library that removes multi-provider friction in 3 lines of Python, the attention is deserved.

Worth installing if you're building anything that touches more than one LLM provider.