Your app/scripts talk to several model providers (OpenAI, Anthropic, Gemini, local ollama...), each with a different API format, keys scattered around, and no single place to track spend. litellm is that unified gateway: one OpenAI-compatible endpoint (/v1/chat/completions) routing to 100+ providers, with unified key management, cost tracking and load balancing.

Use it when:

• Multi-provider switching/failover: change models behind one base_url without touching code;
• Unified key management: keys live at the proxy, clients carry none;
• Cost tracking: unified accounting across models;
• Matches combo combo-model-access (litellm + logfire observability).

Don't use it when: you use a single provider with no cost concerns (direct is simpler); you need local inference itself (that is ollama's job).

# Source: Trove detailed guide · verified 2026-08-29
python3 -m venv ~/.litellm-venv && ~/.litellm-venv/bin/pip install -U "litellm[proxy]" -i https://pypi.tuna.tsinghua.edu.cn/simple
~/.litellm-venv/bin/litellm --version

Expected output: litellm version (measured 1.98.0; Tsinghua PyPI mirror install works)

Key concept: the openai/mock model (dummy key) validates the whole proxy surface without real keys — used in this guide's smoke test.

Confirm:

• litellm --version prints a version;
• with a mock config, curl /health/liveliness returns ok.

Install variants (deep dive)

pip installs litellm[proxy] (isolated venv recommended); litellm --config <yaml> --port <port> starts the proxy. Config (model_list + litellm_settings) lives at the --config path, owned by the user; no system/user-level config writes. Health check at /health/liveliness, model list at /v1/models.

Key concepts:

• model_list: model registry (model_name + litellm_params.model/api_key);
• openai/mock: keyless smoke model (dummy key validates the proxy surface);
• drop_params: ignore parameters the provider doesn't support (used in the measured smoke config).

Where it writes: no system config; proxy config at the --config path; venv at ~/.litellm-venv (measured here).

Step 1: install the proxy + verify version

# Source: Trove detailed guide · verified 2026-08-29
python3 -m venv ~/.litellm-venv && ~/.litellm-venv/bin/pip install -U "litellm[proxy]" -i https://pypi.tuna.tsinghua.edu.cn/simple
~/.litellm-venv/bin/litellm --version

Expected output: litellm version (measured 1.98.0)

If stuck:

• pip slow/timeout → confirm the -i Tsinghua PyPI mirror prefix;
• venv creation fails → confirm python3 exists and version ≥3.8.

Step 2: write a mock config and start

Explanation: openai/mock needs no real key; verify the proxy surface first, then wire real providers.

# Source: Trove detailed guide · verified 2026-08-29
cat > /tmp/litellm-mock.yaml <<'YAML'
model_list:
  - model_name: mock-model
    litellm_params:
      model: openai/mock
      api_key: dummy-key
litellm_settings:
  drop_params: true
YAML
~/.litellm-venv/bin/litellm --config /tmp/litellm-mock.yaml --port 4000

Expected output: proxy startup log (port 4000 listening; mock smoke measured 2026-08-29)

If stuck:

• port busy → change --port;
• startup error → check YAML indentation (model_list is a list; indentation must be consistent).

Step 3: verify the proxy surface (health + models + one call)

# Source: Trove detailed guide · verified 2026-08-29
curl -s http://127.0.0.1:4000/health/liveliness
curl -s http://127.0.0.1:4000/v1/models
curl -s http://127.0.0.1:4000/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"mock-model","messages":[{"role":"user","content":"hi"}]}'

Expected output: health returns ok; model list includes mock-model; chat/completions returns a mock reply (measured 2026-08-29)

If stuck:

• empty model list → model_list config not loaded; check --config path and YAML;
• call 404 → wrong endpoint path (/v1/chat/completions, not /chat/completions).

Step 4: wire a real provider

Explanation: replace model_list with a real provider (example OpenAI; other provider formats in official docs).

# Source: Trove detailed guide · verified 2026-08-29
# example (no real key locally, not tested): model_list with litellm_params.model: openai/gpt-4o + api_key

Expected output: see official docs (no real provider tested here, zero-key)

If stuck:

• 401 → api_key wrong or missing;
• unknown model name → use the openai/<real-model> format.

Same source as the tool-page matrix; full pairs, states and evidence on the conflict matrix. Known relations:

• logfire × litellm: the proxy can send logs to logfire (observability), complementary (combo-model-access members);
• claude-code-router × litellm: both "multi-model routing" but different forms — ccr manages Claude Code local routing, litellm is a general gateway; different write surfaces (ccr ~/.claude-code-router vs litellm --config), static possible-compatible;
• Others: non-overlapping write surfaces (--config owned), static: can coexist.

• Symptom: empty model list → model_list not loaded; check --config path and YAML indentation
• Symptom: call 404 → wrong endpoint path (/v1/chat/completions)
• Symptom: real provider 401 → api_key wrong or missing
• Symptom: pip won't install → confirm Tsinghua PyPI mirror prefix + venv
• Symptom: port busy → --port change

Full four-section entries (symptom→cause→fix→evidence): site docs (if recorded)

# Source: Trove detailed guide · verified 2026-08-29
rm -rf ~/.litellm-venv

Expected output: after removal ~/.litellm-venv/bin/litellm does not exist

Note: litellm writes no system config; your proxy config lives in your own --config file; removing the venv leaves it untouched.

• Q: What is openai/mock? A: litellm's built-in mock model, no real key needed, validates the proxy surface (measured).
• Q: Where is the config? A: fully managed at the --config path; no system/user-level writes; venv at ~/.litellm-venv.
• Q: Does it overlap with claude-code-router? A: adjacent but different forms — ccr manages Claude Code local model routing; litellm is a general OpenAI-compatible gateway; they can coexist.
• Q: Can it connect ollama? A: yes, litellm_params.model with ollama/<model> plus api_base (not tested; officially supported).
• Q: Where do I report issues? A: install/uninstall issues go through this entry's feedback; project issues go to the author repo.

• Version: 1.98.0 (manifest locked; measured 2026-08-29).
• Environment: Python venv (~/.litellm-venv measured).
• Network: PyPI via Tsinghua mirror (measured).
• Write surface: --config path + venv; no system config.
• Port: 4000 by default (example).

• Sources: official project docs (English README/docs) + local measurements (2026-08-29)
• Entry signature: BerriAI/litellm (MIT, kept as 1.98.0)

• Measured: local repro log (venv install + mock proxy start + /health/liveliness + /v1/models + chat/completions smoke; drop_params config)
• Not tested: real provider integration (zero-key red line); load balancing/cost details; ollama integration.

Third-party tutorials are not individually verified; check against this page and official docs.

• 官方文档层（docs.litellm.ai / PyPI 元数据，v1.98.0）：LiteLLM 完整用法 [官方文档 · LiteLLM 官方 · 2026-08-31（包 1.98.0）]
  官方用法全解——litellm[proxy] 隔离 venv 安装、--config 配置（model_list + litellm_settings）、代理启动与端口、健康检查 /health/liveliness、模型列表 /v1/models、mock 冒烟模型、统一 OpenAI 兼容端点