litellm — Agent Guide

No conflict-matrix pairs for this tool yet.

# LiteLLM — Agent Guide

name: litellm ｜ version_range: 1.98.0~1.98.0 ｜ generated: 2026-08-29

title_en: LiteLLM - Unified LLM Gateway (OpenAI-compatible proxy) ｜ desc_en: One OpenAI-compatible endpoint for 100+ model providers, with unified key management, cost tracking, load balancing.

Preflight

note: Requires Python >= 3.8; isolated venv recommended.

python3 --version

预期输出: see command output

Install

note: Tsinghua mirror required in mainland China (verified 80s install). Real usage needs model API keys; smoke test uses openai/mock + dummy key.

python3 -m venv ~/.litellm-venv
~/.litellm-venv/bin/pip install -i https://pypi.tuna.tsinghua.edu.cn/simple litellm[proxy]
~/.litellm-venv/bin/litellm --version

预期输出: LiteLLM: Current Version = 1.98.0

Operations

Write a minimal proxy config (mock smoke)

note: Config file is user-managed via --config path; no system config writes.

cat > /tmp/ltm-config.yaml <<'EOF'
model_list:
  - model_name: mock-gpt
    litellm_params:
      model: openai/mock
      api_key: dummy
litellm_settings:
  drop_params: true
EOF

预期输出: config file ready

Start the proxy

note: Default port 4000; --port to change.

~/.litellm-venv/bin/litellm --config /tmp/ltm-config.yaml --port 8099

预期输出: Uvicorn running on http://0.0.0.0:8099

Health check and model list

note: Proves proxy surface works without real keys.

curl -s http://127.0.0.1:8099/health/liveliness
curl -s http://127.0.0.1:8099/v1/models

预期输出: "I'm alive!"; {"data":[{"id":"mock-gpt",...}]}

Uninstall

note: Stop proxy process, then remove venv.

pkill -f 'litellm.*--config' 2>/dev/null || true
rm -rf ~/.litellm-venv

预期输出: no leftovers

Facts

writes: proxy config file (--config path, user-managed) ｜ runtime: litellm proxy service (Uvicorn, default port 4000) ｜ notes: no system/user config writes; server-side gateway (unified endpoint for multiple services); real forwarding needs real API keys - mock smoke verified 2026-08-29; judged possible-compatible with claude-code-router (client-side) and logfire (observability).

Source: litellm · validated 2026-08-29 · v1.98.0~1.98.0

← Agent index