OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI APIs & Platforms

Litellm

Litellm is an open-source AI gateway and Python SDK that provides a unified OpenAI-format interface for calling more than 100 LLM providers. It can also use native provider formats and run as a self-hosted proxy server.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Litellm — OpenViST project profile in AI APIs & Platforms.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Litellm when an application or team needs to connect multiple LLM backends while centralizing provider routing, cost tracking, logging, load balancing or guardrails.

Who it is for

  • Developers integrating several LLM providers
  • Platform and infrastructure teams operating shared model access
  • Teams that need centralized routing, observability and governance for LLM requests

Documented uses

  • Call multiple LLM APIs through a common integration surface
  • Route requests across supported model providers
  • Load-balance inference traffic
  • Track LLM costs and request logs
  • Apply configurable guardrails to model access
Before you choose

Requirements & limits

The project is publicly available as open-source software and is classified for cloud, server and self-hosted deployment.

  • Whether the required providers are supported, including OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Azure OpenAI, Vertex AI, vLLM and NVIDIA NIM
  • Whether an OpenAI-format interface is preferable to separate provider integrations
  • Need for routing, load balancing, cost tracking, logging or guardrails
  • Whether a self-hosted proxy deployment fits the team’s architecture
  • Python is the primary implementation language

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Use LiteLLM as a Python SDK for OpenAI- and Anthropic-format calls

Install LiteLLM and call documented OpenAI and Anthropic models through its Python SDK.

Before you start

  • Python environment with uv available.
  • An OpenAI API key and/or Anthropic API key.

Steps

  1. Install the SDK with the command below.

    Documented command
    uv add litellm

  2. Set OPENAI_API_KEY and/or ANTHROPIC_API_KEY in the environment.

  3. Import completion from litellm and call it with an OpenAI-format messages list, using a model identifier such as openai/gpt-4o or anthropic/claude-sonnet-4-20250514.

Expected outcome

The application has a unified Python interface for documented OpenAI and Anthropic model calls.

Guide limitations

  • The documented examples do not establish availability for every provider or model.
  • The examples do not document response handling beyond assigning the returned value.
Documentation 1 ↗Documentation 2 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
LitellmCall multiple LLM APIs through a common integration surfaceCloud, Self-hosted, ServerThe project is publicly available as open-source software and is classified for cloud, server and self-hosted deployment. The supplied evidence does not specify pricing or account requirements.
DeepreasoningSubmit conversational messages through a single inference API.Server, Web
LocalaiServe AI models on self-hosted infrastructure.Docker, Self-hosted, ServerThe project is presented as open source and uses the MIT license. It is documented for self-hosted, server, Docker, and macOS-based use; the evidence does not specify pricing or an account requirement.
Evidence and transparency

Sources & review

Official repository ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
30 September 2026
Original website title
BerriAI/litellm · Official homepage title · retained in its source language
How we review sourcesSuggest a correction