OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Xorbits Inference

Xorbits Inference is an open-source Python model-serving system that deploys language, speech and multimodal models behind a unified inference API for cloud, on-premises and local environments.

Independent OpenViST project profile, based on linked sources.

Xorbits Inference — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when an application or internal workflow needs self-hosted access to supported open-source models through a unified inference API across local, cloud or on-premises environments.

Who it is for

  • Researchers deploying open-source AI models
  • Developers building applications around self-hosted inference
  • Data scientists evaluating language, speech and multimodal models
  • Teams operating model services on cloud or on-premises infrastructure

Documented uses

  • Serve open-source language models through a consistent API
  • Deploy speech-recognition and multimodal models
  • Run private inference infrastructure on-premises or locally
  • Switch supported models without redesigning an application's inference interface
  • Provide a local model-serving environment for development
Before you choose

Requirements & limits

The project is available as a public open-source repository under the Apache-2.0 license and includes self-hosting documentation.

  • The project is infrastructure for serving models, not an end-user AI application.
  • Python is the primary implementation language.
  • The documented deployment scope includes cloud, on-premises environments and a laptop.
  • A unified API can provide a common application interface across supported models.
  • The repository highlights automatic batching, distributed inference and support for multiple model types.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Xorbits InferenceServe open-source language models through a consistent APICloud, Desktop, Self-hosted, ServerThe project is available as a public open-source repository under the Apache-2.0 license and includes self-hosting documentation.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
2 October 2026
Original website title
xorbitsai/inference · Official homepage title · retained in its source language
How we review sourcesSuggest a correction