OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Infinity — michaelfeil

Infinity is a Python server-side model-serving engine that exposes a REST API for text embeddings, reranking, CLIP, CLAP and ColPali. It can be launched through a CLI or Docker, and also offers asynchronous Python interfaces for local processing and a client package for calling a remote instance.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Infinity — michaelfeil — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Infinity when you need to operate the model-serving layer for embedding, reranking or supported multimodal inference, and can select models and runtime engines compatible with your environment.

Who it is for

  • Teams building retrieval applications that need to serve text embeddings.
  • Operators who want to host a reranking endpoint for ordering search results.
  • Developers serving supported multimodal models through an API.
  • Python developers who want asynchronous embedding or reranking interfaces.

Documented uses

  • Serve text-embedding models behind an API for application requests.
  • Score a query against multiple documents to support reranking.
  • Host multiple models from one v2 CLI launch.
  • Call a remote Infinity REST API using its Python client.
Before you choose

Requirements & limits

The project is available as a public, MIT-licensed GitHub repository. The README documents Python-package and Docker-based deployment.

  • Engine compatibility is model-specific: Torch requires Sentence Transformers and AutoModel compatibility, Optimum requires an ONNX file, and CTranslate2 supports BERT models.
  • The README lists CUDA, ROCm, CPU, AWS Inferentia 2 and Apple MPS options; accelerator requirements depend on the host and container image.
  • The Docker example publishes a port and mounts a cache volume; its GPU setup calls for NVIDIA Docker and GPU access.
  • CLI arguments can be passed as command-line options or environment variables, with repeated model identifiers for multi-model launches.
  • The Python API uses asyncio for concurrent request processing, while the separate client package is intended to call a remote REST instance.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Start the v2 CLI with one embedding model

Install Infinity and launch its CLI with a Hugging Face model identifier.

Before you start

  • A Python virtual environment, as specified by the README.
  • A model identifier, such as BAAI/bge-small-en-v1.5.

Steps

  1. Install the package with the command below.

    Documented command
    pip install infinity-emb[all]

  2. With the virtual environment active, run infinity_emb v2 --model-id BAAI/bge-small-en-v1.5.

  3. Use infinity_emb v2 --help to inspect available parameters.

Expected outcome

The CLI is launched with the selected model identifier.

Guide limitations

  • This setup guidance does not specify a client request or verify that a model runs on a particular machine.
  • Model and inference-engine compatibility depends on the selected model and engine.
Documentation 1 ↗
Source-based practical guide

Configure a multi-model CLI launch

Specify multiple models and per-model batch sizes using the v2 CLI.

Before you start

  • Infinity CLI v2.
  • The model identifiers and batch sizes to configure.

Steps

  1. Repeat --model-id for each model and, if needed, repeat --batch-size for corresponding values.

  2. Alternatively, set INFINITY_MODEL_ID and INFINITY_BATCH_SIZE with semicolon-separated values.

  3. Launch the v2 CLI using the configured arguments or environment variables.

Expected outcome

The CLI configuration describes multiple models in one launch; the README says API requests are routed using the served model name or model ID.

Guide limitations

  • The README directs users to infinity_emb v2 --help for argument validation and full options.
  • This recipe documents configuration, not a successful deployment or request.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Infinity — michaelfeilServe text-embedding models behind an API for application requests.Docker, Self-hosted, ServerThe project is available as a public, MIT-licensed GitHub repository. The README documents Python-package and Docker-based deployment.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
10 October 2026
Original website title
michaelfeil/infinity · Official homepage title · retained in its source language
How we review sourcesSuggest a correction