OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Inferoxy

Inferoxy is self-hosted infrastructure for deploying Dockerized computer-vision models and serving inference requests through REST, gRPC or ZeroMQ bridges. It batches requests, manages model information through Redis, and supports operation on a single machine or a multi-node cluster.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Inferoxy — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Inferoxy when you already have computer-vision models packaged as Docker images and need a self-hosted serving layer for one machine or a cluster, particularly when batching or stateful model handling matters.

Who it is for

  • Teams packaging computer-vision models as Docker images for inference.
  • Operators running inference on a CPU/GPU machine or Kubernetes cluster.
  • Developers who need batching and model-copy management, including for stateful video-processing tasks.

Documented uses

  • Serve requests to Dockerized computer-vision models through supported bridges.
  • Batch inference work for multi-GPU or multi-node deployments.
  • Store model metadata such as image address and maximum batch size through Redis.
  • Track running model copies, load, health and alerts in a cluster.
Before you choose

Requirements & limits

  • The README documents local Docker and Kubernetes deployment options.
  • The stated minimum is 4 GB RAM plus a CPU or GPU; hardware choice depends on speed and cost trade-offs.
  • A model must be packaged as a Docker image and pushed to an accessible registry.
  • Redis stores model information, and the documented local setup connects it to the Inferoxy Docker network.
  • The repository is public and identifies its license as GPL-3.0.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare a local Docker configuration

Set up the documented configuration files and Redis dependency for a local Inferoxy deployment.

Before you start

  • A machine with at least 4 GB of RAM and a CPU or GPU.
  • Docker and access to the required Docker images and model registry.
  • The Inferoxy repository's local-run instructions.

Steps

  1. Pull the Inferoxy Docker image specified in the README.

  2. Create a .env file with the documented Docker network and Redis model-storage settings.

  3. Create the inferoxy Docker network and start Redis on that network.

  4. Create a models.yaml file with model image address and batch configuration.

Expected outcome

The local configuration and Redis model-information store are prepared for the documented Inferoxy container startup stage.

Guide limitations

  • These steps cover preparation only; they do not establish that a deployment or inference request succeeds.
  • The model image must be accessible to the deployment.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
InferoxyServe requests to Dockerized computer-vision models through supported bridges.Docker, Linux, Self-hosted, Server
FriendliAISend inference requests to a hosted open-weight model through a Model API.Web, CloudThe official documentation lists Model APIs and Dedicated Endpoints. It also describes Model APIs as serverless and pay-per-token; the evidence does not specify prices.
VllmopsDefine model server settings such as model identifier, GPU selection, port, environment, and vLLM arguments.CLI, Server, Self-hostedThe documented installation requires Python 3.10 or later and supports Linux and macOS. The repository specifies the Apache-2.0 license.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
24 October 2026
Original website title
eora-ai/inferoxy · Official homepage title · retained in its source language
How we review sourcesSuggest a correction