OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Vllmops

Vllmops is a self-hosted control plane for bare-metal vLLM model-serving servers. It uses per-model YAML files and project profiles, with CLI and TUI tools for lifecycle operations, logs, health checks, and metrics.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Vllmops — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Vllmops when you already plan to run vLLM on your own bare-metal servers and want project-based configuration plus terminal controls for model operations. Its proxy workflow is relevant when clients should reach several model servers through a single LiteLLM gateway.

Who it is for

  • Teams operating vLLM inference servers on their own Linux or macOS machines.
  • Operators who want model and profile configuration in project YAML files.
  • Developers who need terminal-based lifecycle controls and per-model operational visibility.

Documented uses

  • Define model server settings such as model identifier, GPU selection, port, environment, and vLLM arguments.
  • Start, stop, restart, check, and view logs for individual models or profile groups.
  • Inspect per-model health, logs, and live metrics in a terminal interface.
  • Generate a LiteLLM gateway configuration to route named models through one endpoint.
Before you choose

Requirements & limits

The documented installation requires Python 3.10 or later and supports Linux and macOS. The repository specifies the Apache-2.0 license.

  • Vllmops manages vLLM servers; vLLM remains the serving engine.
  • Each model runs as its own server and port; the optional LiteLLM proxy supplies one gateway endpoint.
  • The proxy requires a separate LiteLLM installation, kept outside the project environment.
  • Metrics are scraped from /metrics into an in-memory ring buffer; the documented workflow does not require Prometheus or Grafana.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Initialize a vLLM workspace and start a model

Create a project workspace, install its vLLM dependency, define a model, and start its server.

Before you start

  • Python 3.10 or later
  • Linux or macOS
  • The vllmops CLI and uv

Steps

  1. Create and enter a project directory, then run vllmops init.

  2. Run the command below to create the project virtual environment with vLLM installed.

    Documented command
    uv sync

  3. Run vllmops create-model and provide a name, Hugging Face model, GPUs, and port.

  4. Run vllmops start ; by default, it waits on the model's /health endpoint.

Expected outcome

A configured model server is started from the project workspace.

Guide limitations

  • The documented start step waits on /health by default.
  • The guide documents model-server setup and startup, not a client inference request.
Documentation 1 ↗
Source-based practical guide

Group model servers into a profile for lifecycle operations

Define a model group and operate on its members together.

Before you start

  • An initialized vllmops project
  • Model YAML files in the project
  • Access to .vllmops/config.yaml

Steps

  1. Declare a profile and its model names under profiles in .vllmops/config.yaml.

  2. Use vllmops start --profile to start the group.

  3. Use vllmops stop --profile or vllmops restart --profile for those lifecycle actions.

  4. Use vllmops profile list or vllmops profile show to inspect profile membership and state.

Expected outcome

Profile members can be managed together, with parallel processing and per-model failure reporting.

Guide limitations

  • Already-running profile members are skipped during start.
  • Models not assigned to a profile are placed in the synthetic general group.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
VllmopsDefine model server settings such as model identifier, GPU selection, port, environment, and vLLM arguments.CLI, Server, Self-hostedThe documented installation requires Python 3.10 or later and supports Linux and macOS. The repository specifies the Apache-2.0 license.
InferoxyServe requests to Dockerized computer-vision models through supported bridges.Docker, Linux, Self-hosted, Server
FriendliAISend inference requests to a hosted open-weight model through a Model API.Web, CloudThe official documentation lists Model APIs and Dedicated Endpoints. It also describes Model APIs as serverless and pay-per-token; the evidence does not specify prices.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
24 October 2026
Original website title
Freim32/vllmops · Official homepage title · retained in its source language
How we review sourcesSuggest a correction