OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Multi Model Server

Multi Model Server is an open-source server tool that starts HTTP endpoints for inference requests to deep-learning models. It can be started through its command-line interface or pre-configured Docker images.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Multi Model Server — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when evaluating a self-managed server tool for exposing model inference through HTTP endpoints and when its documented runtime prerequisites fit your environment.

Who it is for

  • Engineers building server-side inference services for deep-learning models.
  • Teams evaluating a documented model-serving workflow involving MXNet or ONNX.

Documented uses

  • Start a service that accepts model inference requests over HTTP.
  • Register a model archive when starting the server.
  • Package and serve models within a server-side inference workflow.
Before you choose

Requirements & limits

The project is public and open source under the Apache-2.0 license. Its repository is marked archived.

  • The documented runtime requires Python, pip and Java 8.
  • MXNet is not installed by default; the README recommends separate CPU and GPU packages.
  • Startup model registration scales backend workers according to available vCPUs or GPUs and may increase startup time.
  • The repository is marked archived.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install MMS and start the documented SqueezeNet example

Set up Multi Model Server and launch the example model as an inference service.

Before you start

  • Ubuntu, CentOS or macOS; Windows support is experimental.
  • Python, pip and Java 8.
  • An MXNet package in the Python environment: the README recommends mxnet-mkl for CPU inference or mxnet-cu92mkl for GPU inference.

Steps

  1. Create and activate a Python virtual environment.

  2. Install the appropriate MXNet package for the intended CPU or GPU setup.

  3. Install MMS with pip install multi-model-server.

  4. Start the example with multi-model-server --start --models squeezenet=https://s3.amazonaws.com/model-server/model_archive_1.0/squeezenet_v1.1.mar.

Expected outcome

MMS starts on the host with the example model registered and listening for inference requests.

Guide limitations

  • The README's quick-start instructions focus on Linux and macOS; Windows support is experimental.
  • Registering models at startup scales workers to available vCPUs or GPUs and may take considerable time on hosts with many resources.
  • The repository is marked archived.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Multi Model ServerStart a service that accepts model inference requests over HTTP.ServerThe project is public and open source under the Apache-2.0 license. Its repository is marked archived.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
awslabs/multi-model-server · Official homepage title · retained in its source language
How we review sourcesSuggest a correction