OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Bento

BentoML is open-source infrastructure for deploying, optimizing, scaling and managing AI/ML model inference and custom inference pipelines in production, including self-hosted deployments.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Bento — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use BentoML when you need infrastructure to put AI/ML inference or a custom pipeline into production and need to consider deployment, performance or scaling.

Who it is for

  • Developers and teams building production AI/ML model-serving services.
  • Teams deploying custom inference pipelines or compound AI systems.

Documented uses

  • Deploy an inference API for a language model, embedding model, vision model or other custom model.
  • Serve compound AI systems such as agent, retrieval-augmented generation or multi-LLM routing workflows.
  • Choose an example project that matches a model workload or inference runtime.
  • Plan a self-hosted model-serving deployment.
Before you choose

Requirements & limits

  • BentoML is infrastructure for serving models and pipelines, not itself an AI model or end-user application.
  • The official example library covers varied workloads and names runtimes including vLLM, TensorRT-LLM, LMDeploy, MLC-LLM, SGLang, Hugging Face TGI and Triton Inference Server.
  • Deployment options include self-hosting.
  • The project identifies an open-source BentoML offering.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Explore BentoML inference examples

Find a documented BentoML example for a model or inference scenario.

Before you start

  • Access to the BentoML examples overview.

Steps

  1. Open the BentoML examples overview.

  2. Browse its categories, such as LLMs, audio, computer vision, embeddings, or custom models.

  3. Choose a listed example that matches your intended inference scenario.

Expected outcome

A relevant BentoML example project is identified for further study.

Guide limitations

  • The overview lists tutorials and example projects but does not provide the implementation steps for any individual example.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
BentoDeploy an inference API for a language model, embedding model, vision model or other custom model.Cloud, Self-hosted
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

bentoml.com ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
Bento: Run Inference at Scale · Official homepage title · retained in its source language
How we review sourcesSuggest a correction