OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Mesh LLM

Mesh LLM is self-hosted infrastructure that pools GPU and memory resources across machines to run language-model inference and serve it through an OpenAI-compatible API. Skippy handles model management, native inference, split execution and API serving; Mesh adds peer discovery, transport, routing, plugins and a console.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Mesh LLM — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Mesh LLM when you want to operate inference infrastructure across one or more machines, serve models through an OpenAI-compatible API, or split a model across nodes when it exceeds a single machine’s capacity.

Who it is for

  • Operators pooling compute across machines for language-model inference.
  • Developers needing an OpenAI-compatible endpoint for locally hosted or mesh-routed models.
  • Teams serving models too large for one machine through stage splits.

Documented uses

  • Expose a locally hosted model through an OpenAI-compatible API.
  • Route requests to a peer that can serve the requested model.
  • Pool compute across nodes and distribute large-model stages.
  • Join a public mesh, operate a private mesh, or serve a model without mesh networking.
Before you choose

Requirements & limits

The README documents executable installation routes for Linux packages, Apple Silicon Homebrew and Windows PowerShell, followed by the `mesh-llm setup` command. It also documents public-mesh, private-mesh and local-only workflows.

  • A model that fits on one node can run locally; Mesh can route requests to peers when appropriate.
  • Stage splits support models too large for one machine, with package-backed model fragments assigned to stages.
  • Public mesh discovery uses Nostr; private meshes are invite-token based.
  • The serving API is OpenAI-compatible at the documented local /v1 endpoint.
  • The repository identifies the license as Apache-2.0 and the primary language as Rust.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Mesh LLM and complete initial setup

Install the Mesh LLM executable and run its documented setup command.

Before you start

  • A platform with an installation method listed in the README.
  • PowerShell for the documented Windows installer, or a shell for the Unix-style installer.

Steps

  1. Install the executable using the appropriate documented installation method.

  2. Run mesh-llm setup; on Windows PowerShell, run mesh-llm.exe setup.

Expected outcome

The executable is installed and the documented initial setup command has been run.

Guide limitations

  • This guide covers installation and setup only; it does not cover starting a mesh or serving a model.
  • The README refers to platform installation guides for the supported package matrix and commands.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Mesh LLMExpose a locally hosted model through an OpenAI-compatible API.Linux, Self-hosted, ServerThe README documents executable installation routes for Linux packages, Apple Silicon Homebrew and Windows PowerShell, followed by the `mesh-llm setup` command. It also documents public-mesh, private-mesh and local-only workflows.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
10 October 2026
Original website title
Mesh-LLM/mesh-llm · Official homepage title · retained in its source language
How we review sourcesSuggest a correction