OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Mlc LLM

MLC LLM is an open-source machine-learning compiler and deployment engine that compiles and runs large language models through MLCEngine across supported hardware and software environments.

Independent OpenViST project profile, based on linked sources.

Mlc LLM — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use MLC LLM when a project needs a compiler-backed deployment layer that can expose the same inference engine through server, programming-language and mobile interfaces across multiple accelerator and platform paths.

Who it is for

  • Developers deploying language models behind application backends
  • Teams building self-hosted LLM serving layers
  • Engineers integrating compiled models into custom machine-learning runtimes
  • Application developers targeting browser, iOS or Android inference

Documented uses

  • Compile large language models for supported execution targets
  • Serve model inference through a REST server
  • Connect LLM inference to Python or JavaScript applications
  • Build local, server-based, browser or mobile LLM applications
Before you choose

Requirements & limits

The repository is public, active and licensed under Apache-2.0. The supplied material does not specify pricing or account requirements.

  • The project exposes an OpenAI-compatible API through REST server, Python, JavaScript, iOS and Android interfaces.
  • Supported paths differ by target: examples include CUDA, ROCm, Vulkan, Metal, WebGPU, WebAssembly and selected Android OpenCL devices.
  • The deployment model is compiler-based, so model preparation and target-specific configuration are part of the workflow.
  • The repository identifies Python as its primary language and TVM as a related ecosystem.
  • A TVM workflow connection is Likely compatible, but no concrete integration example or compatibility test is supplied.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Mlc LLMCompile large language models for supported execution targetsCloud, Self-hosted, ServerThe repository is public, active and licensed under Apache-2.0. The supplied material does not specify pricing or account requirements.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
2 October 2026
Original website title
mlc-ai/mlc-llm · Official homepage title · retained in its source language
How we review sourcesSuggest a correction