OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Mllm

Mllm is a C++ inference framework for running large language models, including vision-language models, on mobile and edge devices. It exposes model and tokenizer APIs for image-and-text input and streaming generation, and includes model-authoring and checkpoint-conversion workflows.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Mllm — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Mllm when you are developing an application or prototype that needs a C++ multimodal inference layer on mobile or edge hardware, and can match the model and backend requirements to your target device.

Who it is for

  • Developers building mobile applications with local multimodal model inference.
  • Teams prototyping edge-device assistants or other applications that need an embedded inference engine.
  • C++ developers adapting or converting supported model checkpoints for Mllm’s runtime.

Documented uses

  • Integrate a vision-language model into an application and pass image-and-text inputs for generated responses.
  • Build the framework for documented Android, x86 PC or Apple Silicon targets.
  • Author model components or convert PyTorch and SafeTensors checkpoints into Mllm’s runtime format.
  • Explore hardware-acceleration and optimization paths such as quantization, pruning and speculative execution.
Before you choose

Requirements & limits

The repository is public and identifies the project as MIT-licensed.

  • The project is an inference framework; application interfaces and product behavior must be built around it.
  • The documented API example uses Qwen2-VL with configuration, tokenizer, pretrained weights and image-text inputs.
  • Build paths are platform-specific; Android and x86 use project task YAML files, while Apple Silicon has a separate task and requirements file.
  • The README lists Arm CPU, OpenCL GPU, QNN NPU and Ascend NPU support, but availability depends on the chosen backend and model.
  • QNN backend compilation requires a separately installed SDK with its own licensing terms.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Build the Android version

Configure and compile Mllm for Arm Android using the documented build task.

Before you start

  • Python and the project’s requirements installed with pip.
  • An Android build environment; the repository notes that its Arm Docker image includes NDK downloads and related terms apply.

Steps

  1. Clone the Mllm repository and change into its directory.

  2. Install the project dependencies with pip install -r requirements.txt.

  3. Run python task.py tasks/build_android.yaml.

Expected outcome

The documented Android build task is invoked for the Arm Android target.

Guide limitations

  • This covers the build step only; it does not document installing or running an app on a device.
  • For the Android QNN backend, install and configure the QNN SDK separately before using the QNN build task.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
MllmIntegrate a vision-language model into an application and pass image-and-text inputs for generated responses.Android, iOS, Mobile, Self-hostedThe repository is public and identifies the project as MIT-licensed.
MorphingdbImport a libtorch model and create predictions from values in a PostgreSQL table.Docker, Linux, Self-hosted, ServerThe README documents a Docker image and a build-from-repository option, followed by connecting to PostgreSQL and enabling the pgdl extension.
Gateway — pyadavSend chat-completion requests to a provider through a common server endpoint.ServerThe repository is public and identifies the project as Apache-2.0 licensed. README installation instructions include binaries, platform package managers and Docker.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
11 October 2026
Original website title
UbiquitousLearning/mllm · Official homepage title · retained in its source language
How we review sourcesSuggest a correction