OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

WebLLM

WebLLM is an open-source TypeScript library that runs supported large language model inference directly in web browsers with WebGPU acceleration and without server-side processing. It provides a modular npm package and CDN option for developers building browser applications or Chrome extensions, and supports model interaction through an OpenAI-compatible interface.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

WebLLM — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use WebLLM when language-model inference needs to run in the browser and the application can work with the project’s supported models and WebGPU-oriented execution model.

Who it is for

  • Web developers adding language-model features to web applications
  • Teams building browser-based chat or question-answering interfaces
  • Extension developers using Chrome extension integrations
  • Developers evaluating client-side execution of supported open-source models

Documented uses

  • Run supported LLM inference in a browser
  • Add streaming chat completions to a web interface
  • Generate structured JSON from a model
  • Connect browser-side inference to an application UI
  • Use supported built-in models or integrate custom models in MLC format
Before you choose

Requirements & limits

The repository is public, marked Apache-2.0, and publishes the @mlc-ai/web-llm npm package. The project also documents CDN delivery. Browser, device and hardware support details are not specified in the available material.

  • Inference runs in the browser rather than through a hosted server.
  • WebGPU is used for hardware acceleration, so the target browser and device configuration require separate validation.
  • The project lists support for Llama, Phi, Gemma, Mistral and Qwen families, while custom models must use MLC format.
  • The interface documents streaming, JSON mode and OpenAI API compatibility; function-calling is marked work in progress.
  • Web workers and service workers can be used to manage model computation and lifecycle.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Add WebLLM to a web application

Set up WebLLM as a package for client-side language-model inference.

Before you start

  • A web application project
  • A package manager such as npm, Yarn or pnpm, or a CDN-based integration
  • A supported model or a custom model in MLC format

Steps

  1. Install the @mlc-ai/web-llm package with a supported package manager, or import it through a CDN.

  2. Import the WebLLM module or the CreateMLCEngine entry point.

  3. Connect the modular interface to the application’s UI components.

  4. Select a supported built-in model or configure a custom MLC-format model.

Expected outcome

The application is set up to use WebLLM for browser-side LLM inference.

Guide limitations

  • The supplied project material does not specify supported browsers, models’ hardware requirements or a particular device configuration.
  • This recipe documents setup and integration stages; it does not establish a successful run or deployment.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
WebLLMRun supported LLM inference in a browserBrowser Extension, WebThe repository is public, marked Apache-2.0, and publishes the @mlc-ai/web-llm npm package. The project also documents CDN delivery. Browser, device and hardware support details are not specified in the available material.
Cloud AI SDKAdapt and run documented model recipes for computer-vision and NLP inference.Cloud, ServerThe GitHub repository is public. Its public visibility does not establish license terms.
TurboquantReplace a model’s existing inference cache with a compressed cache.Cloud, Linux, macOS, Self-hostedThe project describes itself as open source. The README documents installation through pip or from a source checkout.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
2 October 2026
Original website title
mlc-ai/web-llm · Official homepage title · retained in its source language
How we review sourcesSuggest a correction