OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Local & Self-hosted AI

Inferrlm

InferrLM is an open-source mobile application for running language models locally on iOS and Android, connecting to supported cloud models, and serving local models to other devices on a local network.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Inferrlm — OpenViST project profile in Local & Self-hosted AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use InferrLM when you want a mobile interface for local inference, document-grounded chats, or a local-network endpoint for models. Choose its cloud connections when you want to use supported remote providers and have the required API keys and InferrLM account.

Who it is for

  • Mobile users who want to run GGUF models locally through llama.cpp or MLX models on Apple Silicon devices.
  • People who want to query attached documents with local models using OCR, indexing and RAG.
  • Developers seeking a phone-hosted REST API for local models on their network.

Documented uses

  • Download models from Hugging Face or import them from local storage or a URL, then use them in chat.
  • Attach documents so local OCR and retrieval can provide context to a local-model conversation.
  • Expose models through a local HTTP server and connect other devices or applications through its REST API.
  • Use supported OpenAI, Gemini or Anthropic cloud models with personal API keys and a registered InferrLM account.
Before you choose

Requirements & limits

The project is licensed under AGPL-3.0. Remote model connections require the user’s own API keys and a registered InferrLM account.

  • Local inference uses llama.cpp with GGUF models on Android and iOS; MLX inference is supported on Apple Silicon devices.
  • Cloud connections to OpenAI, Gemini and Anthropic are separate from local inference and require API keys plus an InferrLM account.
  • The built-in server exposes REST APIs to devices on the local network; API documentation is available at the server homepage and in the repository.
  • Local-model document attachments use OCR and indexing for retrieval, while remote models support native file uploads.
  • Developing the app requires Node.js 16 or later but below 23, npm or yarn, Expo CLI, and Android Studio or Xcode.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Start InferrLM’s local server

Expose the app’s models through its local-network server and find the server URL.

Before you start

  • InferrLM installed and open on a mobile device
  • A model available in the app

Steps

  1. Open the Server tab in InferrLM.

  2. Turn on the server switch.

  3. Read the displayed server URL.

Expected outcome

The app displays a URL for accessing its server from devices on the local network.

Guide limitations

  • The server is described for devices on the same local network.
Documentation 1 ↗
Source-based practical guide

Prepare an InferrLM development environment

Install the project dependencies and launch the app on an Android or iOS device or emulator.

Before you start

  • Node.js version 16 or later and below 23
  • npm or yarn
  • Expo CLI
  • Android Studio for Android development or Xcode for iOS development

Steps

  1. Clone the InferrLM repository and change into its directory.

  2. Install dependencies with yarn install.

  3. Configure API keys and backend settings using the variables listed in app.config.js.

  4. Run npx expo run:android or npx expo run:ios for the relevant target.

Expected outcome

The documented setup proceeds to launching the app on an Android or iOS device or emulator.

Guide limitations

  • The repository instructions do not establish that this setup has been independently tested.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
InferrlmDownload models from Hugging Face or import them from local storage or a URL, then use them in chat.Android, iOS, MobileThe project is licensed under AGPL-3.0. Remote model connections require the user’s own API keys and a registered InferrLM account.
Bodhi AppDiscover and download GGUF models from Hugging Face that suit local hardware.Linux, macOS, Self-hosted, WindowsThe README documents a Mac desktop release and Docker images for CPU, CUDA, ROCm and Vulkan configurations. The project identifies itself as open source.
OfflinellmImport a GGUF model and use it for local multi-turn chat.AndroidThe documented release is an APK available from GitHub Releases. It requires Android 13 or later and arm64-v8a; the Vanilla APK requires users to supply and import their own GGUF model.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
13 October 2026
Original website title
loomax-labs/InferrLM · Official homepage title · retained in its source language
How we review sourcesSuggest a correction