OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Local & Self-hosted AI

Offlinellm

Offlinellm is an Android chat application that runs GGUF language models on-device with llama.cpp, supporting CPU inference and optional Vulkan GPU offload.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Offlinellm — OpenViST project profile in Local & Self-hosted AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Offlinellm when you want to run a GGUF model on a supported Android device and chat without network connectivity.

Who it is for

  • Android users who want to chat with a language model without network connectivity.
  • People who already have, or are prepared to obtain, a GGUF model to import.
  • Users who want to adjust local inference settings or try Vulkan GPU offload.

Documented uses

  • Import a GGUF model and use it for local multi-turn chat.
  • Keep model inference on an Android device when offline.
  • Adjust sampling and performance settings for a local chat session.
  • Save, search, rename, and back up conversations.
Before you choose

Requirements & limits

The documented release is an APK available from GitHub Releases. It requires Android 13 or later and arm64-v8a; the Vanilla APK requires users to supply and import their own GGUF model.

  • The documented platform requirement is Android 13+ on arm64-v8a.
  • The Vanilla APK does not include a model.
  • Vulkan acceleration is optional; CPU inference is also supported.
  • The project describes the app as having no INTERNET permission, with inference performed on-device.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Offlinellm and import a GGUF model

Set up the Android app with a user-supplied GGUF model for local chat.

Before you start

  • Android 13 or later
  • An arm64-v8a Android device
  • The Offlinellm APK from GitHub Releases
  • A GGUF model file

Steps

  1. Download the APK from the project’s GitHub Releases.

  2. Allow the file manager to install unknown apps in Android Settings, then open the APK and install it.

  3. Complete onboarding, then open Settings → Model → Import GGUF Model and select a GGUF file.

Expected outcome

The app is installed with a GGUF model imported for on-device inference.

Guide limitations

  • The Vanilla release does not bundle a model; users bring their own GGUF file.
  • The documented minimum requirements are Android 13+ and arm64-v8a.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
OfflinellmImport a GGUF model and use it for local multi-turn chat.AndroidThe documented release is an APK available from GitHub Releases. It requires Android 13 or later and arm64-v8a; the Vanilla APK requires users to supply and import their own GGUF model.
Bodhi AppDiscover and download GGUF models from Hugging Face that suit local hardware.Linux, macOS, Self-hosted, WindowsThe README documents a Mac desktop release and Docker images for CPU, CUDA, ROCm and Vulkan configurations. The project identifies itself as open source.
BoxImport a local .litertlm or GGUF model and use it in AI Chat.AndroidThe project README provides Android APK downloads through GitHub Releases and installation through Obtainium. It lists Android 14+ as a requirement.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
jegly/OfflineLLM · Official homepage title · retained in its source language
How we review sourcesSuggest a correction