Practical overview
When it fits
Use Bodhi App when you want a local model with both a ready-made chat interface and an OpenAI-compatible API for other software.
Who it is for
- People who want to download and chat with open-source models on their own machine.
- Developers who need local chat-completion or model endpoints compatible with the OpenAI API.
Documented uses
- Discover and download GGUF models from Hugging Face that suit local hardware.
- Chat with locally hosted models and adjust application settings.
- Generate API tokens and connect external integrations to local model endpoints.
- Explore and test API endpoints using the built-in Swagger UI.
Requirements & limits
The README documents a Mac desktop release and Docker images for CPU, CUDA, ROCm and Vulkan configurations. The project identifies itself as open source.
- The README identifies llama.cpp as the inference engine and Hugging Face as the model ecosystem.
- Docker hardware prerequisites vary by image: CUDA requires an NVIDIA GPU with CUDA 12.4+ support and compatible drivers; ROCm requires AMD hardware with ROCm 6.4+ support and compatible drivers; Vulkan requires Vulkan su
- Docker uses separate mounts for application data and the Hugging Face cache.
- The README includes a TypeScript client package for API integration.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Start Bodhi App in Docker with the CPU image
Launch the documented CPU container and open Bodhi App locally.
Before you start
- Docker
- An x86_64 (AMD64) or ARM64 processor
- Local directories for the application data and Hugging Face cache mounts
Steps
Run the documented CPU image with port 8080 mapped to 8080 and mount local directories at /data/bodhi_home and /data/hf_home.
Open http://localhost:8080 in a browser.
Expected outcome
Bodhi App is available at localhost:8080, with application data and the Hugging Face cache mapped to the specified directories.
Guide limitations
- The CPU image is documented for AMD64 and ARM64 processors.
- The README gives separate image variants and hardware prerequisites for CUDA, ROCm and Vulkan acceleration.
Compare options for Model inference and serving
These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.
| Project | Documented purpose | Platform | Access and requirements |
|---|---|---|---|
| Bodhi App | Discover and download GGUF models from Hugging Face that suit local hardware. | Linux, macOS, Self-hosted, Windows | The README documents a Mac desktop release and Docker images for CPU, CUDA, ROCm and Vulkan configurations. The project identifies itself as open source. |
| Offlinellm | Import a GGUF model and use it for local multi-turn chat. | Android | The documented release is an APK available from GitHub Releases. It requires Android 13 or later and arm64-v8a; the Vanilla APK requires users to supply and import their own GGUF model. |
| Box | Import a local .litertlm or GGUF model and use it in AI Chat. | Android | The project README provides Android APK downloads through GitHub Releases and installation through Obtainium. It lists Android 14+ as a requirement. |
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 18 October 2026
- Original website title
- BodhiSearch/BodhiApp · Official homepage title · retained in its source language