OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Local & Self-hosted AI

Bonsai Demo

Bonsai Demo is an open-source application and setup repository for running Bonsai language models locally. Its default model is Bonsai 2 27B, with documented llama.cpp and Apple Silicon MLX paths, terminal prompting and a local chat server. The project also documents image input, reasoning controls and tool-calling support.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Bonsai Demo — OpenViST project profile in Local & Self-hosted AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Bonsai Demo when you want to configure and run a Bonsai model on your own hardware, and can match the model format to a supported backend.

Who it is for

  • People who want to run Bonsai models on their own Mac, Linux or Windows system.
  • Developers exploring local model prompting, image questions or tool-calling workflows.
  • Users comparing inference backends and model formats before configuring a local setup.

Documented uses

  • Download and prepare Bonsai 2 27B and the project-specific inference binaries.
  • Send a prompt to the model from a terminal or through a local browser chat server.
  • Ask a local model about photos, screenshots or PDFs.
  • Explore tool calls and MCP server integrations in the documented demo interfaces.
Before you choose

Requirements & limits

The repository is public and identifies the project license as Apache-2.0. The documented model files are downloaded during setup; no account requirement is stated in the README.

  • Bonsai 2 requires PrismML’s llama.cpp fork; stock llama.cpp is not compatible with its model files.
  • The README lists Metal for Mac, CUDA/Vulkan/ROCm for Linux and Windows, and CPU use; Apple Silicon also has a native MLX path.
  • The default setup downloads 7.8 GB of PQ2_0 weights plus a vision projector, and installs optional interface and code-interpreter components that add several more gigabytes.
  • The demo supports GGUF and MLX 2-bit model formats, so the chosen scripts and weights must match.
  • The README documents image questions, OpenAI-style tool calls and MCP servers, with details in separate project guides.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up Bonsai Demo and start local chat

Install the documented default model and start its local chat server.

Before you start

  • A macOS, Linux or Windows machine; the README lists Metal, CUDA, Vulkan, ROCm and CPU options according to platform.
  • Several gigabytes of available storage for model files and optional interface components.
  • Git and an internet connection to clone the repository and download dependencies and model files.

Steps

  1. Clone the Bonsai Demo repository and change into its directory.

  2. Run ./setup.sh on macOS or Linux, or use the documented PowerShell setup commands on Windows.

  3. Start the server with ./scripts/start_llama_server.sh on macOS or Linux, or ./scripts/start_llama_server.ps1 on Windows.

  4. Open http://localhost:8080.

Expected outcome

A local chat server for Bonsai 2 27B, with the documented setup also preparing its vision and tool features.

Guide limitations

  • Bonsai 2 requires the project’s PrismML llama.cpp binaries; stock llama.cpp cannot run its model files.
  • The default setup downloads the model weights and vision projector and also installs Open WebUI and a code-interpreter environment, which add several gigabytes.
  • The guide lists platform and backend options but does not establish performance for a particular machine.
Documentation 1 ↗
Source-based practical guide

Run a single prompt from the terminal

Use the installed GGUF model for one terminal prompt without starting a chat server.

Before you start

  • The Bonsai Demo repository and its setup completed.
  • Bonsai 2 GGUF weights and the project’s llama.cpp binaries.

Steps

  1. From the repository directory, run ./scripts/run_llama.sh -p "What is the capital of France?" on macOS or Linux.

  2. On Windows PowerShell, run .\scripts\run_llama.ps1 -p "What is the capital of France?".

Expected outcome

The script sends the example prompt to the local model and returns a generated response in the terminal.

Guide limitations

  • The documented llama.cpp scripts require GGUF weights.
  • For an MLX-only setup, the README directs users to the MLX scripts or to download GGUF weights.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
10 October 2026
Original website title
PrismML-Eng/Bonsai-demo · Official homepage title · retained in its source language
How we review sourcesSuggest a correction