OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Image & Video AI

Llavavision

Llavavision is a self-hosted web app that sends visual input to a llama.cpp server running BakLLaVA-1, displays the model's description, and narrates the text with the browser Web Speech API.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Llavavision — OpenViST project profile in Image & Video AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you can run the model server and web app yourself and want spoken descriptions of camera input, including access from a mobile browser over HTTPS.

Who it is for

  • People setting up a local camera-description and speech-narration workflow.
  • Developers exploring a mobile-accessible interface for a locally run vision-language model.

Documented uses

  • Generate a model-written description of what a camera sees.
  • Hear that description read aloud through browser speech synthesis.
  • Access the self-hosted interface from a mobile browser, including for mobile video functionality.
Before you choose

Requirements & limits

The README documents self-hosting and mobile-browser access; it does not describe a hosted service.

  • The model server and app are separate components that must be set up.
  • The README estimates about 5 GB RAM or VRAM for the q4_k model version.
  • CUDA is optional; CPU and GPU server launch examples are provided.
  • HTTPS is needed for mobile video functionality.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up LLaVaVision for mobile access

Build the llama.cpp BakLLaVA server and launch LLaVaVision over HTTPS for access from a mobile device.

Before you start

  • A machine with about 5 GB of RAM/VRAM for the q4_k model version.
  • Python 3 and the tools used by the documented build steps, including git, CMake, and OpenSSL.
  • CUDA toolkit is optional; the README says to remove the CUDA build flag if CUDA is unavailable.

Steps

  1. Clone and build llama.cpp: the command below, then run mkdir build, enter the build directory, run cmake .. -DLLAMA_CUBLAS=ON (omit the flag if CUDA is unavailable), and build with cmake --build . --config Release.

    Documented command
    git clone https://github.com/ggerganov/llama.cpp

  2. Download mmproj-model-f16.gguf and ggml-model-q4_k.gguf from the ggml_bakllava-1 model repository, then start the llama.cpp server with ./bin/server -m ggml-model-q4_k.gguf --mmproj mmproj-model-f16.gguf -ngl 35 -ts 100,0 for the documented single-GPU option, or use the README's CPU variant.

  3. Clone LLaVaVision, create and activate its virtual environment, and install its requirements: the command below, cd llavavision, the command below, . ./venv/bin/activate, and the command below.

    Documented command
    git clone https://github.com/lxe/llavavision
    Documented command
    python3 -m venv venv
    Documented command
    pip install -r requirements.txt

  4. Create the certificates with the documented OpenSSL command, then start Flask with flask run --host=0.0.0.0 --key key.pem --cert cert.pem --debug.

  5. Open https://your-machine-ip:5000 on the mobile device. The README also documents an optional localtunnel command for tunneling.

Expected outcome

The README's setup path starts the local model server and LLaVaVision web app, with HTTPS access from a mobile device.

Guide limitations

  • HTTPS is required for mobile video functionality.
  • The approximate 5 GB RAM/VRAM requirement is stated for the q4_k version; other quantizations may differ.
  • The certificate command creates dummy certificates.
  • The README provides server options separately and distinguishes GPU and CPU launch variants.
Documentation 1 ↗Documentation 2 ↗
Evidence and transparency

Sources & review

Official repository ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
lxe/llavavision · Official homepage title · retained in its source language
How we review sourcesSuggest a correction