Practical overview
When it fits
Use it when you want a self-hosted chat workspace that combines local model conversations with document retrieval, image input or transcribed voice input.
Who it is for
- People who want to run a multimodal chat interface on their own machine using Ollama.
- Users who want to ask questions about their own PDFs through retrieval-augmented generation.
- Developers looking for a Python, Streamlit and Docker project to configure or extend.
Documented uses
- Chat with text using a local Ollama model or switch to the OpenAI API.
- Upload PDFs and query their contents using retrieval backed by Chroma and Ollama embeddings.
- Send images to vision-capable models such as LLaVA.
- Record or upload audio and use Whisper transcription as conversation input.
- Keep multi-session chat history, messages and settings in SQLite.
Requirements & limits
The repository is public and licensed under GPL-3.0. The documented application is self-hosted; using the optional OpenAI endpoint requires an API key.
- The application interface is Streamlit; Docker Compose and manual Python installation are documented setup routes.
- PDF retrieval requires an Ollama embedding model such as nomic-embed-text; image chat requires a vision-capable model such as LLaVA.
- Audio input uses Whisper; the README identifies ffmpeg for converting webm audio to wav.
- SQLite stores sessions, messages and settings, while Chroma stores the PDF retrieval index.
- The README describes a test suite for selected contract-level components and says UI, audio and real model calls are tested manually.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up the application with Docker Compose
Start the Streamlit application and prepare Ollama models for the documented chat capabilities.
Before you start
- Docker Compose
- A model save path to set in docker-compose.yml
- Ollama models pulled for the desired tasks
- An OpenAI API key only if using the optional OpenAI endpoint
Steps
Copy .env.example to .env; add an OpenAI key only if you plan to use that endpoint.
Set the model save path in docker-compose.yml.
Run docker compose up and open the app at http://0.0.0.0:8501.
Enter /pull MODEL_NAME in the chat bar to pull models; use an embedding model for PDF retrieval and a vision model for image chat.
Expected outcome
The Streamlit app is running with the selected Ollama models available for chat; PDF retrieval and image chat have their respective required model types.
Guide limitations
- The README says to remove the deploy section from the compose file when running without a GPU.
- The optional OpenAI endpoint requires an API key.
- The repository recommends installing Ollama locally on Windows because running it inside Docker is slow there.
Install and launch the app manually
Prepare the Python application, initialize its SQLite database and launch the Streamlit interface.
Before you start
- Ollama installed
- Python environment; the project notes development on Python 3.10.12
- The repository requirements files
Steps
Install Ollama.
Create a Python virtual environment.
Upgrade pip, install requirements.txt, and install the documented CPU builds of torch, torchvision and torchaudio.
Run python3 src/database_operations.py to initialize SQLite, then run streamlit run src/app.py.
Pull the Ollama models needed for the intended tasks.
Expected outcome
The documented manual setup initializes the chat database and launches the Streamlit application with Ollama available for model use.
Guide limitations
- The OpenAI endpoint requires a key in .env; local-only use can leave it empty.
- PDF retrieval requires an embedding model, and image chat requires a vision-capable model.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 14 October 2026
- Original website title
- Leon-Sander/Local-Multimodal-AI-Chat · Official homepage title · retained in its source language