Rag Chatbot — umbertogriffo: Answer questions from Markdown

Understand the project

Practical overview

When it fits

Use it when you can run the project’s local model-serving and indexing stack and want questions answered from a Markdown collection rather than from an ungrounded conversation alone.

Who it is for

Developers setting up a self-hosted chatbot for a curated Markdown knowledge base.
Teams or individuals who want conversational access to their own Markdown documentation.

Documented uses

Index Markdown documents by splitting them into sections, embedding those sections, and storing them in Chroma.
Ask questions and generate answers grounded in passages retrieved from the indexed documents.
Update indexed chunks when source documents change and target removal when documents are deleted.

Before you choose

Requirements & limits

The documented stack includes llama.cpp, Chroma, Sentence Transformers and SQLite.
Listed prerequisites include Python 3.12+, Poetry 2.3.0+, Docker 24.0.6+, Docker Compose 5.0.2+, and CUDA-capable GPU hardware or Apple Silicon M-series.
The UI has separate Node 22.12+ and Yarn 1.22 prerequisites.
Switching embedding models requires rebuilding the index because their vector spaces are incompatible.
The project warns that generated answers may contain hallucinations or false information.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Bootstrap the RAG chatbot environment

Create the project environment and install dependencies with the documented GPU acceleration option.

Before you start

Python 3.12 or later
Poetry 2.3.0 or later
Docker 24.0.6 or later and Docker Compose 5.0.2 or later
A GPU supporting CUDA 12.4 or later, or Apple Silicon M-series
For CUDA support, NVIDIA Container Toolkit (optional)

Steps

Run the command below to check that which pip3 and which python3 point to the intended paths.
Documented command
```
make check
```
Run the command below for NVIDIA CUDA acceleration, or the command below for Metal acceleration on macOS.
Documented command
```
make setup_cuda
```
Documented command
```
make setup_metal
```
Run Setup as the initialization command, or after running Clean.

Expected outcome

The make target creates an environment and installs dependencies with the selected acceleration option; both setup options start the llama.cpp server.

Guide limitations

The Metal setup is for macOS only.
The documented test environments are Ubuntu 22.04.2 with an NVIDIA GeForce RTX 3060 and macOS Sonoma 14.3.1 on an M1 MacBook Pro; other operating systems or hardware may have model-loading issues.
The README warns that the language model may generate hallucinations or false information.

Documentation 1 ↗

COMPARE OPTIONS

Compare options for Vector search

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

Project	Documented purpose	Platform	Access and requirements
Rag Chatbot — umbertogriffo	Index Markdown documents by splitting them into sections, embedding those sections, and storing them in Chroma.
Chatbot with PDF for Semantic Search	Prepare document content as embeddings and save it to a supported vector database.	Web	The repository is identified as open source and publicly visible on GitHub.
AI LLM Playground	Ask questions about PDF documents through an assistant interface.	Web

Evidence and transparency

Sources & review

Official repository ↗

Source record details

Source availability: Active
Sources checked: 14 October 2026
Original website title: umbertogriffo/rag-chatbot · Official homepage title · retained in its source language

How we review sources Suggest a correction

Rag Chatbot — umbertogriffo