OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Agent Memory & RAG

Local LLM With Rag

Local LLM With Rag is an open-source Python application for asking questions about selected local documents. It uses an Ollama model with Pydantic AI to decide when to search, stores nomic-embed-text embeddings in LanceDB for vector retrieval, and presents a Streamlit chat interface.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Local LLM With Rag — OpenViST project profile in Agent Memory & RAG.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want to experiment with a locally run document-chat workflow and can provide Ollama, UV and a tool-calling-capable model.

Who it is for

  • People who want to query their own document folders through a locally run language model.
  • Developers exploring agent-directed retrieval-augmented generation and local model tool calling.
  • Researchers seeking a starting point for adapting document search or evaluating local models.

Documented uses

  • Ask questions about a folder of personal or work documents.
  • Search and retrieve relevant passages from supported files using an agent that chooses when to call search.
  • Explore or adapt a local document-retrieval application and its tool-calling approach.
Before you choose

Requirements & limits

The repository is public and identifies an MIT license. The application is run locally from the repository; its documented setup requires Ollama 0.13.5 or higher and UV.

  • A compatible model must support tool calling, which is central to the agent’s document-search behavior.
  • Supported document formats include PDF, DOCX, PPTX, XLSX, Markdown, HTML, CSV and JSON.
  • First-run model downloads can take time depending on internet connection.
  • The project reports model evaluations and recommends qwen3.5:9b, with qwen3:8b and qwen3:14b also listed; local results may vary.
  • A four-search cap per question and disabled thinking mode are described as safeguards against runaway behavior.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the local document-chat application

Install the repository’s Python dependencies and start its Streamlit interface.

Before you start

  • Ollama version 0.13.5 or higher
  • UV installed
  • A selected Ollama model that supports tool calling

Steps

  1. Clone the repository to your local machine.

  2. Install UV using the Astral installation instructions.

  3. Create a virtual environment and install the required Python packages with the command below.

    Documented command
    uv sync

  4. Start the interface with the command below.

    Documented command
    uv run streamlit run interfaces/streamlit_app.py

  5. In the interface, select a model, specify a document folder and chat with its documents.

Expected outcome

A local web interface for selecting a model and asking questions about documents in a chosen folder.

Guide limitations

  • The first run downloads the required language and embedding models; download time depends on the internet connection.
  • The selected model must support tool calling; unsupported models may fail to search or respond correctly.
  • The documented formats are PDF, DOCX, PPTX, XLSX, Markdown, HTML, CSV and JSON.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
amscotti/local-LLM-with-RAG · Official homepage title · retained in its source language
How we review sourcesSuggest a correction