OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
RAG & Knowledge Systems

Epsteinfiles Rag

Epsteinfiles Rag is a Python retrieval-augmented generation application built around the Epstein Files 20K dataset. It cleans and reconstructs fragmented records, chunks them for context, indexes vector embeddings, retrieves diverse relevant passages with maximal marginal relevance, and uses retrieved context to generate sourced answers. The repository includes a FastAPI backend and Streamlit interface.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Epsteinfiles Rag — OpenViST project profile in RAG & Knowledge Systems.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when a question concerns material in the Epstein Files 20K collection and you want a RAG workflow that retrieves passages before generating an answer.

Who it is for

  • Researchers exploring the Epstein Files 20K collection through natural-language questions.
  • Developers examining a documented RAG pipeline with dataset ingestion, vector retrieval and a question-answering interface.

Documented uses

  • Clean and reconstruct fragmented records from the referenced dataset.
  • Prepare documents as context-preserving chunks and searchable vector embeddings.
  • Retrieve varied relevant passages in response to a question.
  • Generate answers grounded in retrieved documents and present them through a web interface or API.
Before you choose

Requirements & limits

The public GitHub repository identifies an MIT license. Running the documented workflow requires a Groq API key and local Python setup.

  • The application is built for one named dataset rather than an unspecified general document collection.
  • The documented retrieval method is MMR, intended to balance relevance with diversity of retrieved passages.
  • The repository links both the original Hugging Face dataset and a separately precomputed ChromaDB embeddings dataset.
  • The README specifies Python 3.11+, 16 GB RAM recommended (8 GB minimum), and a Groq API key.
  • The listed query components include LLaMA 3.3 via Groq Cloud, Sentence Transformers embeddings, Chroma, LangChain, FastAPI and Streamlit.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up and run the document question-answering app

Prepare the dataset and start its documented API and Streamlit interface for querying.

Before you start

  • Python 3.11 or newer
  • 16 GB RAM recommended; 8 GB minimum
  • A Groq API key
  • Repository dependencies installed from requirements.txt

Steps

  1. Clone the repository and create and activate a Python virtual environment.

  2. Install dependencies with pip install -r requirements.txt and add the Groq key to a .env file as GROQ_API_KEY.

  3. Run the download, cleaning, chunking and embedding scripts in order: python ingest/download_dataset.py, python ingest/clean_dataset.py, python ingest/chunk_dataset.py and python ingest/embed_chunks.py.

  4. Start the API with uvicorn api.main:app --reload and the interface with streamlit run app.py; the README lists local addresses at 127.0.0.1:8000 and localhost:8501.

Expected outcome

A prepared document index with a local API and Streamlit interface for asking questions against retrieved context.

Guide limitations

  • This workflow requires completing dataset ingestion and embedding before querying; the README estimates 5–15 minutes for downloading, 3–8 for cleaning, 5–12 for chunking and 20–45 for embedding.
  • The project’s README describes answers as grounded in retrieved source context; that description is not a guarantee that every answer is complete or correct.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Retrieval-augmented applications

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Epsteinfiles RagClean and reconstruct fragmented records from the referenced dataset.The public GitHub repository identifies an MIT license. Running the documented workflow requires a Groq API key and local Python setup.
Rag Powered AI Assistant Transforming Data Retrieval And Analysis Across The Web And PdfsRetrieve relevant passages from web and PDF content to answer a question.Web
LangChain in Supabase Edge FunctionsInitialize and start a Supabase project locally.Cloud, Server
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
14 October 2026
Original website title
AnkitNayak-dev/EpsteinFiles-RAG · Official homepage title · retained in its source language
How we review sourcesSuggest a correction