OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
RAG & Knowledge Systems

Bm25S

Bm25S is an open-source Python library that tokenizes text collections, builds sparse BM25 indexes, and ranks documents for text queries. It can serve as a lexical retrieval stage in search or RAG systems, and includes an MCP server for exposing an index to LLMs and agents.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Bm25S — OpenViST project profile in RAG & Knowledge Systems.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Bm25S when a Python application needs lexical BM25 ranking over its own document collection, or when an agent needs retrieval access to an existing BM25 index through MCP.

Who it is for

  • Python developers adding keyword-based document retrieval to an application.
  • RAG builders who need a lexical ranking stage for a document collection.
  • Developers who want to expose BM25 retrieval to LLMs or agents through MCP.

Documented uses

  • Index document text and return the top-ranked documents for a query.
  • Keep corpus values such as document dictionaries associated with retrieved IDs.
  • Save and reload a BM25 index for reuse.
  • Expose retrieval and index information as MCP tools.
Before you choose

Requirements & limits

The project is publicly available on GitHub and lists an MIT license. The README documents pip installation; the MCP server requires the `mcp` extra.

  • The library ranks by BM25 lexical matching, so it is suited to term-based retrieval rather than serving as a complete search or RAG application.
  • NumPy is the core documented dependency; Numba, stemming, JSON loading, and progress-bar support are available through optional extras.
  • The MCP integration is a separate setup path that requires the mcp extra and an index directory.
  • The README documents string and dictionary corpus entries and lets developers select text fields for indexing.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Build and query a local BM25 index

Tokenize a document collection, retrieve its top-ranked matches for a query, and save the index.

Before you start

  • Python environment with NumPy and bm25s installed; the README gives pip install bm25s as the installation command.
  • A collection of document strings, or dictionaries with the text fields selected for indexing.

Steps

  1. Pass the corpus text to bm25s.tokenize().

  2. Create a bm25s.BM25() retriever and call index() with the tokenized corpus.

  3. Tokenize a query and call retrieve() with a top-k value to get document IDs and scores; associate original corpus values by supplying the corpus.

  4. Call save() to store the index, optionally with the corpus.

Expected outcome

A BM25 index that returns ranked document matches and scores, and can be saved for later loading.

Guide limitations

  • Stemming is optional and requires a stemmer dependency such as PyStemmer.
  • The guide covers library-level indexing and retrieval, not a complete search application.
Documentation 1 ↗
Source-based practical guide

Expose an existing index through MCP

Launch the documented MCP server against a BM25 index so an LLM or agent can request retrieval.

Before you start

  • A Python environment and an existing bm25s index directory.
  • Bm25S installed with the mcp extra; the README recommends managing the environment with uv.

Steps

  1. Create and activate a virtual environment as shown in the README.

  2. Install bm25s[mcp] in that environment.

  3. Launch bm25 mcp launch with a port and the index directory.

  4. Use the exposed retrieve tool for top-k query results or get_info for index details.

Expected outcome

An MCP server exposing retrieval and index-information tools for the loaded collection.

Guide limitations

  • This setup requires an index directory; the guide does not cover building one.
  • The README's launch example uses a local environment and a specified port.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Retrieval-augmented applications

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Bm25SIndex document text and return the top-ranked documents for a query.Linux, macOS, WindowsThe project is publicly available on GitHub and lists an MIT license. The README documents pip installation; the MCP server requires the `mcp` extra.
LangsearchCrawl website or local data for a knowledge base.CLI, Self-hosted, Server
Hybrid Search RagCombine document retrieval and language-model generation in a configurable workflow.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
8 October 2026
Original website title
xhluca/bm25s · Official homepage title · retained in its source language
How we review sourcesSuggest a correction