OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
RAG & Knowledge Systems

Adaptive Chunking

Adaptive Chunking is a Python library for evaluating text-chunking strategies and selecting one for each document in a retrieval-augmented generation (RAG) workflow.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Adaptive Chunking — OpenViST project profile in RAG & Knowledge Systems.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when preparing documents for RAG and the same splitting approach may not suit every document, or when evaluating how candidate chunk boundaries affect document structure and context.

Who it is for

  • Developers preparing documents for a RAG ingestion pipeline.
  • Information-retrieval practitioners comparing chunking strategies across documents.
  • Teams extending chunking methods or quality metrics for their own document domains.

Documented uses

  • Select a splitting strategy for each document rather than using one method for an entire corpus.
  • Parse PDFs and create chunk records for downstream retrieval workflows.
  • Measure chunk outputs for size, semantic cohesion, contextual coherence, structural integrity and broken references.
  • Compare alternative chunking approaches using shared intrinsic metrics.
Before you choose

Requirements & limits

The public GitHub repository identifies the project as MIT-licensed. Python 3.11 or later is specified.

  • The core installation contains splitters and metrics; PDF and Excel parsing requires the parsing extra.
  • Docling is the default PDF parsing backend; PyMuPDF and Azure Document Intelligence are also listed.
  • Some metrics require the spaCy en_core_web_sm model.
  • The optional coreference-resolution extra is CC BY-NC-SA 4.0 with a non-commercial condition.
  • The library prepares and evaluates chunks; downstream retrieval and generation remain separate workflow stages.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Parse and chunk PDF documents

Use the library’s PDF parsing interface to turn a PDF file or directory of PDFs into chunk records.

Before you start

  • Python 3.11 or later.
  • The repository cloned locally.
  • Install the parsing dependencies with pip install -e ".[parsing]".
  • A PDF file or directory of PDFs.

Steps

  1. Clone the repository and change into its directory.

  2. Install the parsing extra with the command below.

    Documented command
    pip install -e ".[parsing]"

  3. Import chunk_files from adaptive_chunking and pass it a PDF file or directory, optionally setting chunk_size=600 and chunk_overlap=50.

  4. Inspect the returned records, which include document name, chunk index, text, page information, title context and chunk length.

Expected outcome

A list of chunk records produced from the input PDF file or directory.

Guide limitations

  • The parsing extra is required for the documented PDF parsing workflow.
  • The README does not specify a parser-independent output guarantee or downstream retrieval setup.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Retrieval-augmented applications

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
Adaptive ChunkingSelect a splitting strategy for each document rather than using one method for an entire corpus.Linux, macOS, WindowsThe public GitHub repository identifies the project as MIT-licensed. Python 3.11 or later is specified.
LangsearchCrawl website or local data for a knowledge base.CLI, Self-hosted, Server
Hybrid Search RagCombine document retrieval and language-model generation in a configurable workflow.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
14 October 2026
Original website title
ekimetrics/adaptive-chunking · Official homepage title · retained in its source language
How we review sourcesSuggest a correction