OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
RAG & Knowledge Systems

Chonkiejs

Chonkiejs is an MIT-licensed TypeScript library for splitting text and other supported inputs into chunks for retrieval-augmented generation and language-model workflows. Its core package includes recursive, token, sentence, semantic, code, table and fast chunkers, with local chunking described as using character-based tokenization.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Chonkiejs — OpenViST project profile in RAG & Knowledge Systems.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Chonkiejs when a TypeScript workflow needs a chunking step before retrieval or language-model processing, especially when the splitting strategy must be selected for the input.

Who it is for

  • TypeScript developers building document-ingestion or preprocessing pipelines for retrieval systems.
  • Teams that need to choose between structural, token-sized or embedding-based text boundaries.

Documented uses

  • Split documents into chunks before adding them to a retrieval workflow.
  • Create fixed-size token chunks, optionally with overlap.
  • Group sentences into chunks while respecting sentence boundaries.
  • Use embeddings to identify semantic split points.
  • Prepare code for AST-aware chunking with a language grammar.
Before you choose

Requirements & limits

The project is a public GitHub repository under the MIT license. The README documents installation of @chonkiejs/core through npm.

  • The core package documents local chunkers and character-based tokenization, while Hugging Face tokenizer support is in a separate @chonkiejs/token package.
  • SemanticChunker requires an embeddings function or an object with an embed method.
  • CodeChunker requires web-tree-sitter and a language grammar.
  • The project is a TypeScript port of the Python Chonkie library and is not yet at feature parity with it.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Chunk text with the local core package

Install Chonkiejs core, create a recursive chunker and retrieve text and token counts for each chunk.

Before you start

  • A TypeScript project using npm
  • Text to split

Steps

  1. Install the package with the command below.

    Documented command
    npm install @chonkiejs/core

  2. Import RecursiveChunker from @chonkiejs/core.

  3. Create a chunker with await RecursiveChunker.create({ chunkSize: 512 }).

  4. Call await chunker.chunk(text) and read each result's text and tokenCount.

Expected outcome

A list of chunks with text and token-count values, ready for a later retrieval or language-model workflow.

Guide limitations

  • The README describes Chonkiejs as under active development and not yet at feature parity with the Python library.
  • The example demonstrates chunking but does not document an end-to-end retrieval or RAG pipeline.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Retrieval-augmented applications

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
ChonkiejsSplit documents into chunks before adding them to a retrieval workflow.CloudThe project is a public GitHub repository under the MIT license. The README documents installation of @chonkiejs/core through npm.
LangsearchCrawl website or local data for a knowledge base.CLI, Self-hosted, Server
Hybrid Search RagCombine document retrieval and language-model generation in a configurable workflow.
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
13 October 2026
Original website title
feyninc/chonkiejs · Official homepage title · retained in its source language
How we review sourcesSuggest a correction