Practical overview
When it fits
Use Chariot between obtaining an NLP dataset and training a model when you need a structured, reusable preprocessing pipeline and formatted training batches.
Who it is for
- Python developers preparing text and label data for NLP models.
- Machine-learning practitioners who want reusable, column-specific preprocessing and batch formatting.
Documented uses
- Prepare typical NLP datasets, with the README showing acquisition through chazutsu.
- Normalize and filter text, tokenize it, transform tokens and build vocabularies.
- Apply distinct preprocessing to dataset columns, such as review text and categorical labels.
- Format processed examples into batches, including padding, for model training.
- Save and reload fitted preprocessors or use batch iteration in a training workflow.
Requirements & limits
The README documents installation with pip. The repository identifies an Apache-2.0 license.
- Dataset preparation is shown through the companion project chazutsu; Chariot’s documented role is preprocessing and formatting.
- The README names Joblib for parallel preprocessing across dataset columns and spaCy for multilingual tokenization.
- Japanese tokenization options named in the README are MeCab and Janome.
- The README describes using pre-trained word vectors through the companion project chakin.
- The project documents both model-fit formatting and batch iteration, but does not provide a hosted training service.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Build and save an NLP preprocessing pipeline
Transform training text through normalization, tokenization, stop-word filtering and vocabulary creation, then save the fitted preprocessor.
Before you start
- Python environment with Chariot installed using the documented pip command.
- Training data available as a dataset or text column.
- For the documented tokenizer, spaCy support; the README also names MeCab or Janome as Japanese alternatives.
Steps
Install Chariot with the command below.
pip install chariotCreate a
Preprocessorand stack the documented Unicode normalizer, English tokenizer, English stop-word filter and vocabulary transformer.Fit the pipeline on training data.
Save it to a file with
preprocessor.save(...); load it later withPreprocessor.load(...).
Expected outcome
A fitted and saved preprocessing pipeline for the training data.
Guide limitations
- This recipe covers fitting and saving the preprocessing pipeline, not model training or deployment.
- The README does not specify environment setup for the tokenizer backends.
Prepare and format dataset columns
Apply separate preprocessing to review text and categorical labels, then run the dataset preprocessor on data.
Before you start
- Python environment with Chariot installed.
- Training data with the documented review and polarity columns.
- A padding length for the text formatter and the appropriate number of label classes.
Steps
Create a
DatasetPreprocessorand define a review-column chain using text normalization, tokenization, stop-word filtering, vocabulary creation and padding.Define a categorical-label formatter for the polarity column.
Fit the review preprocessing on the training review column.
Call
preprocess(data)and save the dataset preprocessor as a.tar.gzfile if needed.
Expected outcome
A dataset transformed by column-specific preprocessing, with a documented save-and-load format for the preprocessor.
Guide limitations
- The example uses its own
pad_lengthand class-count values; the README does not prescribe values for other datasets. - This recipe ends at preprocessing and does not cover evaluating model results.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 17 October 2026
- Original website title
- chakki-works/chariot · Official homepage title · retained in its source language