Practical overview
When it fits
Use DeCLUTR when you want research code for training or studying unsupervised transformer-based text embeddings, rather than a hosted search product or a general-purpose text application.
Who it is for
- NLP researchers and developers exploring unsupervised representation learning.
- Practitioners who need sentence embeddings for semantic similarity or semantic search experiments.
- Users who want to train or evaluate transformer-based text representations with AllenNLP and PyTorch.
Documented uses
- Train text encoders to produce sentence embeddings without labeled training examples.
- Represent text for semantic similarity experiments.
- Generate embeddings that can support semantic search.
- Evaluate representations with the SentEval workflow referenced by the project.
Requirements & limits
The public GitHub repository identifies the project as open source under the Apache-2.0 license.
- Training requires a suitably long text corpus; the minimum document length is determined by num_anchors and max_span_len.
- The README recommends PyTorch with CUDA support for training.
- AllenNLP configuration drives the documented training command.
- The repository includes a noted original implementation bug affecting contrastive-loss gradient scaling; review its training notes when interpreting results.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Prepare a text dataset and start DeCLUTR training
Configure a line-based training dataset and launch a DeCLUTR training run with AllenNLP.
Before you start
- Python 3.6.1 or later and a Python virtual environment.
- DeCLUTR installed from Git or cloned and installed in editable mode.
- PyTorch with CUDA support is recommended by the README for training.
- A text file with one document or other text item per line; documents must meet the configured minimum of num_anchors × max_span_len × 2 whitespace tokens.
Steps
Prepare a training text file, using the documented WikiText-103 preprocessing script if appropriate.
Set the training data path in the training configuration, or provide it as an override.
Run AllenNLP training with training_config/declutr.jsonnet, a serialization directory, the train-data-path override, and the declutr package included.
Expected outcome
The training run is configured to save models, vocabulary, configuration and log files in the selected serialization directory.
Guide limitations
- The dataset length requirement depends on the num_anchors and max_span_len configuration values.
- The README notes a bug in the original implementation affecting contrastive-loss gradient scaling; consult the repository's training notes before interpreting results.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 16 October 2026
- Original website title
- JohnGiorgi/DeCLUTR · Official homepage title · retained in its source language