Practical overview
When it fits
Use Textrl when a text-generation project needs a documented trainer interface for reinforcement-learning or preference-based training, and you can provide the model, training data and reward or feedback signals required by the selected method.
Who it is for
- Developers building or fine-tuning text-generation models with Python and Hugging Face Transformers.
- Researchers and practitioners working with reinforcement learning from feedback or preference datasets.
Documented uses
- Configure online reinforcement-learning trainers such as GRPO, RLOO and REINFORCE++.
- Train with preference methods including DPO and KTO using their required data formats.
- Define callable reward functions for generated completions or train a pairwise reward model.
- Use PEFT in a text-generation reinforcement-learning workflow.
Requirements & limits
The repository is public and lists an MIT license.
- Choose the training method based on available data: prompt-only for online methods, chosen/rejected pairs for pairwise preference training, or prompt/completion/label for KTO.
- The README identifies vLLM rollout as a GRPO option and says distributed training can be launched with Accelerate.
- The project documents support for PEFT, but does not specify all supported methods or configurations.
- PPO, OnlineDPO, ORPO, CPO, SimPO and binary BCO are listed as unsupported with TRL 0.29 and later.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up a GRPO training workflow with a callable reward
Configure and start a text-generation training run using GRPO and a custom reward function.
Before you start
- Python environment with Textrl installed.
- A supported Hugging Face Transformers generation model and tokenizer.
- Prompt-only training data, such as a list of prompts.
Steps
Install the core package with the command below.
pip install textrlDefine a reward callable that accepts prompts and completions and returns a list of scores; decorate it with
@reward_fn.Load a model with
load_model, configureTextRLConfigwithalgo="grpo", and create anOnlineTrainerwith the model, tokenizer, reward and prompt dataset.Call
trainer.train()to start training.
Expected outcome
A GRPO training run configured to optimize generated text against the supplied callable reward.
Guide limitations
- The project says specific model and algorithm compatibility should be checked.
- The example uses PEFT with LoRA, but does not specify hardware requirements.
- The README lists vLLM rollout for GRPO as an optional additional setup.
Configure DPO training with preference data
Set up a preference-learning run using DPO and a pairwise preference dataset.
Before you start
- Python environment with Textrl installed.
- A supported Hugging Face Transformers model.
- Preference data with
prompt,chosenandrejectedfields.
Steps
Install the core package with the command below.
pip install textrlLoad the policy model, tokenizer and reference model with
load_model.Set
TextRLConfigtoalgo="dpo"and create aPreferenceTrainerwith the models, tokenizer and preference dataset.Call
trainer.train()to start training.
Expected outcome
A DPO training run configured to learn from chosen and rejected responses.
Guide limitations
- The README’s example uses a dataset from the Hugging Face Hub and 4-bit quantization, but does not specify hardware requirements.
- Specific model and algorithm compatibility should be checked.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 15 October 2026
- Original website title
- voidful/TextRL · Official homepage title · retained in its source language