OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Developer Tools & SDKs

The Alignment Handbook

The Alignment Handbook is an open-source Python repository of scripts and recipes for continued pretraining, fine-tuning, evaluation and preference alignment of language models.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

The Alignment Handbook — OpenViST project profile in AI Developer Tools & SDKs.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you need code and recipes for a language-model training workflow, from continued pretraining and supervised fine-tuning through preference alignment.

Who it is for

  • Developers and researchers training or adapting language models.
  • Practitioners reproducing model-training recipes or working with human- or AI-preference data.

Documented uses

  • Continue pretraining a model on a new language or domain dataset.
  • Fine-tune a model to follow instructions using prepared training data.
  • Train and evaluate preference-alignment approaches including DPO and ORPO.
  • Reproduce a model-training run from a recipe with its parameters recorded in YAML.
Before you choose

Requirements & limits

  • The repository provides scripts for training and evaluation, while recipes capture parameters for individual runs.
  • Supported stages include continued pretraining, SFT, DPO preference alignment and ORPO fine-tuning; the handbook also covers reward modeling and rejection sampling.
  • The scripts support DeepSpeed ZeRO-3 for distributed full-weight training and LoRA/QLoRA for parameter-efficient fine-tuning.
  • Setup specifies Python 3.11 and PyTorch 2.6.0; the PyTorch installation depends on hardware.
  • The repository is licensed under Apache-2.0.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the Python environment

Prepare a Python environment and install the specified PyTorch build before proceeding to the repository's dependency-installation stage.

Before you start

  • Python 3.11
  • uv
  • A PyTorch build selected for the available hardware

Steps

  1. Create and activate a Python 3.11 virtual environment with uv, then upgrade pip.

  2. Install PyTorch 2.6.0 using the repository's CUDA 12.6 package index, or consult the linked PyTorch installation page for the hardware-appropriate installation.

  3. Continue with the repository's remaining package-dependency instructions.

Expected outcome

A Python virtual environment with the specified PyTorch version installed, ready for the next dependency setup stage.

Guide limitations

  • The PyTorch build is hardware-dependent; the repository directs users to PyTorch's installation page for alternatives.
  • This setup does not by itself run a training recipe.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
7 October 2026
Original website title
huggingface/alignment-handbook · Official homepage title · retained in its source language
How we review sourcesSuggest a correction