Practical overview
When it fits
Use Vall E when you want a code-based route from prepared speech data through model training and checkpoint export to reference-audio-conditioned text-to-speech synthesis.
Who it is for
- Developers or researchers building a text-to-speech training workflow from their own paired audio and normalized text.
- Users with DeepSpeed-supported GPU hardware and a CUDA or ROCm compiler who want to train or run the implementation from the command line.
Documented uses
- Quantize WAV audio and derive phonemes from normalized text.
- Configure and train autoregressive or non-autoregressive VALL-E-style models.
- Export trained checkpoints and synthesize speech from text using a reference recording.
Requirements & limits
- The project identifies Python 3.10.7 as its tested environment.
- Installation and training require a DeepSpeed-supported GPU and a pre-installed CUDA or ROCm compiler.
- A pretrained checkpoint is not yet available, so the repository does not offer ready-to-run synthesis from supplied models.
- EnCodec is the tokenizer foundation; substituting another audio tokenizer would require adapting data quantization, model inputs and decoding.
- The repository flags EnCodec’s CC-BY-NC 4.0 license for audio quantization and decoding.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Prepare data and train a VALL-E model
Quantize paired speech and text data, generate phonemes, configure a model and run its training script.
Before you start
- Python 3.10.7 is the version identified as tested.
- A GPU supported and tested by DeepSpeed, plus a pre-installed CUDA or ROCm compiler.
- WAV audio files and corresponding normalized text files.
Steps
Place the audio files with the .wav suffix and text files with the .normalized.txt suffix in a data folder.
Run the command below to quantize the data.
python -m vall_e.emb.qnt data/your_dataRun the command below to generate phonemes from text.
python -m vall_e.emb.g2p data/your_dataCreate autoregressive and non-autoregressive YAML configurations, then run the command below for the chosen configuration.
python -m vall_e.train yaml=config/your_data/ar_or_nar.yml
Expected outcome
The chosen autoregressive or non-autoregressive model is trained, with the latest checkpoint automatically saved when training is stopped through the documented CLI instruction.
Guide limitations
- The repository does not provide a pretrained checkpoint.
- The toy Colab example overfits a single utterance and is described as not usable.
- EnCodec is licensed under CC-BY-NC 4.0; the README says to follow its terms when using its code for quantization or decoding.
Export trained checkpoints and synthesize speech
Export the trained autoregressive and non-autoregressive models and use them with text and a reference recording in the synthesis CLI.
Before you start
- Trained autoregressive and non-autoregressive checkpoints.
- A text input and a reference audio file.
Steps
Export each trained model with the command below.
python -m vall_e.export zoo/ar_or_nar.pt yaml=config/your_data/ar_or_nar.ymlRun the command below with the corresponding exported checkpoint paths.
python -m vall_e --ar-ckpt zoo/ar.pt --nar-ckpt zoo/nar.pt
Expected outcome
The CLI is set up to write synthesized speech to the specified output path.
Guide limitations
- A pretrained model is not yet available; this workflow depends on trained and exported checkpoints.
- The README describes the linked single-utterance Colab as not usable.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 3 November 2026
- Original website title
- enhuiz/vall-e · Official homepage title · retained in its source language