Practical overview
When it fits
Use FaceFormer when you want to explore a local research workflow for turning speech audio into 3D facial motion, and can work with the repository's Python environment, mesh data and model weights.
Who it is for
- Researchers exploring speech-driven facial animation.
- 3D-character developers prototyping audio-to-facial-motion workflows.
Documented uses
- Generate a 3D facial-motion sequence from speech audio and a neutral mesh.
- Render a demo animation using the documented BIWI or FLAME topology workflows.
- Prepare BIWI, VOCASET or custom data for model training and visualization.
Requirements & limits
The repository is public and lists an MIT license.
- The README specifies Ubuntu 18.04.1, Python 3.7 and PyTorch 1.9.0.
- The project requires dependencies including ffmpeg and MPI-IS/mesh.
- Demo generation requires pretrained weights and a WAV input.
- BIWI and VOCASET data must be requested from their respective sources.
- Custom datasets require vertex data, WAV audio, subject templates and dataset-specific settings.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Generate a demo animation from WAV audio
Use a documented pretrained BIWI or VOCASET model to render an audio-driven 3D facial animation.
Before you start
- Ubuntu 18.04.1, Python 3.7 and PyTorch 1.9.0.
- The packages listed in requirements.txt, ffmpeg and MPI-IS/mesh.
- The pretrained model for BIWI or VOCASET and a WAV audio file.
Steps
Download the pretrained model for BIWI or VOCASET and place it in the corresponding dataset folder.
Place the WAV file in demo/wav, or set the demo's --wav_path to its location.
Run demo.py with the documented settings for the chosen BIWI or FLAME topology.
Find the rendered video in demo/output.
Expected outcome
A rendered video of speech-driven facial motion saved in demo/output.
Guide limitations
- The README's demo commands specify dataset-specific mesh topology and model settings.
- The documented environment is Ubuntu 18.04.1, Python 3.7 and PyTorch 1.9.0.
Prepare a custom dataset for FaceFormer training
Arrange custom mesh and audio data and define the dataset splits needed by the training workflow.
Before you start
- A local FaceFormer checkout.
- Vertex data in .npy format, audio in .wav format, subject templates in templates.pkl and an exported template in .ply format.
Steps
Create a dataset directory with vertices_npy and wav subfolders, and place the corresponding files there.
Save the subject templates as templates.pkl in the dataset directory and place an exported template in its templates subfolder.
Specify the dataset name, vertex dimension and train, validation and test subjects in the documented main.py arguments.
Adjust feature_dim or period for the dataset if needed.
Expected outcome
The custom data and split settings are arranged for the documented training stage.
Guide limitations
- The README describes preparation and training commands but does not establish that a particular custom dataset will work without adjustment.
- The vertex dimension is the number of mesh vertices multiplied by three.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 16 October 2026
- Original website title
- EvelynFan/FaceFormer · Official homepage title · retained in its source language