Practical overview
When it fits
Use it when you want to study or prototype frame-by-frame video prediction from sequential image data, particularly when comparing adversarial and non-adversarial training.
Who it is for
- Researchers exploring adversarial approaches to video-frame prediction.
- Developers prototyping a next-frame forecasting model from sequential image data.
- Learners examining a TensorFlow GAN implementation with frame preprocessing and recursive prediction.
Documented uses
- Predict a video’s next frame from its four preceding frames.
- Prepare organized video-frame sequences as cropped training clips.
- Train or test adversarial and non-adversarial prediction runs.
- Generate a sequence of future frames by recursively feeding predictions back as input.
Requirements & limits
- The model uses four preceding frames to predict the next frame.
- Training data must be arranged as train and test directories containing separate folders for each video.
- The preprocessing script extracts random 32-by-32-pixel crops and filters for clips with movement.
- The README reports sharper-looking long sequences in its adversarial example, but no significant difference in its stated PSNR and Sharp Difference measurements.
- The example receives no action inputs, limiting its ability to predict specific changes in motion.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Prepare frame sequences and process training clips
Turn organized training and test video frames into processed clips for model training.
Before you start
- A local copy of the repository.
- Training and test frames organized in separate directories, with each video in its own subdirectory.
- A destination directory for processed clips.
Steps
Place the prepared frame-sequence directories in the locations you intend to use for training and testing.
From the Code directory, run the command below with
-tpointing to the full training-frame directory and-cpointing to the clips destination.python process_dataOptionally set
-nto choose the number of clips or use-oto overwrite prior processed data.
Expected outcome
The script processes random 32-by-32-pixel crops into clips for training.
Guide limitations
- The README says processing can take a few hours depending on the number of clips.
- The clips destination may contain many files; the README cautions against listing the directory with
ls.
Train or test the frame-prediction model
Run the repository’s training or testing workflow using processed clips and, optionally, a saved model.
Before you start
- A local copy of the repository.
- Processed training clips and a test-frame directory.
- For test-only use, a saved model path; the README provides a trained-model download for its Ms. Pac-Man example.
Steps
From the Code directory, run the command below.
python avg_runner.pySet the documented options as needed, including
-tfor the test-image directory,-lfor a saved model, and-rfor the number of recursive predictions.Use
-Tto run only a test step, or set-ato choose whether adversarial training is used.
Expected outcome
The runner trains or tests the network and saves run output under the selected name.
Guide limitations
- The README’s reported training time is about 24 hours for its specific Ms. Pac-Man experiment on a GTX 980TI GPU.
- Recursive predictions are not conditioned on game actions, so the example cannot reliably forecast specific turns.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 7 November 2026
- Original website title
- dyelax/Adversarial_Video_Generation · Official homepage title · retained in its source language
Related options
VideoPredictionICLR2016 — The README identifies this as the authors’ official Torch-based code for the paper implemented here.