Practical overview
When it fits
Use it when studying or prototyping model components for temporal fusion in image and video pipelines, rather than seeking a complete hosted text-to-video generation service.
Who it is for
- Researchers and developers exploring video-generation architectures.
- PyTorch users experimenting with temporal attention, pseudo-3D convolutions, or transitions from image to video model training.
Documented uses
- Experiment with spatiotemporal attention and pseudo-3D convolution on video features.
- Use the temporal modules with image features, where temporal processing is skipped.
- Configure a SpaceTimeUnet example for image or video tensors, including video input processed without time operations.
Requirements & limits
The public GitHub repository lists an MIT license. The README provides a pip installation command and Python usage examples.
- The README labels the implementation work in progress.
- Examples demonstrate model components and tensor handling, not a complete text-prompt-to-video generation workflow.
- The README lists compatibility with dalle2-pytorch for training as a to-do item.
- The examples use PyTorch and show image and video tensor shapes, including an option to disable temporal processing.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Install the package and pass video features through temporal modules
Use the package’s PseudoConv3d and SpatioTemporalAttention modules with a video-feature tensor.
Before you start
- Python environment with pip
- PyTorch (
torch)
Steps
Install the package with the command below.
pip install make-a-video-pytorchImport
PseudoConv3dandSpatioTemporalAttentionfrommake_a_video_pytorch.Create the modules with
dim=256,kernel_size=3forPseudoConv3d, anddim=256,dim_head=64,heads=8forSpatioTemporalAttention.Pass a video tensor shaped
(1, 256, 8, 16, 16)to both modules; the README identifies its dimensions as batch, features, frames, height, and width.
Expected outcome
The documented example produces convolution and attention outputs shaped `(1, 256, 8, 16, 16)`.
Guide limitations
- The project labels itself work in progress.
- This example processes video features with component modules; it does not document a text-to-video generation workflow or training procedure.
Use SpaceTimeUnet with image and video tensors
Instantiate the documented SpaceTimeUnet and pass it image and video tensors, with an option to ignore time for video input.
Before you start
- Python environment with PyTorch (
torch) - CUDA available, because the documented example moves the model and tensors to CUDA
Steps
Import
SpaceTimeUnetfrommake_a_video_pytorch.Instantiate it with the README’s example parameters:
dim=64,channels=3,dim_mult=(1, 2, 4, 8),resnet_block_depths=(1, 1, 1, 2),temporal_compression=(False, False, False, True),self_attns=(False, False, False, True),condition_on_timestep=False,attn_pos_bias=False, andflash_attn=True; move the moPass a CUDA image tensor shaped
(1, 3, 128, 128)to the model, then pass a CUDA video tensor shaped(1, 3, 16, 128, 128).Optionally pass the video tensor with
enable_time=Falseto treat the video as images, as shown in the README.
Expected outcome
The README shows image and video outputs with the same shapes as their respective inputs, and shows a video call with temporal processing disabled.
Guide limitations
- The README’s shape checks are examples, not independent verification of successful execution.
- The project is marked work in progress, and the README lists compatibility with dalle2-pytorch training as a todo.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 14 October 2026
- Original website title
- lucidrains/make-a-video-pytorch · Official homepage title · retained in its source language