OpenViSTAI · Agents · Blockchain · Crypto
Menu
AI Models

Puppeteer

Puppeteer is an open-source research implementation of hierarchical world models for whole-body humanoid control from visual observations, with code to train and evaluate low-level tracking and high-level control models and checkpoints for both levels.

Independent OpenViST project profile, based on linked sources.

Puppeteer — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Puppeteer when you want to study or experiment with hierarchical humanoid control in the repository’s MuJoCo and DMControl tasks, rather than a general-purpose AI model or physical-robot deployment package.

Who it is for

  • Robotics and machine-learning researchers studying humanoid control.
  • Developers experimenting with simulated movement and visual terrain navigation.

Documented uses

  • Evaluate released low- and high-level model checkpoints.
  • Train models for tracking, walking or corridor navigation.
  • Compare simulated locomotion with vision-based navigation across terrain obstacles.
Before you choose

Requirements & limits

The public GitHub repository is licensed under MIT; its README notes that third-party code has separate licenses. Model checkpoints are linked from the repository.

  • Training requires a GPU with at least 24 GB of memory.
  • The documented environment options are Docker and Conda.
  • The repository provides eight tasks: stand, walk, run, corridor, hurdles-corridor, gaps-corridor, walls-corridor and stairs-corridor.
  • The README recommends default hyperparameters and suggests Weights & Biases for tracking training.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Evaluate a released corridor-navigation checkpoint

Evaluate a downloaded high-level checkpoint for a visual corridor task using its low-level tracking checkpoint.

Before you start

  • A Docker or Conda environment with the repository dependencies.
  • A machine with a GPU containing at least 24 GB of memory for training; the repository states this requirement for training, not evaluation.
  • Downloaded low-level tracking and high-level corridor checkpoints.

Steps

  1. Set up the dependencies using the provided Dockerfile or the Conda environment file.

  2. Download the low-level tracking and high-level corridor checkpoints.

  3. From the repository, run the command below, replacing the checkpoint paths with the downloaded files.

    Documented command
    python evaluate.py task=corridor low_level_fp=/path/to/tracking.pt checkpoint=/path/to/corridor-1.pt

Expected outcome

Runs the documented evaluation workflow for the corridor task with a low-level tracking checkpoint and a high-level corridor checkpoint.

Guide limitations

  • The guide follows the repository's example command; it does not establish that a particular setup or run has been tested.
  • The README says high-level checkpoints were trained with the same low-level checkpoint.
Source-based practical guide

Set up Puppeteer’s dependencies

Prepare the repository environment for the documented training and inference workflows.

Before you start

  • A machine with Docker, or Conda and the system packages recommended in the repository Dockerfile.

Steps

  1. For Docker setup, enter the repository's docker directory.

  2. Build the image with the command below.

    Documented command
    docker build . -t /puppeteer:1.0.0
  3. Alternatively, create the Conda environment with conda env create -f docker/environment.yaml.

Expected outcome

Creates the provided Docker image or Conda environment for the repository's training and inference dependencies.

Guide limitations

  • Manual Conda setup may require additional dependencies depending on existing system packages; consult the repository Dockerfile for recommended system packages.
  • This is a dependency setup guide; it does not cover running a training or evaluation task.
Evidence and transparency

Sources & review

Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
nicklashansen/puppeteer · Official homepage title · retained in its source language