OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Mimic Video

Mimic Video is a Python implementation of a video-action modeling approach for generalizable robot control, with PyTorch interfaces for training and action sampling.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Mimic Video — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want a Python codebase for research experiments connecting video observations and robot state to language-conditioned actions, especially when exploring training on LIBERO or comparing documented video-wrapper options.

Who it is for

  • Robotics researchers prototyping video-action models.
  • Python developers building robot-learning experiments with video, language prompts and joint-state inputs.
  • Researchers exploring training workflows on the documented LIBERO dataset.

Documented uses

  • Train action prediction from prompt, video, joint-state and action data.
  • Sample an action sequence from a prompt, video and joint state.
  • Build a LIBERO training loop using dataset-provided dimensions and batches.
  • Experiment with Cosmos or MiniMax H3 video wrappers and optional audio conditioning.
Before you choose

Requirements & limits

The public repository declares an MIT license. The documented install command is `pip install mimic-video`; the LIBERO example uses the `libero` extra.

  • The model interface expects video, joint state and text prompts; training additionally uses action sequences.
  • The README's tiny random-weight Cosmos wrapper is explicitly not suitable for actual learning.
  • The MiniMax H3 example requires installing Diffusers from its Git repository.
  • The code examples demonstrate model training and action sampling, but do not document physical-robot deployment.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up a LIBERO training example

Install the LIBERO extra and follow the documented pattern for loading a dataset and updating Mimic Video with its batches.

Before you start

  • Python environment with PyTorch.
  • The Mimic Video package and its LIBERO extra.
  • A LIBERO dataset identifier.
  • A real Cosmos or MiniMax H3 wrapper for actual learning; the README's tiny random-weight Cosmos example is not for actual learning.

Steps

  1. Install the package with the LIBERO extra: the command below.

    Documented command
    pip install "mimic-video[libero]"

  2. Create a LiberoDataset with a dataset identifier.

  3. Construct MimicVideo using the dataset's joint-state and action dimensions and a video wrapper.

  4. Iterate over dataset.get_dataloader(batch_size=1), calculate loss with model(**batch), and update action parameters using an optimizer.

Expected outcome

A documented pattern for training the model on batches from a LIBERO dataset.

Guide limitations

  • The README's tiny random-weight backbone example is described as runnable anywhere, but requires a real Cosmos or MiniMax H3 wrapper for actual learning.
  • This is a code recipe, not evidence of execution or physical-robot deployment.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
lucidrains/mimic-video · Official homepage title · retained in its source language
How we review sourcesSuggest a correction