OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Developer Tools & SDKs

Ema Pytorch

Ema Pytorch is a Python/PyTorch library for wrapping neural-network modules and maintaining exponential moving-average versions of their parameters during training.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Ema Pytorch — OpenViST project profile in AI Developer Tools & SDKs.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when a PyTorch training workflow needs a maintained moving-average model alongside the live network, including workflows that require configurable update timing or nested teacher-output routing.

Who it is for

  • PyTorch developers building custom model-training pipelines
  • Researchers comparing live and EMA model weights during evaluation
  • Teams implementing self-supervised or teacher-student training with nested modules

Documented uses

  • Maintain an EMA copy of a PyTorch model
  • Control EMA decay, warm-up and update frequency
  • Record multiple EMA checkpoint streams and synthesize another EMA model
  • Route target-module outputs into selected online submodules
Before you choose

Requirements & limits

The repository is public and MIT-licensed. Installation is documented with `pip install ema-pytorch`; the examples use Python and PyTorch code run in the developer's own environment.

  • It is a library integrated into an existing PyTorch loop, not a hosted training application.
  • The standard wrapper exposes configurable beta, warm-up and update intervals.
  • Post-hoc synthesis requires multiple EMA settings and recorded checkpoints.
  • Nested-module routing requires matching online and EMA submodule paths and compatible forward arguments.
  • The documented examples show Python usage with PyTorch modules and caller-managed update steps.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Add an EMA model to a PyTorch training loop

Install the library, wrap a PyTorch module, update its EMA copy and invoke both models.

Before you start

  • Python and PyTorch
  • A PyTorch neural-network module
  • A training loop that can call the EMA update method
  • The ema-pytorch package

Steps

  1. Install the package with the command below.

    Documented command
    pip install ema-pytorch

  2. Create the PyTorch network and wrap it with EMA, setting values such as beta, update_after_step and update_every.

  3. After the network parameters are changed by the training process, call ema.update().

  4. Invoke the live network or the EMA wrapper with input data; access the EMA copy through ema.ema_model when needed.

  5. Save the complete EMA wrapper when preserving its step count and warm-up state.

Expected outcome

A PyTorch model and an updating EMA counterpart are available through the documented wrapper interface.

Guide limitations

  • The examples require the caller to integrate parameter updates and ema.update() calls into its own training loop.
  • The documented recommendation is to save the complete wrapper when preserving step and warm-up information.
Documentation 1 ↗
Source-based practical guide

Configure post-hoc EMA synthesis

Record multiple EMA checkpoint streams during training and synthesize a model with another sigma-relative setting.

Before you start

  • Python and PyTorch
  • A PyTorch neural-network module
  • At least two sigma_rels values
  • A checkpoint folder for the recorded EMA checkpoints
  • A loop that calls emas.update()

Steps

  1. Create PostHocEMA around the network with multiple sigma_rels, an update interval, checkpoint frequency and checkpoint folder.

  2. Run the training process and call emas.update() as the network changes.

  3. After checkpoints have been collected, call synthesize_ema_model with the desired sigma_rel.

  4. Invoke the synthesized model with input data.

Expected outcome

A synthesized EMA model is produced from the recorded EMA checkpoints using the selected sigma-relative value.

Guide limitations

  • The example states that at least two EMA settings are needed for synthesis.
  • The method depends on checkpoints being recorded during the training process.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
4 October 2026
Original website title
lucidrains/ema-pytorch · Official homepage title · retained in its source language
How we review sourcesSuggest a correction