OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Infrastructure & Serving

Megatron Lm

Megatron-LM is an open-source project comprising a reference training example with pre-configured scripts and Megatron Core, a composable library of GPU-optimized transformer training building blocks.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Megatron Lm — OpenViST project profile in AI Infrastructure & Serving.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Megatron-LM when developing or experimenting with transformer training workflows, especially where distributed training across GPUs or custom training-framework components are relevant.

Who it is for

  • Research teams learning distributed training or running training experiments.
  • Framework developers and ML engineers building custom training pipelines.

Documented uses

  • Use pre-configured scripts as a reference for transformer training experiments.
  • Build custom training workflows with transformer components, parallelism strategies and mixed-precision support.
  • Consult documented end-to-end training examples that include data preparation.
Before you choose

Requirements & limits

  • The repository provides both a reference-example path and a composable-library path.
  • Core lists tensor, pipeline, data, expert and context parallelism strategies.
  • Mixed-precision support listed in the README includes FP16, BF16, FP8 and FP4.
  • The quickstart covers an end-to-end training example with data preparation.
  • Source builds can require substantial memory.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Megatron-LM from a source clone

Clone the repository and install it in editable mode using the documented commands.

Before you start

  • Git and uv available in the environment.

Steps

  1. Run the command below.

    Documented command
    git clone https://github.com/NVIDIA/Megatron-LM.git

  2. Change into the Megatron-LM directory.

  3. Run the command below.

    Documented command
    uv pip install -e .

Expected outcome

The repository documents an editable source installation.

Guide limitations

  • Building from source can use substantial memory.
  • The README suggests setting MAX_JOBS to limit parallel compilation if the build runs out of memory.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
NVIDIA/Megatron-LM · Official homepage title · retained in its source language
How we review sourcesSuggest a correction