OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Ngpt Pytorch

Ngpt Pytorch is a Python implementation of nGPT, a normalized GPT transformer that learns representations on the hypersphere. It provides a PyTorch model example for computing training loss and gradients and producing token logits, plus a script identified for testing on Enwik8.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Ngpt Pytorch — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want a compact PyTorch starting point for experimenting with nGPT-style language modeling, or to inspect the documented model example before exploring the Enwik8 script.

Who it is for

  • Developers prototyping language-model training in PyTorch.
  • Researchers exploring normalized transformer components and hypersphere representation learning.

Documented uses

  • Instantiate a normalized GPT model with selected vocabulary size, dimension and depth.
  • Compute training loss and gradients from token inputs.
  • Produce token logits for a model input.
  • Invoke the repository's Enwik8 test script.
Before you choose

Requirements & limits

The public GitHub repository specifies an MIT license and documents installation with pip.

  • The README's example shows direct model construction and a loss/backpropagation/logits flow.
  • Enwik8 is named as the test target, but further test setup and result interpretation are not described.
  • The vision-transformer adaptation is linked as a separate project rather than included as a demonstrated feature here.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install nGPT-pytorch and run the documented model example

Set up the package and use its documented example to compute loss, gradients and token logits.

Before you start

  • Python environment with pip
  • PyTorch

Steps

  1. Install the package with the command below.

    Documented command
    pip install nGPT-pytorch

  2. Import torch and nGPT from nGPT_pytorch.

  3. Create an nGPT model and integer token input as shown in the README.

  4. Call the model with return_loss=True, backpropagate with loss.backward(), then call the model to obtain logits.

Expected outcome

The example computes a training loss and gradients and returns token logits.

Guide limitations

  • The README example uses randomly generated token IDs and does not document data preparation or a complete training loop.
  • The example documents model use, not a deployment procedure.
Documentation 1 ↗
Source-based practical guide

Run the named Enwik8 test script

Use the repository's documented script for its Enwik8 test.

Before you start

  • A local checkout of the repository
  • Python environment with the project's dependencies

Steps

  1. Open the repository in a Python environment.

  2. Run the command below as listed under the README's Enwik8 test section.

    Documented command
    python train.py

Expected outcome

Invokes the repository's Enwik8 test script.

Guide limitations

  • The README does not provide further test configuration, expected output or result interpretation.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
13 October 2026
Original website title
lucidrains/nGPT-pytorch · Official homepage title · retained in its source language
How we review sourcesSuggest a correction