OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Developer Tools & SDKs

N Grammer Pytorch

N Grammer Pytorch is a Python/PyTorch library that implements N-Grammer, augmenting Transformer models with latent n-grams through a VQNgrammer module and providing helpers to group its parameters for optimizer learning-rate configuration.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

N Grammer Pytorch — OpenViST project profile in AI Developer Tools & SDKs.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use the library when implementing latent n-gram augmentation in a PyTorch Transformer or when you need its helper functions to create distinct optimizer parameter groups.

Who it is for

  • Developers building or adapting PyTorch Transformer models
  • Researchers and practitioners configuring separate optimizer learning rates for N-Grammer parameters

Documented uses

  • Add a VQNgrammer component to a PyTorch model workflow.
  • Apply the component to tensor inputs.
  • Separate N-Grammer parameters from other model parameters when configuring an optimizer.
Before you choose

Requirements & limits

The public GitHub repository identifies the project as MIT-licensed and documents installation through pip.

  • The library supplies a model component and parameter-group helpers, not a complete model or training pipeline.
  • The documented VQNgrammer initialization requires model-specific dimension and vocabulary settings.
  • The README gives 1e-2 as the paper-recommended N-Grammer learning rate and shows how to set it separately from other parameters.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install and add VQNgrammer to a PyTorch model

Install the package and use its documented VQNgrammer component on a tensor.

Before you start

  • Python environment with PyTorch available
  • An existing PyTorch model or tensor inputs

Steps

  1. Install the package with the command below.

    Documented command
    pip install n-grammer-pytorch

  2. Import VQNgrammer from n_grammer_pytorch.

  3. Initialize it with the model dimensions and settings, including cluster count, head dimensions, n-gram vocabulary size, embedding dimension, and decay.

  4. Pass a tensor to the module in the model workflow.

Expected outcome

A VQNgrammer component can be incorporated into a PyTorch model; the README example returns a tensor with the same illustrated dimensions as its input.

Guide limitations

  • The README demonstrates a component call, not a complete Transformer integration or training workflow.
  • The example configuration is illustrative and may need to match the model's dimensions.
Documentation 1 ↗
Source-based practical guide

Separate N-Grammer optimizer parameters

Create optimizer parameter groups that give N-Grammer parameters a distinct learning rate.

Before you start

  • A PyTorch model containing VQNgrammer modules
  • PyTorch optimizer such as Adam

Steps

  1. Import get_ngrammer_parameters or get_ngrammer_param_groups from n_grammer_pytorch.

  2. Pass the root model to the selected helper.

  3. Supply the resulting parameter groups to an optimizer; the documented example uses Adam.

Expected outcome

The optimizer can treat N-Grammer and other model parameters as separate groups; the documented helper sets the N-Grammer learning rate to 1e-2.

Guide limitations

  • The README reports 1e-2 as the paper-recommended N-Grammer learning rate; it does not establish that this setting suits every model or training task.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
lucidrains/n-grammer-pytorch · Official homepage title · retained in its source language
How we review sourcesSuggest a correction