OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

G Mlp Pytorch

G Mlp Pytorch is a Python library implementing gMLP architectures in PyTorch, with model classes and examples for masked language modeling and image classification. The repository includes optional attention dimensions for both model types and an explicitly experimental multi-head configuration for sequence modeling.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

G Mlp Pytorch — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want Python/PyTorch model components and documented examples for experimenting with gMLP on token sequences or image classification, rather than a hosted inference service or a complete training pipeline.

Who it is for

  • PyTorch developers prototyping masked-language-modeling architectures.
  • Computer-vision developers building gMLP image-classification models.
  • Researchers comparing gMLP configurations, including optional attention or experimental multi-head settings.

Documented uses

  • Configure a token-sequence gMLP and obtain per-position vocabulary logits.
  • Configure a vision gMLP and obtain image-classification logits.
  • Explore optional attention dimensions or prototype the README's experimental multi-head sequence setup.
Before you choose

Requirements & limits

The repository is public and identifies the project as MIT-licensed. The README gives `pip install g-mlp-pytorch` as the installation command.

  • The examples provide model construction and forward-pass outputs, but do not document a complete training or evaluation workflow.
  • The vision examples support non-square image and patch dimensions as well as square dimensions.
  • The README describes the multi-head approach as experimental, unlike its standard masked-language-modeling and vision examples.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Build a masked-language-modeling gMLP

Create a token-sequence gMLP and obtain vocabulary logits for an input sequence.

Before you start

  • Python and PyTorch
  • Install the package with pip install g-mlp-pytorch

Steps

  1. Import torch, torch.nn, and gMLP from g_mlp_pytorch.

  2. Construct gMLP with vocabulary size, dimension, depth and sequence length; the example also sets circulant_matrix=True and uses nn.Tanh() for the spatial-gate activation.

  3. Create a token tensor with the configured sequence length, as in the README example.

  4. Pass the tensor to the model to produce logits shaped as batch, sequence length and vocabulary size.

Expected outcome

A configured gMLP returns per-token vocabulary logits for a masked-language-modeling experiment.

Guide limitations

  • The example demonstrates model construction and a forward pass, not a training or evaluation procedure.
  • The shown token tensor is randomly generated rather than drawn from a dataset.
Documentation 1 ↗
Source-based practical guide

Build an image-classification gMLP

Configure the vision model and obtain class logits for an image-shaped tensor.

Before you start

  • Python and PyTorch
  • Install the package with pip install g-mlp-pytorch

Steps

  1. Import torch and gMLPVision from g_mlp_pytorch.

  2. Set image size, patch size, number of classes, model dimension and depth when constructing gMLPVision.

  3. Create an image tensor matching the configured dimensions, as in the documented example.

  4. Pass the tensor to the model to produce class logits.

Expected outcome

A gMLPVision model returns one class-logit vector per input image.

Guide limitations

  • The README shows a forward-pass example, not a training or evaluation workflow.
  • The example uses a randomly generated tensor rather than a labeled image dataset.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
lucidrains/g-mlp-pytorch · Official homepage title · retained in its source language
How we review sourcesSuggest a correction