OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Rq Transformer

Rq Transformer is a Python library implementing autoregressive and hierarchical causal transformers for modeling multidimensional sequences of discrete tokens, including residual-quantized image codes. It provides methods for training through a loss and generating token sequences; the residual-VQ encoder is a separate component.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Rq Transformer — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when the modeling task is autoregressive prediction or generation of discrete token sequences and the residual-quantization encoder is available separately or handled elsewhere.

Who it is for

  • Researchers and developers building autoregressive image-generation systems around residual-quantized codes.
  • Practitioners experimenting with hierarchical causal transformers over multidimensional token sequences.

Documented uses

  • Train a transformer to model residual-quantized image-token sequences.
  • Generate token sequences from a trained model or obtain logits for sampling.
  • Prototype hierarchical causal modeling across multiple sequence dimensions.
Before you choose

Requirements & limits

The public GitHub repository identifies the project as MIT-licensed.

  • The repository contains the transformer, not the residual-VQ encoder; the README points to a separate vector-quantization library.
  • The documented RQTransformer example accepts token tensors and can return training loss, logits, or generated sequences.
  • The hierarchical causal transformer example supports sequence dimensions configured in stages.
  • The README says compatibility with memcodes and possible usefulness with multi-headed VQ, but does not show integrated examples.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Train and sample a residual-quantized token sequence

Use the documented RQTransformer example to calculate training loss and generate a sequence.

Before you start

  • Python package installed as RQ-transformer
  • PyTorch
  • Input token tensors shaped to match the model's configured sequence dimensions

Steps

  1. Import torch and RQTransformer.

  2. Configure the model with a token count, spatial and depth sequence lengths, and transformer dimensions.

  3. Pass a token tensor to the model with return_loss=True and call backward() on the loss during training.

  4. After training, call model.generate() to sample a token sequence, or call the model to obtain logits.

Expected outcome

A trained autoregressive model can produce sampled residual-quantized token sequences, or logits for further sampling.

Guide limitations

  • The example generates token sequences, not decoded images.
  • The repository does not demonstrate a connected residual-VQ encoder-to-transformer pipeline.
  • The README notes generation after training; it does not provide a complete training schedule.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
lucidrains/RQ-Transformer · Official homepage title · retained in its source language
How we review sourcesSuggest a correction