OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Performer Pytorch

Performer Pytorch is a Python library implementing the Performer transformer architecture in PyTorch, using FAVOR+ positive orthogonal random features for linear attention. It provides language-model and encoder–decoder classes, plain Performer models, and standalone self-attention, cross-attention and FastAttention modules.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Performer Pytorch — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when developing a PyTorch transformer and you want the documented Performer/FAVOR+ attention implementation, either as a model or as attention modules for integration into a larger architecture.

Who it is for

  • PyTorch developers prototyping Performer-based language models.
  • Researchers and engineers experimenting with FAVOR+ linear attention.
  • Developers adapting transformer attention layers or building encoder–decoder models.

Documented uses

  • Create a Performer language model with configurable dimensions, depth, heads and sequence length.
  • Build a plain Performer or encoder–decoder model for supplied tensor inputs.
  • Replace or experiment with transformer self-attention using standalone linear-attention modules.
  • Train the documented encoder–decoder example with a loss and backpropagation.
Before you choose

Requirements & limits

The public GitHub repository lists an MIT license. Installation is documented through pip.

  • The README offers both complete model classes and standalone attention modules, allowing different levels of integration.
  • The language-model example includes adjustable random-feature count and projection redraw interval; its comments note that more frequent redraws slow training.
  • The README calls for installing repository requirements when training an autoregressive model.
  • The documented examples use PyTorch tensors and show masks for language-model and encoder–decoder inputs.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Performer Pytorch

Install the library for use in a Python/PyTorch project.

Before you start

  • A Python project using PyTorch.

Steps

  1. Run the command below.

    Documented command
    pip install performer-pytorch

  2. If you plan to train an autoregressive model, run the command below from the repository.

    Documented command
    pip install -r requirements.txt

Expected outcome

The package is installed; autoregressive-model training also has the repository requirements installed.

Guide limitations

  • These instructions cover installation, not a complete training workflow.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
lucidrains/performer-pytorch · Official homepage title · retained in its source language
How we review sourcesSuggest a correction