OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Complex Valued Transformer

Complex Valued Transformer is an MIT-licensed Python library implementing a complex-valued Transformer architecture for sequence modeling in PyTorch. Its documented model includes complex-domain rotary embeddings and FlashAttention v1 compatibility, and its example creates a causal transformer that turns token IDs into per-position logits.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Complex Valued Transformer — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want to prototype a complex-valued Transformer component in a Python/PyTorch sequence-modeling project and are prepared to build the surrounding training or evaluation workflow.

Who it is for

  • PyTorch developers exploring complex-valued attention and Transformer designs.
  • Researchers and practitioners prototyping character-level language models or algorithmic sequence tasks.

Documented uses

  • Construct a causal Transformer with configurable vocabulary size, depth, dimensions and attention heads.
  • Generate logits from batches of token IDs as a building block for sequence-model experiments.
  • Explore complex-domain rotary embeddings and compare complex attention approaches.
Before you choose

Requirements & limits

The public GitHub repository identifies the project as MIT licensed. The README documents installation through pip.

  • The README shows model construction and token-to-logit output, not a complete training or evaluation pipeline.
  • The project describes enwik8 and algorithmic tasks as evaluation areas but does not supply benchmark results in the provided documentation.
  • The author reports no remarkable result, so the repository supports experimentation rather than a documented performance claim.
  • FlashAttention v1 compatibility is listed, but the supplied documentation does not specify additional environment or hardware requirements.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install the library and construct a causal transformer

Set up the package and follow the documented example to create a model and produce logits from token IDs.

Before you start

  • Python environment with PyTorch available, as the example imports torch.
  • Install access for the complex-valued-transformer package.

Steps

  1. Install the package with the command below.

    Documented command
    pip install complex-valued-transformer

  2. Import torch and ComplexTransformer from complex_valued_transformer.

  3. Construct ComplexTransformer with the documented parameters, including causal=True and complete_complex=True.

  4. Create token IDs with torch.randint and pass them to the model to obtain logits.

Expected outcome

The example returns logits shaped (2, 1024, 256) for a batch of two 1,024-token sequences and a vocabulary of 256 tokens.

Guide limitations

  • The README example demonstrates model construction and inference only; it does not document training, dataset preparation or evaluation steps.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
lucidrains/complex-valued-transformer · Official homepage title · retained in its source language
How we review sourcesSuggest a correction