OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Transformer In Transformer

Transformer In Transformer is a Python/PyTorch implementation of a vision architecture that pairs pixel-level attention with patch-level attention for image classification.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Transformer In Transformer — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want a documented PyTorch model interface for image classification with attention at both pixel and patch levels, and can supply the surrounding data and training workflow.

Who it is for

  • PyTorch developers adding a configurable vision model to an image-classification pipeline.
  • Researchers and practitioners exploring attention across both pixel and patch tokens.
  • Prototypers comparing image-classification configurations using different patch and pixel sizes.

Documented uses

  • Instantiate a TNT model with configurable image, token, depth, class-count, and dropout settings.
  • Feed image tensors to the model and receive class logits.
  • Use a pixel- and patch-level attention architecture as a component in image-classification experiments.
Before you choose

Requirements & limits

The source code is in a public GitHub repository under the MIT license. The README documents installation with pip.

  • The model exposes image size, patch and pixel dimensions, patch and pixel sizes, depth, class count, and attention/feed-forward dropout as configuration settings.
  • The documented interface returns class logits, not a hosted prediction service or a complete training pipeline.
  • The README example uses 256-pixel images, patch size 16, pixel size 4, and 1,000 output classes; these are example settings.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install the package and construct a TNT model

Set up the documented Python package and instantiate its configurable image-classification model.

Before you start

  • Python environment with PyTorch available
  • Package installation using pip

Steps

  1. Install the package with the command below.

    Documented command
    pip install transformer-in-transformer

  2. Import torch and TNT from transformer_in_transformer.

  3. Construct TNT with the desired image size, token dimensions, patch and pixel sizes, depth, class count, and dropout settings.

Expected outcome

A configured TNT model instance ready to receive image tensors.

Guide limitations

  • The documented setup and example do not specify a training or evaluation procedure.
Documentation 1 ↗
Source-based practical guide

Pass image tensors to TNT and obtain class logits

Use the documented model call to produce class logits from a batch of image tensors.

Before you start

  • An installed transformer-in-transformer package
  • PyTorch image tensor input matching the model's configured image size

Steps

  1. Create a TNT model and set its image_size and num_classes.

  2. Provide an image tensor batch, as in the documented example of shape (2, 3, 256, 256).

  3. Call the model with the tensor to obtain logits.

Expected outcome

A logits tensor; the example returns shape `(2, 1000)` for two images and 1,000 classes.

Guide limitations

  • The example demonstrates model input and output only; it does not document image preprocessing, training, or how to interpret or threshold logits.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
lucidrains/transformer-in-transformer · Official homepage title · retained in its source language
How we review sourcesSuggest a correction