OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Spline Based Transformer

Spline Based Transformer is a Python transformer autoencoder that represents latent tokens as high-dimensional spline control points and includes an image-autoencoder wrapper.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Spline Based Transformer — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when a Python project needs a transformer autoencoder with spline-based latent control points, or when exploring how edits to those points affect reconstructed sequences or images.

Who it is for

  • Python developers experimenting with transformer autoencoders for sequential data.
  • Researchers exploring latent spline representations and reconstruction.
  • Developers building image-autoencoder experiments with the provided wrapper.

Documented uses

  • Train a model to reconstruct sequential tensor data.
  • Encode input data into spline control points and decode a reconstruction.
  • Use the image wrapper to train and reconstruct image tensors.
  • Modify latent control points and decode a controlled sequence or image reconstruction.
Before you choose

Requirements & limits

The repository is public and specifies the MIT license. The README documents installation with pip; its examples use Python and PyTorch.

  • The representation is distinctive: latent tokens serve as high-dimensional spline control points.
  • Sequence and image paths are both shown, with image support provided through a wrapper.
  • The documented examples require training before reconstruction and use tensor inputs.
  • The README illustrates continuity-preserving edits as an intended property, not a quantified guarantee.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Train and decode sequence data with spline latent control points

Train the transformer autoencoder on sequence data, then reconstruct it and decode a reconstruction from modified latent control points.

Before you start

  • Python package spline-based-transformer installed with pip install spline-based-transformer.
  • PyTorch available.
  • Sequence data represented as a tensor; the example uses shape (1, 1024, 512).

Steps

  1. Import torch and SplineBasedTransformer, then create a model with dim=512, enc_depth=6, and dec_depth=6.

  2. Provide sequence data and calculate the training loss with model(data, return_loss=True); call loss.backward() and train the model.

  3. After training, obtain recon, control_points = model(data, return_latents=True).

  4. Modify the control points and decode with model.decode_from_latents(control_points, num_times=1024). The README example adds 1 to the control points.

Expected outcome

The model returns a sequence reconstruction and latent control points; decoding the modified control points produces a controlled reconstruction with the same shape as the input in the documented example.

Guide limitations

  • The README marks reconstruction as occurring after training but does not specify a training loop, optimizer, or duration.
  • Continuity preservation from control-point edits is described as expected, not guaranteed.
Documentation 1 ↗
Source-based practical guide

Train and decode images with the image autoencoder wrapper

Use the image wrapper to train on image tensors, obtain latent control points, and decode after modifying them.

Before you start

  • Python package spline-based-transformer installed with pip install spline-based-transformer.
  • PyTorch available.
  • Images represented as tensors; the example uses shape (2, 3, 256, 256).

Steps

  1. Import torch, SplineBasedTransformer, and ImageAutoencoderWrapper.

  2. Create an ImageAutoencoderWrapper with image_size=256, patch_size=32, and a SplineBasedTransformer configured with dim=512, enc_depth=6, and dec_depth=6.

  3. Calculate loss = model(images, return_loss=True), call loss.backward(), and train the model.

  4. After training, obtain recon_images, control_points = model(images, return_latents=True), modify the control points, and call model.decode_from_latents(control_points). The README example adds 1 to the control points.

Expected outcome

The wrapper returns reconstructed images and latent control points; decoding modified control points produces controlled reconstructed images with the input shape in the documented example.

Guide limitations

  • The README does not specify a training loop, optimizer, or duration.
  • The example documents the wrapper configuration and tensor dimensions but does not establish broader image-size or input-format support.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
lucidrains/spline-based-transformer · Official homepage title · retained in its source language
How we review sourcesSuggest a correction