OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

LVSM

LVSM is a Python and PyTorch implementation of the decoder-only architecture for large view synthesis, exposing model interfaces for image-and-ray tensors and camera parameters.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

LVSM — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use LVSM when you want a code-level starting point for large-view-synthesis experiments in PyTorch and can supply image, ray, or camera-parameter tensors.

Who it is for

  • Researchers experimenting with large view synthesis and decoder-only models.
  • Developers incorporating ray-based view synthesis into a PyTorch vision workflow.
  • Practitioners exploring self-supervised image-and-ray reconstruction with a masked autoencoder.

Documented uses

  • Train a model using input images, input rays, target rays, and target images.
  • Generate predicted target-view images from input images and target rays after training.
  • Pass camera intrinsic and extrinsic parameters through the CameraWrapper interface.
  • Experiment with masked-autoencoder reconstruction of images and rays.
Before you choose

Requirements & limits

The implementation is available as a public GitHub repository and lists an MIT license. Its README gives a pip installation command; it does not document account requirements.

  • The repository focuses on the decoder-only architecture rather than describing multiple LVSM variants.
  • The primary interface accepts image and ray tensors; CameraWrapper provides a documented route from camera parameters.
  • The README includes an optional MAE-based self-supervised workflow, but does not provide a complete data pipeline or training loop.
  • The prediction example assumes substantial training before inference.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Train and use LVSM with image and ray tensors

Set up LVSM, compute a loss from input and target images and rays, and call the model for target-view image prediction.

Before you start

  • Python with PyTorch available
  • Image and ray tensors matching the documented example shapes: input images (2, 4, 3, 256, 256), input rays (2, 4, 6, 256, 256), target images (2, 3, 256, 256), and target rays (2, 6, 256, 256)

Steps

  1. Install the package with the command below.

    Documented command
    pip install lvsm-pytorch

  2. Import torch and LVSM, then prepare input and target image and ray tensors.

  3. Create LVSM(dim=512, max_image_size=256, patch_size=32, depth=2).

  4. Call the model with input_images, input_rays, target_rays, and target_images to obtain a loss, then call loss.backward().

  5. After training, call the model with input_images, input_rays, and target_rays to obtain predicted images; the example expects shape (2, 3, 256, 256).

Expected outcome

The example demonstrates a model loss and gradient-backpropagation call, followed after training by a target-view image prediction call.

Guide limitations

  • The README says prediction follows substantial training; it does not provide a training loop or dataset procedure.
  • The repository focuses on the decoder-only architecture.
  • The documented tensor values are random examples, not evidence of image quality or successful real-data reconstruction.
Documentation 1 ↗
Source-based practical guide

Use LVSM with camera parameters through CameraWrapper

Set up the documented camera-parameter interface and compute a loss with camera inputs and images.

Before you start

  • Python with PyTorch available
  • Input and target camera-parameter, uniform-point, and image tensors in the shapes shown in the README

Steps

  1. Install the package with the command below.

    Documented command
    pip install lvsm-pytorch

  2. Import torch, LVSM, and CameraWrapper; prepare the input and target intrinsic rotations, extrinsic rotations, translations, uniform points, and images.

  3. Create LVSM(dim=512, max_image_size=256, patch_size=32, channels=4, depth=2) and wrap it with CameraWrapper.

  4. Call the wrapper with the documented input and target camera parameters, uniform points, and images to obtain a loss, then call loss.backward().

  5. After training, call the wrapper with the camera parameters and input images, omitting target_images, as shown for target-image prediction.

Expected outcome

The example shows camera-parameter inputs being passed through `CameraWrapper` for a loss and backpropagation, with a separate post-training prediction call.

Guide limitations

  • The README cautions that the camera-parameter approach may contain errors and invites issue reports or pull requests.
  • The prediction example is described as occurring after training; no training loop or dataset procedure is provided.
  • The shown inputs are random example tensors, not evidence of successful reconstruction from real camera data.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
lucidrains/lvsm-pytorch · Official homepage title · retained in its source language
How we review sourcesSuggest a correction