OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Laser

Laser is a Python research-code repository for layer-selective rank reduction in language models. It replaces selected transformer weight matrices with low-rank approximations computed using SVD, with transformations specified by layer, parameter type, and retained rank. The repository contains code accompanying research that evaluates this intervention on language models and question-answering tasks.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Laser — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Laser when you want research code for experimenting with selective low-rank changes to transformer weights and evaluating their effects on language-model tasks.

Who it is for

  • Researchers reproducing or exploring experiments on language-model reasoning and rank reduction.
  • Python practitioners studying transformer weight interventions and model-compression workflows.

Documented uses

  • Apply a low-rank approximation to a selected transformer weight matrix.
  • Specify interventions by layer, weight type, and retained-rank fraction, and compose transformations.
  • Run documented model-and-dataset experiment scripts and record accuracy and log-loss.
  • Explore the repository’s research results and contribute a reported result through a pull request.
Before you choose

Requirements & limits

The repository is public and lists an MIT license. The README provides a pip requirements-file installation command.

  • Experiment setups are separate files, so identify the relevant model and dataset script in the codebase.
  • The documented rate parameter is legacy: 0 retains all components and 10 removes all components, with retained fraction ρ = 1 − 0.1 × rate.
  • Adding a new language model requires adapting model mappings and the Laser wrapper.
  • Some experiments require separate validation-set hyperparameter selection before test-set evaluation.
  • The README characterizes the release as early development and notes that a comprehensive tutorial was to be provided later.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Laser’s experiment dependencies

Prepare the repository’s Python dependencies before consulting its experiment scripts.

Before you start

  • A Python environment with pip3.
  • The repository’s requirements.txt file.
  • PyTorch, Hugging Face Datasets, and Transformers are identified as principal dependencies.

Steps

  1. From the repository, install the listed dependencies with the command below.

    Documented command
    pip3 install -r requirements.txt

  2. For CounterFact experiments, optionally run the command below; the README says other datasets are available on Hugging Face.

    Documented command
    python scripts/get_counterfact.py

Expected outcome

The documented project dependencies are installed, with CounterFact downloaded if its optional script is used.

Guide limitations

  • This covers dependency installation, not successful execution of an experiment.
  • The README describes the code as an early development release and says individual setups are organized in separate files.
Documentation 1 ↗
Source-based practical guide

Run the documented GPT-J and FEVER example

Invoke the repository’s sample single-intervention experiment for GPT-J on FEVER.

Before you start

  • Install the project dependencies using the repository’s requirements file.
  • Use the repository’s experiment files and dataset setup.
  • The README’s example uses intervention_gptj_fever.py and parameters fc_in, rate 9.9, and layer number 26.

Steps

  1. From the repository’s documented experiment context, run the command below.

    Documented command
    python3 intervention_gptj_fever.py --lname fc_in --rate 9.9 --lnum 26

  2. Interpret lnum as the layer, lname as the parameter type, and rate as related to retained-rank fraction by ρ = 1 − 0.1 × rate.

Expected outcome

The command invokes the documented single-LASER GPT-J experiment on FEVER; the README says experiments save accuracy and log-loss for each datapoint.

Guide limitations

  • The README does not provide a complete tutorial or all command-line details.
  • Some experiments require separate validation-set hyperparameter selection and test-set evaluation.
  • The rate argument is described as a legacy parameter.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
pratyushasharma/laser · Official homepage title · retained in its source language
How we review sourcesSuggest a correction