OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Curlora

Curlora is a public, MIT-licensed research-code repository implementing CURLoRA, a method that combines CUR matrix decomposition and LoRA for continual fine-tuning of large language models. It includes adapter classes, helper functions, research notebooks and a paper describing the approach.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Curlora — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use the repository when you want research code and examples for evaluating CUR-based LoRA adapters in continual LLM fine-tuning, rather than a hosted model service.

Who it is for

  • Researchers studying continual learning and catastrophic forgetting in large language models.
  • Practitioners experimenting with parameter-efficient fine-tuning across successive tasks.
  • Developers adapting LoRA-style attention layers in supported model architectures.

Documented uses

  • Apply CURLoRA layers to a language model's attention projections.
  • Explore the included Mistral 7B and GPT-2 Large fine-tuning experiments.
  • Compare CURLoRA and LoRA in experiments focused on retaining performance across tasks.
Before you choose

Requirements & limits

The code is publicly available under the MIT License.

  • The repository includes corresponding LoRA notebooks for comparison with its CURLoRA experiments.
  • The documented quick start uses a full, unquantized model on CUDA.
  • Some implementation values, including target layers, rank, alpha and device, are hardcoded.
  • The README reports better catastrophic-forgetting mitigation and accuracy and perplexity results than standard LoRA in its experiments; these are project-reported findings.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install requirements and apply CURLoRA to GPT-2 Large

Set up the repository requirements and follow the documented example for adding CURLoRA layers to GPT-2 Large attention.

Before you start

  • The CURLoRA repository and its code directory.
  • Python package installation with pip3.
  • The GPT-2 Large model and a CUDA device, as used in the example.

Steps

  1. Install the listed dependencies with the command below.

    Documented command
    pip3 install -r code/requirements.txt

  2. Load GPT-2 Large with AutoModelForCausalLM.from_pretrained("gpt2-large") and move it to CUDA.

  3. Freeze the existing model parameters, then replace the GPT-2 attention c_attn layer with LinearWithCURLoRA using the example rank 24 and alpha 1.

  4. Move the model to CUDA again after applying the CURLoRA layers.

Expected outcome

The example leaves GPT-2 Large with CURLoRA layers applied to its attention projections, ready for fine-tuning or inference according to the README.

Guide limitations

  • This example is GPT-2 Large-specific; other architectures may require identifying and replacing their attention projection layers.
  • Some layer selections and settings are hardcoded in the implementation.
  • The documented path loads the full model without quantization.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
MNoorFawi/curlora · Official homepage title · retained in its source language
How we review sourcesSuggest a correction