Practical overview
When it fits
Use Nola when you want to investigate a LoRA variant that controls trainable parameters through coefficients over random bases, or reproduce the repository’s documented language-model and vision-model experiments.
Who it is for
- Researchers studying parameter-efficient fine-tuning and adapter compression.
- Practitioners experimenting with Nola workflows for LLaMA-2, GPT-2 or Vision Transformers.
Documented uses
- Fine-tune language models using the repository’s Nola-modified LoRA or QLoRA workflows.
- Run Nola fine-tuning experiments on Vision Transformers.
- Compare Nola with baseline approaches in the documented vision workflow, including LoRA, linear and full training.
Requirements & limits
The public GitHub repository is released under the MIT license.
- The method optimizes coefficients while keeping the random basis fixed and regenerating it as needed, trading additional computation for reduced need to retain the basis in GPU memory.
- The repository provides distinct code paths: QLoRA-derived LLaMA-2 fine-tuning, LoRA-derived GPT-2 fine-tuning and a LoRA-ViT-based vision workflow.
- The vision example includes multi-seed runs and dataset-path configuration; its README notes high variance in low-shot settings.
- The README names PyTorch as the experiments’ framework and timm as a requirement for the vision workflow.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Fine-tune LLaMA-2 with the Nola workflow
Run the repository’s documented LLaMA-2 fine-tuning script with a downloaded model.
Before you start
- A Python virtual environment with the dependencies in the repository’s llama/requirements.txt installed.
- A LLaMA model downloaded from Hugging Face, with its path available for the script.
- The repository’s Alpaca example dataset.
Steps
From the repository, enter the llama directory and install its listed requirements.
Download a LLaMA model and set --model_name_or_path to its local path.
Run bash scripts/finetune_llama2_guanaco_13b_nola.sh.
Expected outcome
The repository’s example LLaMA-2 Nola fine-tuning workflow is invoked.
Guide limitations
- The README identifies the script as an example for LLaMA-2 13B and Alpaca.
- The project says LLaMA-3 can also be used but reports no paper experiments with it.
Run the documented Vision Transformer experiments
Start the repository’s multi-run Nola fine-tuning workflow for vision models.
Before you start
- PyTorch and the timm library; the README reports using PyTorch 2.0.1 and timm 0.4.12.
- The fine-tuning dataset downloaded, with dataset path variables set in the script.
- A GPU setup; the README assumes a four-GPU server, and provides a sequential option for one GPU.
Steps
Install timm and prepare the dataset, then set its paths in the bash script.
From the repository, enter the vit directory and run bash run.sh.
For a single GPU, remove the ampersand after the script’s last argument and set the gpu argument to 0.
Expected outcome
The script runs the configured vision experiment; the README describes 12 runs for its stated setting.
Guide limitations
- The README reports high variance for the 5-shot and 10-shot settings.
- The 12-run description reflects four dataset-sampling seeds and three network-initialization seeds.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 16 October 2026
- Original website title
- UCDvision/NOLA · Official homepage title · retained in its source language
Related options
LoRA — Nola re-parameterizes LoRA’s low-rank matrices using random basis combinations; the repository also includes LoRA baseline options in its vision workflow.
Explore project →QLoRA — The repository’s LLaMA-2 code modifies QLoRA and adds Nola to the LoRA model.