OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Flora

Flora is a Python research library that applies low-rank random projections to compress optimizer moments during neural-network training, aiming to reduce GPU memory needs for pre-training and fine-tuning.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Flora — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Flora when investigating lower optimizer-state memory in model pre-training or fine-tuning, especially when working with the PyTorch or JAX paths documented in the repository.

Who it is for

  • Researchers reproducing or extending the Flora paper's training experiments.
  • Practitioners adapting PyTorch or JAX model-training workflows to explore optimizer-state compression.

Documented uses

  • Replace an optimizer and accelerator in a documented PyTorch training pattern with Flora components.
  • Install the library with its PyTorch or JAX extra.
  • Invoke the repository's JAX paper-replication workflow.
  • Explore combining optimizer-state compression with activation checkpointing or layer-by-layer updates.
Before you choose

Requirements & limits

The repository is public and specifies the LGPL-3.0 license. It documents pip installation for PyTorch and JAX.

  • Flora compresses optimizer moments with low-rank random projections; the README relates memory use to projection rank and parameter dimension.
  • The PyTorch example replaces both the optimizer and the Accelerate accelerator, while the README describes similar arguments for JAX.
  • Flora can be paired with activation checkpointing and layer-by-layer updates such as LOMO, according to the repository.
  • The repository contains experimental code for the paper, so readers should assess it in their own training workflow.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Flora for PyTorch or JAX

Install the Flora library with the framework-specific extra documented in the repository.

Before you start

  • A Python environment with pip.

Steps

  1. For PyTorch, run the command below.

    Documented command
    pip install 'flora-opt[torch]'

  2. For JAX, run the command below.

    Documented command
    pip install 'flora-opt[jax]'

Expected outcome

The Flora package is installed with the selected framework extra.

Guide limitations

  • The installation instructions do not specify framework versions or a hardware configuration.
Documentation 1 ↗
Source-based practical guide

Set up Flora for reproducing the JAX paper experiments

Install the listed requirements and invoke the repository's replication script.

Before you start

  • A Python environment able to install the packages listed in examples/flax/requirements.txt.
  • The Flora repository, including replicate.sh.

Steps

  1. Install the dependencies with the command below.

    Documented command
    pip install -r examples/flax/requirements.txt

  2. Invoke the replication script with the command below.

    Documented command
    sh replicate.sh

Expected outcome

The documented JAX paper-replication workflow is invoked; individual experiments can be selected through scripts listed in `replicate.sh`.

Guide limitations

  • The README does not specify hardware requirements or expected runtime.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
BorealisAI/flora-opt · Official homepage title · retained in its source language
How we review sourcesSuggest a correction