OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Lamp — LaMP-Benchmark

LaMP is a research benchmark and Python codebase for training and evaluating language models that produce personalized outputs. Its seven tasks include classification and text generation, with multiple entries for each user profile; the project also describes retrieval augmentation and per-user LoRA fine-tuning workflows.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Lamp — LaMP-Benchmark — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use LaMP when an experiment needs task-based evaluation of personalized model outputs, or when comparing profile retrieval and per-user fine-tuning approaches.

Who it is for

  • Researchers evaluating language-model personalization.
  • Developers comparing profile retrieval, personalized prompting and per-user fine-tuning.
  • Teams studying personalized classification, text generation or information retrieval.

Documented uses

  • Evaluate model outputs across seven personalized tasks.
  • Compare outputs that use profile context with those that do not.
  • Prepare benchmark data and evaluate predictions on development or test sets.
  • Explore retrieval-augmented prompting and per-user LoRA experiments.
Before you choose

Requirements & limits

The public GitHub repository is accessible. The README states that code and data-creation methods are licensed under CC BY-NC-SA 4.0; dataset terms are separate. The Avocado dataset is not publicly accessible.

  • The benchmark comprises seven tasks: three classification and four text-generation tasks.
  • The repository documents both retrieval augmentation and per-user LoRA fine-tuning as personalization approaches.
  • The Avocado email-subject task requires access to a dataset the repository says is not publicly accessible.
  • The README describes separate instructions for test-set evaluation and scripts for development-set evaluation.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare the documented PEFT environment

Create the Python virtual environment and install the requirements used by LaMP’s per-user LoRA workflow.

Before you start

  • Python 3
  • A shell environment
  • The LaMP repository, including PEFT/requirements.txt

Steps

  1. Create an environment with the command below.

    Documented command
    python3 -m venv peft_venv

  2. Activate it with source peft_venv/bin/activate.

  3. Install the listed dependencies with the command below.

    Documented command
    pip install -r PEFT/requirements.txt

Expected outcome

The documented PEFT virtual environment has been created and its listed requirements installed.

Guide limitations

  • These setup steps do not train a model; training requires sorted training data and additional task and output arguments.
  • The repository states that its code and data-creation methods use CC BY-NC-SA 4.0; dataset terms are separate.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
LaMP-Benchmark/LaMP · Official homepage title · retained in its source language
How we review sourcesSuggest a correction