OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Data & Analytics

Data Whisperer

Data Whisperer is a Python research implementation for scoring and selecting examples from a dataset before task-specific LLM fine-tuning. It evaluates model answers to sampled queries and uses attention-based weighting of demonstration-example scores to support data ranking and coreset construction.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Data Whisperer — OpenViST project profile in AI Data & Analytics.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want a configurable research workflow to score and prune task-specific LLM training data before fine-tuning.

Who it is for

  • Researchers studying data selection for language-model fine-tuning
  • Practitioners preparing task-specific fine-tuning datasets

Documented uses

  • Score examples using few-shot in-context evaluation against ground-truth answers.
  • Rank dataset points and select a coreset for a fine-tuning workflow.
  • Compare data-selection experiments across the documented datasets, metrics and model configurations.
Before you choose

Requirements & limits

  • The README specifies dataset-to-metric pairings: rouge-L for bioinstruct and dialogsum, exact_match for gsm8k.
  • Model configuration includes a model type, model name and local model path.
  • The workflow generates scored data for later coreset selection; the README identifies Llama-Factory as the approach followed for fine-tuning experiments.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up and run a documented data-selection experiment

Configure a supported dataset, metric and model, then generate a scored dataset for coreset selection.

Before you start

  • Python environment capable of installing the repository's requirements
  • A local model path for a supported Llama, Qwen or Mistral model
  • A supported dataset and its corresponding evaluation metric

Steps

  1. Clone the repository, enter its directory and install dependencies with the command below.

    Documented command
    pip install -r requirements.txt

  2. Edit scripts/run.sh to set the dataset, matching metric, model type and name, local model path, demonstration and query counts, and parallel size.

  3. Run the command below.

    Documented command
    bash scripts/run.sh

  4. Use the scored dataset generated in results/pruning to select points for a coreset.

Expected outcome

A scored dataset in `results/pruning` that can be used to construct a coreset.

Guide limitations

  • The listed datasets are bioinstruct, gsm8k and dialogsum; rouge-L is listed for bioinstruct and dialogsum, while exact_match is listed for gsm8k.
  • The README does not demonstrate arbitrary datasets or scoring metrics.
  • A local model path is a configured parameter; hardware requirements are not specified.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
gszfwsb/Data-Whisperer · Official homepage title · retained in its source language
How we review sourcesSuggest a correction