OpenViSTAI · Agents · Blockchain · Crypto
Menu
AI Data & Analytics

Curator

Curator is an open-source Python toolkit for building repeatable pipelines that load, filter, deduplicate and transform text, image, video and audio datasets for AI training and fine-tuning.

Independent OpenViST project profile, based on linked sources.

Curator — OpenViST project profile in AI Data & Analytics.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Curator when dataset preparation involves repeatable multi-stage pipelines or data-heavy curation across text, image, video or audio. The project documents both a CPU text setup and GPU or distributed execution paths.

Who it is for

  • Machine-learning engineers preparing datasets for model training.
  • Data teams building repeatable curation and preprocessing workflows.

Documented uses

  • Filter, classify, transform or deduplicate text datasets.
  • Apply image operations such as aesthetic filtering, NSFW detection and embedding generation.
  • Prepare video with scene detection, clip extraction and motion filtering.
  • Build audio datasets using transcription, quality assessment or word-error-rate filtering.
Before you choose

Requirements & limits

Curator is published as an open-source project under the Apache-2.0 license. The repository documents installation through uv and a CPU text extra.

  • The documented workload spans text, image, video and audio, but available operations differ by modality.
  • The CPU text path is distinct from the CUDA 12 GPU quickstart, which has specific Linux, driver, CUDA, memory and network prerequisites.
  • CUDA 12 installation uses uv pip with a project-provided dependency override; standard pip install is not supported for that extra.
  • Audio and video workflows that use FFmpeg may require a locally built image that includes the system dependency for every worker.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Curator and check the CPU text setup

Create a Python environment, install the CPU text extra and check that Curator imports and reports its version.

Before you start

  • The uv installer command documented by the project.
  • A shell and Python environment.

Steps

  1. Install uv using the command shown in Curator's README.

  2. Create and activate a virtual environment with uv.

  3. Install the nemo-curator[text_cpu] extra using uv pip.

  4. Run the documented Python import and version check.

Expected outcome

The environment has the CPU text extra installed and the import check prints Curator's version.

Guide limitations

  • This setup check does not run a data-curation workflow.
Source-based practical guide

Set up the CUDA 12 text quickstart

Install the CUDA 12 text extra and run Curator's bundled sentiment-classification quickstart.

Before you start

  • Linux x86_64.
  • CUDA 12 toolkit and an NVIDIA driver that supports CUDA 12.
  • Approximately 16 GB of GPU memory.
  • Network access to Hugging Face.
  • uv.

Steps

  1. Create and activate a virtual environment with uv.

  2. Download the text_cuda12 dependency override file from the Curator repository.

  3. Install nemo-curator[text_cuda12] using the documented uv pip command with the override, CUDA 12.9 PyTorch backend and vLLM package index.

  4. Run python tutorials/quickstart.py.

Expected outcome

The bundled quickstart starts Ray, downloads a Hugging Face model and runs a GPU sentiment-classification pipeline.

Guide limitations

  • The project says standard pip install is not supported for text_cuda12 because vLLM and RAPIDS declare incompatible Numba requirements.
  • The override is also required for uv pip installs that include text_cuda12, including nemo-curator[all].
  • The stated prerequisites apply to this CUDA 12 quickstart, not every Curator workflow.
Evidence and transparency

Sources & review

Source record details
Source availability
Active
Sources checked
11 October 2026
Original website title
NVIDIA-NeMo/Curator · Official homepage title · retained in its source language