OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

T2V Metrics

T2V Metrics is an Apache-2.0 Python library that applies VQAScore to evaluate text-to-image, text-to-video, and text-to-3D models against text prompts.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

T2V Metrics — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you have visual outputs and text prompts and want to compute VQAScore measurements from Python as part of model evaluation.

Who it is for

  • Researchers evaluating generative visual models
  • Developers adding prompt-based visual scoring to Python research workflows

Documented uses

  • Score generated images against one or more prompts.
  • Compare video outputs with text descriptions, including pairwise comparisons.
  • Add an automated VQAScore measurement to a model evaluation workflow.
Before you choose

Requirements & limits

  • The README shows image and video scoring calls; it identifies text-to-3D evaluation as supported but the provided usage examples do not show a 3D scoring call.
  • GPT-4o and GPT-4.1 scoring requires an API key.
  • Gemini scoring requires a Google Cloud Vertex AI project; a standard Gemini Developer API key does not support scoring.
  • The default PyTorch SDPA attention path requires no extra installation; optional Flash Attention 2 has CUDA, PyTorch, and GLIBC requirements.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install T2V Metrics in a Conda environment

Set up the package using the README's documented repository installation path.

Before you start

  • Conda
  • Python 3.10
  • FFmpeg installed through conda-forge

Steps

  1. Clone the t2v_metrics repository and change into its directory.

  2. Create and activate a Conda environment with Python 3.10.

  3. Install pip and FFmpeg from conda-forge in that environment.

  4. Install the repository in editable mode with pip.

Expected outcome

The repository's documented installation steps are complete in the Conda environment.

Guide limitations

  • This covers installation only, not a scoring run.
  • The README also lists pip installation as an alternative.
Documentation 1 ↗
Source-based practical guide

Score images against text prompts in Python

Use the documented image-scoring example to obtain prompt comparison scores.

Before you start

  • T2V Metrics installed in a Python environment
  • Image file paths and corresponding text prompts
  • A VQAScore model, such as the documented qwen2.5-vl-7b example

Steps

  1. Import t2v_metrics in Python.

  2. Initialize VQAScore with the selected model.

  3. Pass image paths in the images argument and prompts in the texts argument.

  4. For multiple comparisons, pass lists of images and prompts to obtain pairwise scores.

Expected outcome

The scorer returns a score for the supplied image–prompt pair or a pairwise score tensor for lists.

Guide limitations

  • The example documents image scoring; it does not define a universal interpretation or threshold for scores.
  • Model-specific requirements may apply; API-backed scorers require provider setup.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
linzhiqiu/t2v_metrics · Official homepage title · retained in its source language
How we review sourcesSuggest a correction