OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Vefx Bench

Vefx Bench is an open-source research benchmark for evaluating text-driven video editing and visual-effects systems, paired with VEFX-Reward models that score edits for instruction following, render quality and edit exclusivity.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Vefx Bench — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when evaluating text-directed video edits and you need separate measures for prompt compliance, visual and temporal quality, and whether changes stayed within the intended regions.

Who it is for

  • Researchers comparing video-editing and visual-effects systems.
  • AI teams measuring whether generated edits follow prompts, preserve visual quality and avoid unintended changes.

Documented uses

  • Score an edited video against its source video and editing instruction.
  • Compare editing systems using three separate quality dimensions and an aggregate score.
  • Evaluate individual edits or process batches using the documented Python API and repository scripts.
Before you choose

Requirements & limits

The code repository specifies an Apache-2.0 license. The documented local setup requires Python 3.10 or later and a CUDA GPU.

  • The reward model expects the original video, edited video and editing instruction as inputs.
  • The three reported dimensions expose different failure modes; the overall value should not replace examining them separately.
  • The repository estimates about 10 GB VRAM for the 4B model and about 65 GB for the 32B model in bfloat16.
  • The benchmark is an evaluation toolkit, not a system for generating or editing video.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Score a video edit with VEFX-Reward

Use the documented Python interface to obtain evaluation scores for an original and edited video paired with an editing instruction.

Before you start

  • Python 3.10 or later and a CUDA GPU.
  • PyTorch installed for the matching CUDA version, the repository requirements, and VEFX-Bench installed as a package.
  • The original video, edited video and corresponding text instruction.
  • About 10 GB VRAM for VEFX-Reward-4B or about 65 GB VRAM for VEFX-Reward-32B in bfloat16.

Steps

  1. Create and activate a Python 3.10 environment, install PyTorch for the matching CUDA version, install the repository requirements and install the package with the command below.

    Documented command
    pip install -e .

  2. Import VEFXReward from vefx_reward and initialize it with the chosen model ID and device="cuda".

  3. Call model.score(original_video=..., edited_video=..., instruction=...) with the two video paths and prompt.

  4. Read the returned IF, RQ, EE and Overall values.

Expected outcome

A score result for the submitted edit, including separate instruction-following, render-quality and edit-exclusivity values.

Guide limitations

  • The guide describes evaluation with the reward model; it does not create or edit videos.
  • The 32B model has a substantially higher stated VRAM requirement than the 4B model.
  • The repository instructs users to match the PyTorch CUDA build to their driver.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
8 October 2026
Original website title
Visko-Platform/VEFX-Bench · Official homepage title · retained in its source language
How we review sourcesSuggest a correction