OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Agent Evaluation & Observability

Workspace Bench

Workspace Bench is an open-source benchmark and dataset for evaluating AI agents on realistic workspace tasks that involve large-scale dependencies among heterogeneous files.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Workspace Bench — OpenViST project profile in Agent Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Workspace Bench when an evaluation must test how an agent handles dependencies across many files, rather than isolated prompts or single-file tasks.

Who it is for

  • AI agent developers building regression evaluations
  • Researchers studying workspace-aware planning and agent performance
  • Teams comparing agents on multi-file repository or document-workspace tasks

Documented uses

  • Evaluate an agent on tasks requiring work across related files
  • Compare agent performance using explicit dependency graphs and fine-grained rubrics
  • Create a smaller evaluation run with the Workspace-Bench-Lite subset
  • Develop workspace-aware agent planning methods
Before you choose

Requirements & limits

The repository is public, MIT-licensed and provides CLI and self-hosted workflows. The documented evaluation setup requires Docker, Python 3 and agent credentials; rubric judging also requires an Anthropic-compatible judge endpoint.

  • The full dataset contains 388 tasks, 20,476 files and workspaces up to 20GB, while Lite contains 100 tasks.
  • Tasks cover five worker profiles and 74 file types, supporting heterogeneous workspace evaluation.
  • Each task is paired with an explicit file dependency graph and the dataset includes 7,399 fine-grained rubrics.
  • The documented runner uses Docker, with a task-isolated launcher and a stated default profile of 2 CPUs, 8 GiB memory, 512 PIDs and 20 GiB writable task storage.
  • The supplied material does not document a general integration interface or compatibility with a specific external agent runner.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up a Workspace-Bench-Lite smoke evaluation

Download the English Lite dataset, build the Docker environment and run one documented Codex smoke evaluation.

Before you start

  • Docker
  • Python 3
  • API credentials for the agent being evaluated
  • An Anthropic-compatible API endpoint for the judge model
  • The Workspace-Bench repository

Steps

  1. From the repository's evaluation directory, copy .env.example to .env and fill the required agent and, when using rubric judging, judge settings.

  2. Download the English Lite task set and workspace files with the documented download_hf_assets.py command.

  3. Build the Docker environment with the documented Docker Compose build and bootstrap commands.

  4. Run the documented one-task Codex smoke evaluation using the smoke dataset.

  5. Check the generated agent-runner report with assert_agent_runner_report.py.

Expected outcome

A one-task smoke evaluation produces an agent-runner report that can be checked with the supplied report assertion script.

Guide limitations

  • The evidence documents a smoke evaluation workflow, not a result for a particular agent beyond the example command.
  • The default smoke command requires the documented Kimi K2.5 base URL and API key settings.
  • The project does not document compatibility with every agent runner or model.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
4 October 2026
Original website title
OpenDataBox/Workspace-Bench · Official homepage title · retained in its source language
How we review sourcesSuggest a correction