OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Agent Evaluation & Observability

Clawbench

Clawbench is an open-source Python benchmark and evaluation toolkit for comparing browser AI agents on daily tasks. Its project resources include a dataset, agent traces, a leaderboard, a Python package and a repository-based workflow with container-isolated harnesses.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Clawbench — OpenViST project profile in Agent Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Clawbench when you need benchmark resources and tooling to assess or compare browser agents on daily tasks, rather than a browser agent that performs tasks for end users.

Who it is for

  • Researchers comparing browser-agent performance on everyday tasks.
  • Developers evaluating changes to browser agents over time.
  • Teams looking for an open-source benchmark and Python tooling for browser-agent evaluation.

Documented uses

  • Evaluate a browser AI agent against daily-task benchmark material.
  • Compare browser-agent evaluation results.
  • Track changes in browser-agent performance.
  • Inspect benchmark data and agent traces.
Before you choose

Requirements & limits

The project is open source under the Apache-2.0 license. The README documents package installation through uv, pipx or pip, and a repository workflow requiring Python 3.11 or later, uv and Docker or Podman.

  • The documented focus is browser agents and daily tasks; individual benchmark task details are not included in the cited material.
  • Local repository use requires Python 3.11 or later, uv and Docker or Podman.
  • The project separates its dataset and traces from its leaderboard and evaluation tooling.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install the Clawbench evaluation package

Install the published clawbench-eval package and identify its command-line tools.

Before you start

  • Python 3.11 or later
  • One of uv, pipx or Python pip

Steps

  1. Install with the command below, pipx install clawbench-eval or the command below.

    Documented command
    uv tool install clawbench-eval
    Documented command
    python -m pip install clawbench-eval

  2. Review the installed command names listed in the README: clawbench, clawbench-run, clawbench-batch, clawbench-rescore, clawbench-analyze, clawbench-reproduce and clawbench-harbor-adapt.

Expected outcome

The clawbench-eval package is installed, and its documented command names are available for further use.

Guide limitations

  • The cited installation instructions do not explain how to configure or run an evaluation through these commands.
Documentation 1 ↗
Source-based practical guide

Prepare the repository quick start

Clone the repository and prepare its documented local-run prerequisites.

Before you start

  • Python 3.11 or later
  • uv
  • Docker or Podman

Steps

  1. Install Python 3.11 or later and uv.

  2. Install Docker or Podman as the container engine.

  3. Clone the Clawbench repository and change into its directory.

  4. Run ./run.sh from the repository directory.

Expected outcome

The documented repository quick-start command is invoked with the stated prerequisites.

Guide limitations

  • The available README material does not specify the evaluation task selection, agent configuration or expected run output.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
14 October 2026
Original website title
TIGER-AI-Lab/ClawBench · Official homepage title · retained in its source language
How we review sourcesSuggest a correction