OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Agent Evaluation & Observability

Airtbench Code

Airtbench Code is the implementation repository for an autonomous AI red-teaming agent that evaluates language models by attempting AI/ML capture-the-flag challenges on Dreadnode’s Strikes platform. Its harness uses challenge notebooks and Python execution in network-local containers to run tasks and submit a flag when found.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Airtbench Code — OpenViST project profile in Agent Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you need to evaluate an agent’s ability to work through AI/ML CTF challenges and have access to the Dreadnode Strikes platform those runs require.

Who it is for

  • Researchers evaluating autonomous AI red-team capabilities in language models.
  • Teams using Dreadnode Strikes challenges to assess agent performance on AI/ML CTF tasks.

Documented uses

  • Run an autonomous agent against selected AI/ML CTF challenges.
  • Filter challenge runs to LLM-based challenges.
  • Compare language models using a standardized red-team challenge workflow.
Before you choose

Requirements & limits

The README says running the code requires access to Dreadnode Strikes. It identifies the repository as public and licensed under Apache-2.0.

  • The harness depends on Dreadnode Strikes for challenge access and execution.
  • Runs use Python in network-local containers, which the README says are isolated from each other.
  • The CLI accepts model, challenge, organization and optional workspace parameters, and can filter for LLM-based challenges.
  • The README describes stopping conditions including a found flag, an error, giving up, or reaching the maximum step count.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the AIRTBench virtual environment

Create the project’s virtual environment and install its dependencies using uv.

Before you start

  • The AIRTBench-Code repository
  • uv installed

Steps

  1. Open a terminal in the repository directory.

  2. Run the command below.

    Documented command
    uv sync

Expected outcome

The project environment is set up according to the repository’s uv configuration.

Guide limitations

  • Running evaluations requires access to the Dreadnode Strikes platform; setup alone does not run an evaluation.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
12 October 2026
Original website title
dreadnode/AIRTBench-Code · Official homepage title · retained in its source language
How we review sourcesSuggest a correction