OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Evaluation & Observability

Arc Agi

ARC-AGI-1 is a visual abstraction and reasoning benchmark comprising JSON grid tasks and a browser interface for people to solve and check them manually.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Arc Agi — OpenViST project profile in AI Evaluation & Observability.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use the training split while developing a solver and reserve the evaluation split for final assessment. Use the included browser interface when you want to inspect demonstrations and solve a task manually.

Who it is for

  • Researchers evaluating AI systems on visual abstraction and reasoning
  • Developers prototyping algorithms or program-synthesis approaches with ARC tasks
  • People studying or attempting the benchmark's grid puzzles by hand

Documented uses

  • Load ARC JSON tasks into a data workflow
  • Prototype an algorithm with training demonstrations
  • Evaluate a final algorithm on a separate task split
  • Construct and check grid answers manually in the browser interface
Before you choose

Requirements & limits

The repository is public and licensed under Apache-2.0. Its instructions describe opening the included HTML testing interface in a browser and selecting a task JSON file.

  • Task files are JSON objects with training demonstrations and test inputs; grids use integers from 0 to 9.
  • The repository separates 400 training tasks from 400 evaluation tasks and advises against using evaluation data during development.
  • The human interface supports manual construction and answer checking, but does not enforce the three-attempt benchmark rule.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Load and solve an ARC task in the browser interface

Use the included interface to construct and check an output grid for a task.

Before you start

  • A task JSON file from the repository
  • A web browser; Chrome is recommended

Steps

  1. Open apps/testing_interface.html in a browser.

  2. Select a task JSON file when prompted.

  3. Use the demonstrations and test input to construct an output grid with the grid and symbol controls.

  4. Submit the grid to check the answer; use Next test input if another test input is available.

Expected outcome

The interface checks the submitted grid against the task answer and allows progression to another test input when available.

Guide limitations

  • The interface does not enforce the benchmark's three-trials-per-test-input rule.
  • A task is counted as solved only when every test input has an exact output, including dimensions.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
13 October 2026
Original website title
fchollet/ARC-AGI · Official homepage title · retained in its source language
How we review sourcesSuggest a correction

Related options

ARC-AGI-2 — The repository points to a separate ARC-AGI-2 repository.