Practical overview
When it fits
Use the training split while developing a solver and reserve the evaluation split for final assessment. Use the included browser interface when you want to inspect demonstrations and solve a task manually.
Who it is for
- Researchers evaluating AI systems on visual abstraction and reasoning
- Developers prototyping algorithms or program-synthesis approaches with ARC tasks
- People studying or attempting the benchmark's grid puzzles by hand
Documented uses
- Load ARC JSON tasks into a data workflow
- Prototype an algorithm with training demonstrations
- Evaluate a final algorithm on a separate task split
- Construct and check grid answers manually in the browser interface
Requirements & limits
The repository is public and licensed under Apache-2.0. Its instructions describe opening the included HTML testing interface in a browser and selecting a task JSON file.
- Task files are JSON objects with training demonstrations and test inputs; grids use integers from 0 to 9.
- The repository separates 400 training tasks from 400 evaluation tasks and advises against using evaluation data during development.
- The human interface supports manual construction and answer checking, but does not enforce the three-attempt benchmark rule.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Load and solve an ARC task in the browser interface
Use the included interface to construct and check an output grid for a task.
Before you start
- A task JSON file from the repository
- A web browser; Chrome is recommended
Steps
Open apps/testing_interface.html in a browser.
Select a task JSON file when prompted.
Use the demonstrations and test input to construct an output grid with the grid and symbol controls.
Submit the grid to check the answer; use Next test input if another test input is available.
Expected outcome
The interface checks the submitted grid against the task answer and allows progression to another test input when available.
Guide limitations
- The interface does not enforce the benchmark's three-trials-per-test-input rule.
- A task is counted as solved only when every test input has an exact output, including dimensions.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 13 October 2026
- Original website title
- fchollet/ARC-AGI · Official homepage title · retained in its source language
Related options
ARC-AGI-2 — The repository points to a separate ARC-AGI-2 repository.