OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Browser Agents

UI R1

UI R1 is an open-source Python research framework that explores rule-based reinforcement learning for improving multimodal language models’ reasoning and action prediction on graphical user interfaces. Its repository provides training and inference examples, model checkpoints and a dataset.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

UI R1 — OpenViST project profile in Browser Agents.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use UI R1 when you need a code-based starting point for research experiments on reinforcement-learning approaches to GUI grounding and action prediction, rather than a packaged end-user browser agent.

Who it is for

  • Researchers studying multimodal GUI agents and interface action prediction.
  • Developers building experiments for GUI grounding, model training or evaluation.

Documented uses

  • Train multimodal models to predict interface targets or actions from screenshots and instructions.
  • Run the documented inference example on annotated GUI images using a model checkpoint.
  • Compare GUI grounding results across model variants and benchmark tasks.
Before you choose

Requirements & limits

The repository is public and identifies the project as Apache-2.0 licensed.

  • The documented setup uses Conda and Python 3.10.
  • Inference requires users to supply a checkpoint and image/test-data paths.
  • The README provides separate scripts for standard and efficient training.
  • The repository reports its own benchmark comparisons; the figures are project claims.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare data and run the documented inference example

Use image files and a test JSON with bounding-box annotations as inputs to the repository's inference example.

Before you start

  • A UI-R1 model checkpoint and image files.
  • A test JSON containing each image filename, a bounding box in [x1, y1, x2, y2] order, and an instruction.
  • A Conda environment with Python 3.10 and the repository setup completed with setup.sh.

Steps

  1. Create and activate a Conda environment with Python 3.10, then run bash setup.sh from the repository.

  2. Place the input images in an images directory and prepare test.json with image filenames, bounding boxes and instructions.

  3. In evaluation/, set MODEL_PATH, IMG_PATH and TEST_JSON in test.sh to the checkpoint and data paths.

  4. Run bash test.sh from the evaluation directory.

Expected outcome

The repository's inference example is invoked with the supplied checkpoint and test data.

Guide limitations

  • The README does not specify an inference output format in the provided instructions.
  • The guide describes the repository instructions; it is not a claim that the workflow was executed.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
15 October 2026
Original website title
lll6gg/UI-R1 · Official homepage title · retained in its source language
How we review sourcesSuggest a correction