Practical overview
When it fits
Use Vision SR1 when you need repository-level training and evaluation resources for experiments on visual perception reasoning in VLMs, rather than a hosted chat interface.
Who it is for
- Researchers experimenting with reinforcement learning for vision-language models.
- Practitioners comparing visual-reasoning training approaches using linked checkpoints and datasets.
Documented uses
- Train a VLM with accuracy rewards and self-generated perception feedback.
- Compare self-reward training with a standard accuracy-reward mode.
- Evaluate fine-tuned checkpoints across benchmark datasets and review per-dataset accuracy.
Requirements & limits
The GitHub repository is public and lists an Apache-2.0 license. The evidence does not establish the licensing of all linked model and dataset artifacts.
- The project offers both self-reward GRPO and a standard accuracy-reward mode.
- LoRA training is documented with rank 64 and a learning rate of 1e-5; the visual tower is excluded, and the README says results are not verified.
- Hardware estimates are substantial and vary by model size and training method; the repository labels them estimated.
- Qwen3-VL is described as needing further debugging because its format reward is always zero.
- The evaluation workflow depends on an LLM judge identified as Qwen2.5-14B-Instruct via vLLM.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up the Vision SR1 repository
Create the documented Conda environment and run the repository setup script.
Before you start
- Python 3.9 or later is listed as a software requirement.
- The README specifies transformers 4.49.0.
- Conda and Git are needed for the documented setup commands.
Steps
Clone https://github.com/zli12321/Vision-SR1.git.
Change into the Vision-SR1 directory.
Create the Conda environment with
conda create -n Vision-SR1 python=3.12.Run the command below.
bash setup.sh
Expected outcome
The repository's documented setup sequence has been followed to create its Conda environment and invoke setup.sh.
Guide limitations
- The setup instructions do not themselves train a model.
- No execution or compatibility test is implied.
Evaluate a fine-tuned Vision-SR1 checkpoint
Run the documented full-fine-tuning evaluation pipeline for a Vision-SR1 checkpoint.
Before you start
- A Vision-SR1 full-fine-tuning checkpoint.
- The repository environment and evaluation dependencies.
- An available Qwen2.5-14B-Instruct model through vLLM for the documented LLM judging stage.
Steps
Pass the checkpoint path to
evaluation/full_rl/eval_vision_sr1.sh.The script generates benchmark responses with greedy decoding and saves them under
evaluation/responses/.The pipeline extracts boxed answers and compares them with ground truth using its LLM judge.
Use the printed per-dataset accuracy table to inspect the judgments.
Expected outcome
The documented pipeline produces benchmark responses, judgment files and a per-dataset accuracy table.
Guide limitations
- This recipe covers full fine-tuning checkpoints; the repository documents separate evaluation scripts for LoRA checkpoints.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 15 October 2026
- Original website title
- zli12321/Vision-SR1 · Official homepage title · retained in its source language