OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Agent Frameworks

Vagen

Vagen is an open-source Python reinforcement-learning framework for training and evaluating multi-turn vision-language model agents, including experiments that reward visual state estimation and prediction of what happens next.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Vagen — OpenViST project profile in Agent Frameworks.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Vagen for research or development that requires training and evaluating vision-language agents across visual interaction turns, especially when explicit state and transition reasoning are part of the experiment.

Who it is for

  • Researchers studying multi-turn vision-language agents and world-model reasoning.
  • Developers adapting agent harnesses, environments or training algorithms for visual tasks.

Documented uses

  • Train a vision-language agent through multi-turn reinforcement learning.
  • Reward state estimation and transition prediction in addition to task success.
  • Evaluate agents in environments such as Sokoban or FrozenLake.
  • Adapt environment, harness or advantage-estimator components.
Before you choose

Requirements & limits

The repository is public and specifies an MIT license. The documented workflow uses a local Python/CUDA stack; no account or pricing information is provided.

  • The installation instructions specify Python 3.12 and a CUDA 13 stack, including Torch 2.11.0, SGLang 0.5.13 and Transformers 5.8.1.
  • SGLang is the documented default rollout engine; vLLM can be selected separately, but the README warns not to install both in one environment because of incompatible FlashInfer builds.
  • The environment registry includes Sokoban, FrozenLake, SpatialGym, PrimitiveSkill and RemoteEnv; several environments need their own setup or dependencies.
  • The quick-start training flow includes Weights & Biases login; the README also describes self-hosted W&B login.
  • The framework separates harness, environment, model, rollout and evaluation concerns, while some environment or model configurations require task-specific work.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install VAGEN with the default SGLang backend

Set up the VAGEN environment and install its default training stack.

Before you start

  • Conda and Git
  • A Linux environment with the CUDA 13 stack supported by the installer

Steps

  1. Create and activate a Python 3.12 Conda environment: conda create -n vagen python=3.12 -y and conda activate vagen.

  2. Clone the repository and enter it: the command below and cd VAGEN.

    Documented command
    git clone --branch dev/bi_level https://github.com/JamesKrW/VAGEN.git

  3. Run the command below to initialize the pinned verl submodule and install the default SGLang stack.

    Documented command
    bash scripts/install.sh

Expected outcome

The installation script sets up the pinned `verl` submodule and installs Torch 2.11.0, SGLang 0.5.13, and Transformers 5.8.1.

Guide limitations

  • The documented install command clones the dev/bi_level branch from JamesKrW/VAGEN, not the repository's main branch.
  • The installer uses the CUDA 13 stack.
  • Do not install vLLM and SGLang in the same environment because their flashinfer builds are incompatible.
Documentation 1 ↗
Source-based practical guide

Run a documented Sokoban training example

Start VAGEN training using the Qwen2.5-VL default GAE Sokoban example.

Before you start

  • A VAGEN installation
  • Weights & Biases login, as shown in Quick Start

Steps

  1. Log in with wandb login.

  2. Change to the VAGEN repository directory with cd VAGEN.

  3. Run the command below.

    Documented command
    bash examples/train/sokoban/train_default_gae_qwen25vl3b.sh

Expected outcome

The documented command starts the Qwen2.5-VL default GAE Sokoban training example.

Guide limitations

  • The source does not specify training duration, required compute, or a guaranteed training result.
  • The example depends on the VAGEN environment and model configuration.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
14 October 2026
Original website title
mll-lab-nu/VAGEN · Official homepage title · retained in its source language
How we review sourcesSuggest a correction