Practical overview
When it fits
Use Osworld when an agent must be evaluated through tasks performed in a real computer environment, rather than through a text-only benchmark.
Who it is for
- Researchers benchmarking multimodal computer-use agents
- Developers comparing models or testing changes to GUI-operating agents
- Teams reviewing agent task completion across computer workflows
Documented uses
- Run agents against standardized GUI benchmark tasks.
- Compare task success across models and domains.
- Review a specific task manually and inspect its execution evidence.
- Track regression in agents that operate computer environments.
Requirements & limits
The public GitHub repository is licensed under Apache-2.0.
- Choose VMware or VirtualBox for a non-virtualized desktop or bare-metal host; use the documented Docker route for virtualized servers.
- Docker evaluation is intended for hosts with KVM support; the project notes that macOS generally lacks KVM support.
- Baseline runs require an API key for the model provider. Some benchmark tasks additionally require Google OAuth configuration or proxy settings.
- The repository documents Python 3.10 or later and installation of its requirements.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Set up OSWorld on a desktop or bare-metal host
Install the repository dependencies and configure a VMware-based environment.
Before you start
- Python 3.10 or later
- Git and pip
- VMware Workstation Pro, or VMware Fusion on systems with Apple chips
- The vmrun command configured
Steps
Clone the OSWorld repository and change into its directory.
Install the dependencies with pip install -r requirements.txt.
Install VMware and configure vmrun.
Run vmrun -T ws list to check that VMware is configured.
Expected outcome
The documented setup is ready for OSWorld to download the required virtual machines and configure the environment.
Guide limitations
- The guide distinguishes non-virtualized hosts from virtualized platforms; consult the Docker instructions for the latter.
- VirtualBox is also supported, but parallelism and macOS on Apple chips may not be well-supported.
Configure Docker as the OSWorld environment provider
Prepare the Docker provider for experiments on a server with KVM support.
Before you start
- A Linux host with Docker installed
- KVM capability is recommended
Steps
On Linux, check KVM capability with egrep -c '(vmx|svm)' /proc/cpuinfo.
Install Docker Desktop or Docker Engine as appropriate for the host.
When initializing DesktopEnv, set provider_name to docker and os_type to Ubuntu or Windows according to the virtual machine operating system.
Expected outcome
The documented configuration selects Docker to host the experiment's virtual machine.
Guide limitations
- macOS hosts generally do not support KVM; the project advises using VMware on macOS.
- Abnormally interrupted experiments may leave Docker containers that can affect system performance.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 30 September 2026
- Original website title
- xlang-ai/OSWorld · Official homepage title · retained in its source language