OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Browser Agents

Omnimcp

OmniMCP is an experimental Python system for AI-driven computer use. It captures desktop screens, uses OmniParser for visual interface understanding, plans actions with a language model, and executes them through mouse and keyboard input. Its documented primary workflow is a command-line agent loop; its experimental MCP server is a separate component.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Omnimcp — OpenViST project profile in Browser Agents.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use OmniMCP when you want to explore or prototype an AI agent that perceives and acts on a desktop interface, and can provide the Python environment, credentials and graphical runtime it requires.

Who it is for

  • Developers exploring visual AI agents that interact with desktop interfaces.
  • Researchers prototyping a perceive-plan-act computer-use loop.
  • Builders investigating how computer-use actions might connect with MCP-oriented tooling.

Documented uses

  • Inspect a desktop screen and identify interface elements.
  • Plan a next action from a task goal, interaction history and visual state.
  • Execute mouse and keyboard actions through a CLI-run agent loop.
  • Review timestamped screenshots, action highlights and logs from a run.
Before you choose

Requirements & limits

  • The primary documented workflow is the CLI and AgentExecutor; the MCP server is experimental and separate.
  • The project specifies Python 3.10 to below 3.13 and requires uv.
  • Linux use of pynput requires an active X11 or Wayland graphical session and may need system libraries.
  • The optional OmniParser deployment path uses AWS resources and can incur costs.
  • This repository is explicitly described as a lab experiment, distinct from OpenAdapt’s workflow recording, compilation and replay product.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up OmniMCP for a computer-use task

Install the repository, configure its environment and launch the documented CLI workflow.

Before you start

  • Python >=3.10 and <3.13
  • uv installed
  • A graphical session on Linux for pynput; system libraries may also be needed
  • An environment configured from .env.example with the required model credentials

Steps

  1. Clone the OmniMCP repository and enter its directory.

  2. Run ./install.sh, copy .env.example to .env, and configure the required credentials.

  3. Activate the created virtual environment.

  4. Run python cli.py for the default calculator goal, or use python cli.py --goal "Your goal here" for a custom goal.

Expected outcome

The CLI starts the documented perceive-plan-act workflow; timestamped debugging outputs are saved under runs/.

Guide limitations

  • This is an experimental computer-use project; the README’s calculator example is a demo, not a guarantee for arbitrary tasks.
  • The optional AWS OmniParser deployment can create chargeable resources and needs AWS credentials.
  • The experimental MCP server is separate from the primary CLI workflow.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
OpenAdaptAI/OmniMCP · Official homepage title · retained in its source language
How we review sourcesSuggest a correction