OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Browser Agents

Handsy AI

Handsy AI is an iPhone task-execution app that uses a selected AI service or vision model to interpret the mirrored screen and direct an ESP32 board to tap, swipe and type.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Handsy AI — OpenViST project profile in Browser Agents.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Handsy when a task requires an agent to observe and interact with an iPhone screen over multiple steps, rather than merely answer a question. The app-led route suits direct phone tasks; the CLI route is intended for computer-based agents controlling the same board.

Who it is for

  • People who want an AI service to carry out multi-step tasks on a real iPhone.
  • Users of computer-based agents who want to direct an iPhone through Handsy CLI.
  • People who want to save recurring phone tasks and invoke them through Siri, Shortcuts, widgets or Control Center.

Documented uses

  • Turn a natural-language goal into a sequence of iPhone screen actions.
  • Navigate screens or complete tasks such as setting an alarm.
  • Repeat saved tasks through supported iPhone shortcuts and controls.
  • Review a run and intervene when the app requests help.
Before you choose

Requirements & limits

The support documentation describes free AI models as available, while a subscription AI service is described as future-only. Free OpenRouter models have request limits and may be busy.

  • Requires an ESP32 board for hardware-based phone interaction; setup includes board preparation, Screen Mirroring and pointer calibration.
  • The iPhone screen is the agent’s visual input, and the board provides pointer and tap actions through AssistiveTouch.
  • The app checks the screen again after acting, supporting multi-step task progression.
  • The app provides stop and intervention controls, including prompts before paying, messaging or deleting.
  • Free model rate limits can constrain task duration; the support page estimates 20–40 requests per run.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up Handsy with an iPhone and ESP32 board

Prepare the board and app so Handsy can see and interact with the iPhone screen.

Before you start

  • An iPhone
  • A Seeed Studio XIAO ESP32S3 board and a suitable cable
  • The Handsy app and an AI service

Steps

  1. Build and flash the board using the linked board guide.

  2. Open the app and connect the board.

  3. Start Screen Mirroring to the board.

  4. Calibrate the pointer, then choose an AI service.

Expected outcome

The app is set up to use the board with the mirrored iPhone screen.

Guide limitations

  • The documentation is marked beta and may change.
  • Free OpenRouter models can be rate-limited or busy; a run may be slow or stop partway.
Documentation 1 ↗Documentation 2 ↗
Evidence and transparency

Sources & review

handsy.tech ↗Guide documentation ↗Guide documentation ↗
Source record details
Source availability
Active
Sources checked
3 November 2026
Original website title
Handsy AI · ESP32 iPhone Agent · An AI that uses your iPhone with its own hands · Official homepage title · retained in its source language
How we review sourcesSuggest a correction