Practical overview
When it fits
Use it when a task can be completed through visible desktop interfaces and you want an LLM to control the computer rather than merely provide instructions. The project demos include solving Wordle, creating a meal plan in Google Docs and writing a web app.
Who it is for
- People who want an LLM to carry out tasks through desktop applications using keyboard and mouse input.
- Developers configuring OpenAI, Gemini or a compatible custom LLM as a computer-control backend.
Documented uses
- Translate a natural-language request into computer actions.
- Use keyboard and mouse input to interact with desktop apps.
- Send screenshots to an LLM to reassess progress and adjust actions.
- Configure OpenAI, Gemini or a custom OpenAI-style LLM backend.
Requirements & limits
The repository is public and specifies the GPL-3.0 license. The README describes installation for macOS, Linux and Windows; it notes Linux binary testing on Ubuntu 20.04 and Windows binary testing on Windows 10. macOS use requires Accessibility and Screen Recording permissions. OpenAI GPT-4o setup instructions say a minimum $5 payment is needed to unlock GPT-4o.
- The agent uses simulated keyboard and mouse input, so tasks need to be actionable through the computer’s interface.
- Screenshots are sent to the LLM backend to assess progress and guide adjustments.
- OpenAI GPT-4o, Gemini and custom OpenAI-style backends are documented; each requires configuring backend credentials and settings.
- macOS users need to grant Accessibility and Screen Recording permissions.
- The project provides desktop installation instructions across macOS, Linux and Windows, with specific binary testing notes for Ubuntu 20.04 and Windows 10.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Connect Open Interface to an LLM backend
Configure a documented OpenAI, Gemini or custom OpenAI-style backend and restart the app.
Before you start
- Open Interface installed
- For OpenAI: an API key; the README says GPT-4o setup requires adding at least $5 to the OpenAI account
- For Gemini: a Google Gemini API key
- For a custom backend: its base URL and model name, and an API key if needed
Steps
For OpenAI, enter the API key in the app’s Settings menu; for Gemini, select a model in Advanced Settings and enter the key in Settings.
For a custom OpenAI-style backend, enter the base URL and model name in Advanced Settings and provide the API key in Settings as needed.
Save the settings and restart the app.
Expected outcome
The app is configured with the selected LLM backend for computer-control requests.
Guide limitations
- This guide covers backend setup only, not completion of a computer-control task.
- Custom LLMs must use an OpenAI-style API; the README suggests a library such as LiteLLM for backends that do not.
- The README notes that Llama may require a placeholder API key and a /v1/ suffix on the base URL.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 10 October 2026
- Original website title
- AmberSahdev/Open-Interface · Official homepage title · retained in its source language