OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Screenai

Screenai is a Python implementation of a vision-language model for understanding user interfaces and infographics by processing image and text inputs together. Its documented design includes a vision transformer, multimodal encoder, attention layers and decoder, and the repository demonstrates model construction and a tensor-input forward pass.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Screenai — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use Screenai when you want to inspect or prototype with Python code for a model that combines image and text inputs, particularly for UI or infographic understanding.

Who it is for

  • Python developers exploring multimodal models
  • Researchers and prototypers working on UI screenshot or infographic understanding

Documented uses

  • Install the package and construct a ScreenAI model instance.
  • Pass image and text tensors into the model's documented forward method.
  • Use the implementation as a starting point for screenshot-and-text or infographic question-answering experiments.
Before you choose

Requirements & limits

The public GitHub repository identifies the code as MIT-licensed and provides a pip installation command.

  • The README demonstrates a forward pass with random tensors, not interpretation of real screenshots or infographic questions.
  • The project is implementation code rather than a documented end-user application.
  • The repository lists an encoder and decoder module-list item as unfinished.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install Screenai and construct the model

Set up the documented Python package and create a ScreenAI model instance.

Before you start

  • Python with pip3
  • PyTorch, imported as torch in the example

Steps

  1. Install the package with the command below.

    Documented command
    pip3 install screenai

  2. Import torch and ScreenAI from screenai.main.

  3. Create a model instance using the documented parameters, such as image size 224, patch size 16 and dimension 512.

Expected outcome

A ScreenAI model instance is constructed and ready for the documented tensor-input forward pass.

Guide limitations

  • The guide covers installation and construction, not a real-world UI or infographic task.
  • The repository lists implementation of an encoder and decoder module-list item as unfinished.
Documentation 1 ↗
Source-based practical guide

Run the documented tensor-input example

Pass example image and text tensors through a constructed ScreenAI model.

Before you start

  • The Screenai package installed
  • PyTorch
  • A model instance configured as in the repository example

Steps

  1. Create an image tensor with shape (1, 3, 224, 224) using torch.rand.

  2. Create a text tensor with shape (1, 1, 512) using torch.randn.

  3. Call model(text, image) and print the returned value.

Expected outcome

The example produces and prints the model's forward-pass output for the supplied tensors.

Guide limitations

  • The example uses randomly generated tensors rather than a real screenshot or text prompt.
  • It does not document how to interpret model output for a user-facing task.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
7 October 2026
Original website title
kyegomez/ScreenAI · Official homepage title · retained in its source language
How we review sourcesSuggest a correction