OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Palm E

Palm E is an open-source Python implementation of an embodied multimodal model architecture that combines language and visual observations for embodied reasoning tasks.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Palm E — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when studying or adapting an embodied multimodal architecture and you can provide the tokenizer and training work needed beyond the included architecture.

Who it is for

  • AI researchers exploring embodied multimodal models
  • Developers prototyping language-and-vision approaches for robotics-related reasoning

Documented uses

  • Inspect an implementation of an embodied language-and-vision architecture
  • Explore a model call with image-shaped and token-shaped tensors
  • Investigate research tasks such as robotic manipulation planning, navigation and visual question answering
Before you choose

Requirements & limits

  • The repository provides architecture code, not pretrained weights or a tokenizer.
  • The README says inference requires setting up text and image tokenizers and training first.
  • A basic example uses random image and token tensors; it is not an example of trained-model inference.
  • Python is the documented language, and the repository lists an Apache-2.0 license.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Install and exercise the PALM-E example

Install the Python package and call its model with the example image and token inputs.

Before you start

  • Python environment with pip
  • PyTorch, imported as torch in the example

Steps

  1. Install the package with the command below.

    Documented command
    pip install palme

  2. Import torch and PalmE from palme.model.

  3. Create the example inputs: img = torch.randn(1, 3, 256, 256) and caption = torch.randint(0, 20000, (1, 1024)).

  4. Instantiate model = PalmE(), call output = model(img, caption), and print output.shape.

Expected outcome

The README example indicates an output shape of `(1, 1024, 20000)`.

Guide limitations

  • The repository describes this as model architecture only; it includes no pretrained weights or tokenizer.
  • For actual inference, the README says to set up tokenizers for text and images, then train, then perform inference; it does not document those stages.
  • The documented example uses generated tensors rather than real images or text.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
15 October 2026
Original website title
kyegomez/PALM-E · Official homepage title · retained in its source language
How we review sourcesSuggest a correction