PALI3: Python vision-language model

Understand the project

Practical overview

Use PALI3 when you want to inspect or prototype a Python-based image-and-text model workflow, rather than use a hosted consumer application.

Developers prototyping image-question answering or image-description applications.
Researchers studying multimodal model architectures and task-specific adaptation.

Explore a model flow that turns image and query inputs into generated text.
Build a prototype for visual question answering or image captioning.
Examine a multimodal architecture connecting a SigLIP vision encoder to a UL2 text generator.

Before you choose

The repository lists an MIT license.
The README provides a pip installation command and a PyTorch-based usage example.
The README marks full-model training scripts and other implementation tasks as unfinished, so it does not describe a complete training package.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the PALI3 Python package and follow the README’s example of passing image, prompt, output-text, and mask tensors to the model.

Install the package with the command below.
Documented command
```
pip install pali3
```
Import torch and Pali3 from pali3.main, then create a model with model = Pali3().
Prepare tensors as shown in the README: an image shaped (1, 3, 256, 256), a prompt shaped (1, 1024), a boolean mask shaped (1, 1024), and output-text input shaped (1, 1024).
Call model.process(img, prompt, output_text, mask) and print the returned result.

The example calls PALI3’s processing method with image and text-related tensors and prints the returned result.

The README’s sample tensors are randomly generated placeholders, not a demonstrated workflow with a real image or natural-language query.
The README describes the architecture and training stages at a high level and notes that the actual implementation may involve additional details and complexities.
The README lists full-model training scripts as a TODO; this guide does not cover training or deployment.

Evidence and transparency

Source record details

Source availability: Active
Sources checked: 8 October 2026
Original website title: kyegomez/PALI3 · Official homepage title · retained in its source language