OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Gill — kohjingyu

GILL is a multimodal language model project that processes interleaved image-and-text inputs to generate text, retrieve existing images or generate novel images. Its public repository provides the model code, pretrained weights and an inference notebook.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Gill — kohjingyu — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use GILL when experimenting with image-and-text dialogue and you need a model that can generate text, retrieve images or create novel images. Retrieval requires the separate visual-embedding files; without them, the model can still generate novel images.

Who it is for

  • Researchers studying multimodal language models and image generation.
  • Developers building research prototypes that combine image-and-text conversations with generated or retrieved images.

Documented uses

  • Generate novel images from prompts in a multimodal language-model workflow.
  • Explore text generation using image-and-text inputs.
  • Retrieve images using the provided Conceptual Captions visual embeddings.
  • Train or adapt a research model using Conceptual Captions data and precomputed text embeddings.
Before you choose

Requirements & limits

The public GitHub repository lists an Apache-2.0 license. The project instructions use a Python virtual environment and provide model weights in the repository.

  • The included pretrained weights are described as around 96MB.
  • Image retrieval requires separate precomputed Conceptual Captions visual embeddings of about 3GB; without these, retrieval is unavailable.
  • The inference notebook provides examples for generating text and images.
  • Training requires Conceptual Captions images and captions plus precomputed Stable Diffusion text-encoder embeddings.
  • The repository reports training on two A6000 GPUs for 48 hours and notes that smaller-memory GPUs may require configuration changes.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the GILL Python environment

Prepare the documented virtual environment and make the repository's GILL library available on PYTHONPATH.

Before you start

  • A Python installation with the venv module.
  • A local checkout of the GILL repository.

Steps

  1. Create and activate a virtual environment with the command below and source venv/bin/activate.

    Documented command
    python -m venv venv

  2. Install the repository dependencies with the command below.

    Documented command
    pip install -r requirements.txt

  3. Add the local GILL directory to PYTHONPATH, replacing the example path with the checkout path.

Expected outcome

The documented Python environment is prepared to use the repository's GILL library and included model checkpoint.

Guide limitations

  • These steps cover environment setup only; inference examples are in GILL_example_notebook.ipynb.
  • The repository's inference notes warn that outputs may differ slightly when Conceptual Captions images are no longer available.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
kohjingyu/gill · Official homepage title · retained in its source language
How we review sourcesSuggest a correction