OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

Fromage

Fromage, also known as FROMAGe, is an open-source research model and code repository for grounding language models to images. It provides pretrained model weights, inference examples, and scripts for image retrieval, text generation, evaluation, and training.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Fromage — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use FROMAGe when you need research code and model weights for experimenting with image-grounded language-model inputs and outputs, or when reproducing the repository's documented retrieval and generation evaluations.

Who it is for

  • Researchers exploring language models grounded in visual inputs.
  • Developers prototyping image-question answering or image-conditioned dialogue.
  • Teams studying contextual image retrieval or adapting the model with Conceptual Captions data.

Documented uses

  • Run inference examples with a model that takes visual inputs.
  • Prototype question answering or dialogue conditioned on images.
  • Evaluate contextual image retrieval and text generation on the documented Visual Storytelling and VisDial tasks.
  • Prepare Conceptual Captions training data and train a FROMAGe model.
Before you choose

Requirements & limits

The repository is public and licensed under Apache-2.0. Local setup uses a Python virtual environment and installed requirements; no account requirement or price is specified in the documentation.

  • The standard included weights are about 11 MB; the optional fromage_vis4 checkpoint uses four visual tokens and is described as generally better for dialogue and more complex images.
  • Image retrieval uses a separate precomputed Conceptual Captions embeddings file of about 3 GB.
  • Training requires downloading and formatting Conceptual Captions images and captions; the README's example uses an A6000 GPU and notes that lower-memory GPUs may require changes to batch size or other settings.
  • The README notes that some image outputs may differ because Conceptual Captions images can become unavailable over time.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Set up the FROMAGe Python environment

Prepare the documented library environment for working with FROMAGe.

Before you start

  • A local copy of the FROMAGe repository
  • Python and pip available in the environment

Steps

  1. Create a virtual environment with the command below.

    Documented command
    python -m venv venv

  2. Activate it with source venv/bin/activate.

  3. Install the repository requirements with the command below.

    Documented command
    pip install -r requirements.txt

  4. Add the local FROMAGe library path to PYTHONPATH, replacing the example path with the repository location.

Expected outcome

The documented virtual environment has the project dependencies installed and the library path configured for Python.

Guide limitations

  • This covers environment setup only; the README points to FROMAGe_example_notebook.ipynb for inference examples.
  • The PYTHONPATH command uses an example filesystem path that must be adjusted to the local checkout.
Documentation 1 ↗
COMPARE OPTIONS

Compare options for Model inference and serving

These projects document a common task and share a project role. This is a source-based comparison, not an execution benchmark or a guarantee of interchangeability.

ProjectDocumented purposePlatformAccess and requirements
FromageRun inference examples with a model that takes visual inputs.Linux, Self-hostedThe repository is public and licensed under Apache-2.0. Local setup uses a Python virtual environment and installed requirements; no account requirement or price is specified in the documentation.
Pulse Of MotionEstimate a video's physical frame rate from its visual dynamics.LinuxThe repository is public. Its README describes a released checkpoint that downloads from Hugging Face on first inference, with a manual download option. The strongest internal model is not fully released; the README directs interested users to contact the project about access. No license information is provided in the repository evidence.
Neural Image AssessmentTrain the model using AVA images and rating annotations.Cloud, Linux, macOS, Windows
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
16 October 2026
Original website title
kohjingyu/fromage · Official homepage title · retained in its source language
How we review sourcesSuggest a correction