OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
AI Models

OpenCLIP

OpenCLIP is an open-source Python and PyTorch implementation of CLIP for learning aligned image-and-text representations. It provides pretrained and untrained model interfaces, image preprocessing and text tokenization tools, and training code for multimodal models.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

OpenCLIP — OpenViST project profile in AI Models.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use OpenCLIP when you need model code and pretrained interfaces to integrate image-text comparison into a Python application, or when you want a CLIP-based starting point for multimodal training.

Who it is for

  • Python developers building image-and-text matching or retrieval workflows
  • Researchers and engineers prototyping zero-shot image classification
  • Teams training or adapting multimodal models with PyTorch

Documented uses

  • Compare an image with candidate labels or descriptions using image-text similarity scores.
  • Create image and text embeddings for a matching or retrieval workflow.
  • Select from the pretrained models exposed by the library.
  • Train multimodal models using the repository’s training stack.
Before you choose

Requirements & limits

The repository identifies OpenCLIP as open source and is publicly accessible. Its README documents installation through pip.

  • The README’s inference example uses a ViT-B-32 model with a named LAION checkpoint; available pretrained choices can be listed with open_clip.list_pretrained().
  • Some model families need additional packages: timm for timm-based image encoders and transformers for Transformers tokenizers.
  • The main branch has a refactored training API; the README points users seeking the older release-stable training API to the v3 branch or latest 3.x release.
  • The README notes that current model defaults use nn.GELU, while many existing checkpoints use QuickGELU and may require model definitions with the -quickgelu postfix.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Compare an image with candidate text labels

Encode one image and several text prompts, then calculate relative scores for each prompt.

Before you start

  • Python with open_clip_torch installed
  • PyTorch and Pillow
  • A supported model and pretrained checkpoint; the README example uses ViT-B-32 and laion2b_s34b_b79k
  • A local image file

Steps

  1. Install the package with the command below.

    Documented command
    pip install open_clip_torch

  2. Create the model, image transform, and tokenizer with open_clip.create_model_and_transforms and open_clip.get_tokenizer.

  3. Preprocess the image and tokenize candidate prompts.

  4. Encode both inputs, normalize their features, and apply the scaled image-text similarity and softmax calculation shown in the README.

Expected outcome

A relative score for each candidate text prompt against the image, as in the documented label-probability example.

Guide limitations

  • The README’s inference example uses torch.autocast("cuda"); it does not document hardware setup or performance requirements.
  • For timm-based image encoders, the README says to install or upgrade timm if the encoder is unknown.
  • Models using Transformers tokenizers require transformers.
Documentation 1 ↗
Source-based practical guide

List available pretrained model choices

Inspect the pretrained model entries exposed by OpenCLIP before selecting one for an inference workflow.

Before you start

  • Python with open_clip_torch installed

Steps

  1. Import open_clip.

  2. Call open_clip.list_pretrained() to display available pretrained models.

  3. Consult the repository’s pretrained-model documentation for additional details.

Expected outcome

A listing of pretrained model choices and a pointer to the project’s model documentation.

Guide limitations

  • The listing does not itself run inference or determine which model suits a particular dataset.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
mlfoundations/open_clip · Official homepage title · retained in its source language
How we review sourcesSuggest a correction