Openai Clip: Train image–text embeddings in PyTorch

Understand the project

Practical overview

Use it when you want a code-level example of CLIP-style training and text-to-image retrieval over a dataset you provide.

Developers and learners studying contrastive image–text models in PyTorch.
Researchers or practitioners adapting a training example to paired image-caption data.

Train projections that align image and text representations.
Encode a validation image set and retrieve images matching a text query.
Study how contrastive targets handle multiple captions associated with one image.

Before you choose

The repository is public and MIT-licensed. It is code to run in a Python/PyTorch environment, not a documented hosted service.

The example expects local image and caption data configured for its dataset pipeline.
The documented text encoder uses DistilBERT, and the notebook references timm, Transformers, OpenCV, NumPy, pandas and Albumentations.
Training targets use sample IDs so multiple captions for an image can be treated as positives in both image-to-text and text-to-image directions.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Use a trained CLIP model to rank validation images against a text query.

A trained model checkpoint such as best.pt.
A validation dataframe and the project’s configured device, tokenizer, and model components.

Create the validation dataframe with make_train_valid_dfs().
Call get_image_embeddings(valid_df, "best.pt") to load the checkpoint and compute validation image embeddings.
Call find_matches(model, image_embeddings, query, image_filenames, n=9) with a text query and the image filenames to find relevant images.

The documented inference workflow computes validation image embeddings and uses a text query to retrieve relevant images.

The documented embeddings have shape (valid_set_size, 256).
The find_matches implementation is truncated in the source, so ranking and display details are not fully available.

Evidence and transparency

Source record details

Source availability: Active
Sources checked: 7 November 2026
Original website title: moein-shariatnia/OpenAI-CLIP · Official homepage title · retained in its source language