Practical overview
When it fits
Use Custom Diffusion when you have example images for a concept and want to adapt a text-to-image diffusion model to generate that concept in new prompted contexts.
Who it is for
- Researchers evaluating few-shot text-to-image model customization.
- Developers building workflows that fine-tune diffusion models on user-provided concept images.
- Practitioners who want to train a model to combine multiple specified concepts.
Documented uses
- Adapt a pretrained diffusion model to a new subject or visual style using a small image set.
- Train a model on multiple concepts, such as an object paired with a style.
- Use CustomConcept101 and its evaluation prompts in a customization evaluation workflow.
Requirements & limits
- The method fine-tunes selected cross-attention key and value projection matrices rather than the full model.
- The original-code examples use Stable Diffusion 1.4; the README separately documents Diffusers support and SDXL training code.
- The original training example specifies 30 GB on two GPUs; Diffusers instructions recommend two GPUs and describe a one-GPU adjustment.
- The project reports about 75 MB of additional storage per concept.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Prepare the original Stable Diffusion environment
Set up the repository’s documented original-code environment and download its pretrained checkpoint.
Before you start
- Git
- Conda
- A pretrained Stable Diffusion 1.4 checkpoint
Steps
Clone the Custom Diffusion repository and the Stable Diffusion repository.
In the Stable Diffusion directory, create and activate the environment with its environment.yaml file.
Install clip-retrieval and tqdm.
Download the sd-v1-4.ckpt checkpoint and retain its path for the training commands.
Expected outcome
A prepared environment and checkpoint for proceeding to the repository’s fine-tuning instructions.
Guide limitations
- This guide covers setup only, not training or image generation.
- The README identifies a particular Stable Diffusion code commit for development; it does not establish compatibility with all revisions.
Configure the documented Diffusers training environment
Install the packages and configure Accelerate for the repository’s Diffusers training instructions.
Before you start
- Python and pip
- A pretrained model identifier or checkpoint for the later training stage
Steps
Install Accelerate, modelcards, Transformers, DeepSpeed, and Diffusers pinned to version 0.21.4.
Run the documented Accelerate configuration step.
Set the model name to the Stable Diffusion 1.4 model identifier, or follow the separate SDXL model-name and training-script note.
Expected outcome
A configured Diffusers environment ready for the repository’s separate training commands.
Guide limitations
- This guide stops at environment configuration; it does not cover launching training.
- The README’s version reference does not establish compatibility with every Diffusers or SDXL version.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 17 October 2026
- Original website title
- adobe-research/custom-diffusion · Official homepage title · retained in its source language