OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Image & Video AI

Instruction Tuned Sd

Instruction Tuned Sd is a Python research repository that explores instruction-tuning Stable Diffusion for image editing: a model takes an input image and a natural-language instruction and produces a transformed image.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Instruction Tuned Sd — OpenViST project profile in Image & Video AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use it when you want code and examples for researching or building instruction-guided image transformations, rather than a hosted, general-purpose image editor.

Who it is for

  • Researchers exploring instruction-conditioned image generation and editing.
  • Developers adapting Stable Diffusion training or inference for image transformations.
  • Teams prototyping image-editing workflows for tasks such as cartoonization or deraining.

Documented uses

  • Study a training approach that teaches Stable Diffusion to follow image-editing instructions.
  • Prepare or adapt datasets for cartoonization and low-level image processing.
  • Train a model using the repository’s Diffusers-based scripts.
  • Load an example pipeline, apply an instruction to an input image and save the result.
Before you choose

Requirements & limits

The public GitHub repository is listed under the Apache-2.0 license.

  • Training uses Python and a stack including PyTorch, Diffusers, Accelerate and Transformers.
  • The README’s reported experiments used PyTorch 1.13.1, CUDA 11.6 and a single A100 GPU; these are reported settings, not a compatibility guarantee.
  • Custom datasets may require changes to the dataset format, data loader or training scripts.
  • The inference examples use a CUDA device and half-precision tensors.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Run the documented cartoonization inference example

Load the example cartoonizer with Diffusers, apply an instruction to an image, and save the edited image.

Before you start

  • Python environment with PyTorch and Hugging Face Diffusers installed.
  • An environment with CUDA, as used by the example’s pipeline setup.
  • Access to the model and sample image referenced in the README.

Steps

  1. Load StableDiffusionInstructPix2PixPipeline from instruction-tuning-sd/cartoonizer using torch.float16 and move it to cuda.

  2. Load the README’s sample mountain image with Diffusers’ load_image.

  3. Call the pipeline with the instruction Cartoonize the following image and the loaded image.

  4. Save the first generated image as image.png.

Expected outcome

The example writes a cartoonized image to `image.png`.

Guide limitations

  • The README example uses a CUDA device and half-precision tensors.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
18 October 2026
Original website title
huggingface/instruction-tuned-sd · Official homepage title · retained in its source language
How we review sourcesSuggest a correction