Practical overview
When it fits
Use it when you need a code-based workflow for prompt-guided changes restricted to a mask, and can provide the image, mask, environment and model resources required by the chosen implementation.
Who it is for
- Researchers studying text-driven image editing and latent diffusion.
- Developers who want to run or adapt a command-line image-editing implementation.
- Image creators who can prepare masks and run the required local model environment.
Documented uses
- Change an object or background within a selected image region using a text prompt.
- Generate candidate predictions for a masked edit.
- Optionally reconstruct the original background after generating edits with the older LDM workflow.
- Explore additions, scribble edits and text generation shown among the repository's applications.
Requirements & limits
The repository is public and lists an MIT license. Its workflows are run locally from the command line; the older LDM path requires downloading 5.7 GB of pretrained weights.
- The newer route uses Diffusers with PyTorch 2.1 and offers SDXL and Stable Diffusion 2.1 scripts.
- The repository says SDXL requires a stronger GPU and notes smaller batch sizes as a way to reduce GPU memory use.
- The older LDM route has a separate weights download and optional background-reconstruction command.
- Both example workflows require a source image and mask, so the intended edit area must be prepared.
Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.
Generate a masked edit with Stable Diffusion XL
Run the repository's documented SDXL image-editing script with a prompt, source image and mask.
Before you start
- Conda
- PyTorch 2.1.0, torchvision 0.16.0 and PyTorch CUDA 11.8, installed using the repository's commands
- Diffusers 0.19.3
- An image at inputs/img.png and a corresponding mask at inputs/mask.png
- A stronger GPU for Stable Diffusion XL
Steps
Install the listed PyTorch, torchvision and CUDA packages with the repository's conda command.
Install Diffusers 0.19.3 with the listed pip command.
Run the SDXL script with a prompt and the image and mask paths, for example: python scripts/text_editing_SDXL.py --prompt "a stone" --init_image "inputs/img.png" --mask "inputs/mask.png".
Expected outcome
The documented command runs the SDXL text-editing workflow on the supplied image and masked region.
Guide limitations
- The repository notes that SDXL requires a stronger GPU.
- A smaller batch_size can be used to save GPU memory.
Generate and optionally reconstruct edits with the older LDM workflow
Prepare the older Latent Diffusion Model weights, create edit predictions, and optionally reconstruct the original background.
Before you start
- The repository's Conda environment, created from environment.yaml and activated as ldm
- The pretrained model weights, listed as 5.7 GB
- An image at inputs/img.png and a corresponding mask at inputs/mask.png
Steps
Create and activate the environment with conda env create -f environment.yaml and conda activate ldm.
Download the pretrained weights to models/ldm/text2img-large/model.ckpt using the repository's provided download command; a Hugging Face mirror is also listed.
Run scripts/text_editing_LDM.py with a prompt, image and mask, for example: python scripts/text_editing_LDM.py --prompt "a pink yarn ball" --init_image "inputs/img.png" --mask "inputs/mask.png".
Optionally run scripts/reconstruct.py with the image, mask and selected prediction indices to reconstruct the original background.
Expected outcome
Initial predictions are saved under outputs/edit_results/samples; optional reconstructed results are saved under outputs/edit_results/samples/reconstructed_optimization.
Guide limitations
- The pretrained weights require a 5.7 GB download.
- Background reconstruction is a separate optional step.
Sources & review
Source record details
- Source availability
- Active
- Sources checked
- 15 October 2026
- Original website title
- omriav/blended-latent-diffusion · Official homepage title · retained in its source language