Magictailor

Train and generate with a concept–component pair

Prepare reference images, train a MagicTailor model, and use its concept and component tokens in an image-generation prompt.

Before you start

• MagicTailor repository and its Conda environment.
• PyTorch 1.13.1 and torchvision 0.14.1 with CUDA 11.6 are provided as an installation example.
• Grounded-SAM installation for the full setup; it may be skipped for a quick try with the supplied example images.
• Reference images for a concept and a component, or the repository's example images.

Steps

1. Create a folder named in the {CONCEPT}_{ID}+{COMPONENT}_{ID} pattern and place the reference images inside, naming concept files 0_{CONCEPT}_{ID}.jpg and component files 1_{COMPONENT}_{ID}.jpg.

2. Install the Conda environment from environment.yml; install the indicated PyTorch and torchvision packages, then set up Grounded-SAM if using the full workflow.

3. Run the command below and check the quality of the Grounded-SAM masks.

   Documented commandCopy command

   python train.py --instance_data_dir {PATH-TO-PREPARED-DATA}

4. Use the saved model in outputs/magictailor with the command below.

   Documented commandCopy command

   python inference.py --model_path "outputs/magictailor" --prompt " with " --output_path "outputs/inference/result.jpg"

Guide limitations

• The README warns that GPU memory limits may require fewer reference images or moving the momentum denoising U-Net to another GPU where applicable.
• Hyperparameter adjustments may be needed to balance text alignment and identity fidelity.
• The README asks users to inspect Grounded-SAM masks; this recipe does not establish that any particular run will succeed.