OpenViSTAI · Agents · Blockchain · Crypto
Submit a project
Menu
Image & Video AI

Matfuse

MatFuse is a Python and PyTorch research implementation that uses diffusion models to generate and edit SVBRDF material maps for computer graphics. It accepts conditioning from text, pictures, sketches and color palettes, and represents diffuse, normal, roughness and specular maps separately.

Explore the practical guide ↓

Independent OpenViST project profile, based on linked sources.

Matfuse — OpenViST project profile in Image & Video AI.
OpenViST project illustration. See the linked sources for product details.
Understand the project

Practical overview

When it fits

Use MatFuse when a graphics workflow calls for controllable SVBRDF map generation or targeted map-level editing, and local compute and prepared model files are available.

Who it is for

  • 3D artists and rendering practitioners who need controllable material maps.
  • Researchers and developers exploring diffusion-based PBR material synthesis or material editing.

Documented uses

  • Generate material maps guided by text, images, sketches or color palettes.
  • Combine conditioning inputs, such as a sketch and text description.
  • Refine generated materials by masking areas of specific maps or the complete material.
  • Train the autoencoder and diffusion stages using a prepared material dataset.
  • Convert a checkpoint to Diffusers format for use in an existing pipeline.
Before you choose

Requirements & limits

The public repository is licensed under MIT. The documented local inference route requires at least 12 GB of GPU VRAM; the README specifies Python 3.10.13 and PyTorch 1.13.1 for its environment.

  • MatFuse generates structured material maps rather than only ordinary images.
  • Conditioning can combine text, pictures, sketches and color palettes.
  • Map-level masked editing is supported after synthesis.
  • Local inference requires a full checkpoint, a configuration and at least 12 GB of GPU VRAM.
  • Training requires both autoencoder and diffusion stages, plus a dataset with material maps, sketches, renders and metadata.

Check current requirements in the official documentation. Source-based guidance does not mean OpenViST has run or independently tested this project.

Source-based practical guide

Prepare material data for training

Structure a dataset and extract color palettes before configuring MatFuse training.

Before you start

  • Material folders containing diffuse, normal, roughness and specular maps, a sketch, renders, and metadata.json with a caption and color palette.
  • Separate train and test data splits.

Steps

  1. Arrange the dataset under the documented data/MatFuse/{split}/ structure, with a folder for each material.

  2. Run the command below to extract palettes from renders.

    Documented command
    python src/scripts/data/extract_palette.py --data

  3. Set the data_root property in the selected configuration to the dataset location.

Expected outcome

The dataset is organized for use with the repository's MatFuse dataset class and training configurations.

Guide limitations

  • This recipe prepares data only; model training requires separate autoencoder and diffusion stages.
  • The repository says it does not plan to release its training dataset.
Documentation 1 ↗
Evidence and transparency

Sources & review

Official repository ↗
Source record details
Source availability
Active
Sources checked
17 October 2026
Original website title
giuvecchio/matfuse · Official homepage title · retained in its source language
How we review sourcesSuggest a correction