---
license: mit
language:
- en
tags:
- material
- pbr
- svbrdf
- texture
- editing
---


# MatFuse: Controllable Material Generation with Diffusion Models

## 🧩 Model Overview

MatFuse leverages diffusion models to simplify the creation of Spatially-Varying Bidirectional Reflectance Distribution Function (SVBRDF) maps. 
It allows for fine-grained control over material synthesis through multiple conditioning sources like color palettes, sketches, text, and images. 
Additionally, it supports post-generation editing of materials.

For more details, visit the [project page](https://gvecchio.com/matfuse/) or read the full paper on [arXiv](https://arxiv.org/abs/2308.11408).

## 🧑‍💻 Usage

### 💿 Installation

1. Clone the repository:
    ```bash
    git clone https://github.com/giuvecchio/matfuse-sd.git
    cd matfuse-sd
    ```

2. Set up the environment:
    ```bash
    # create environment (can use venv instead of conda)
    conda create -n matfuse python==3.10.13
    conda activate matfuse
    # install required packages
    pip install -r requirements.txt
    ```
    
3. Download the checkpoint.

### 🧪 Inference

To run inference on a trained model:
```bash
python src/gradio_app.py --ckpt <path/to/checkpoint.ckpt> --config src/configs/diffusion/<config.yaml>
```

## 📜 Citation

```bibtex
@inproceedings{vecchio2024matfuse,
  author    = {Vecchio, Giuseppe and Sortino, Renato and Palazzo, Simone and Spampinato, Concetto},
  title     = {MatFuse: Controllable Material Generation with Diffusion Models},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2024},
  pages     = {4429-4438}
}
```

## License

This project is licensed under the MIT License.