---
language:
- en
license: creativeml-openrail-m
library_name: diffusers
tags:
- art
- diffusion
- Interior
---

# KuJiaLe Layout ControlNet
The models are not permitted for commercial usage. For inquiries regarding business, commercial licensing, custom models, and consultation, please contact [shuer@qunhemail.com](mailto:shuer@qunhemail.com).

The model is trained on [runwayml/stable-diffusion-v1-5](https://huggingface.co./runwayml/stable-diffusion-v1-5) for interior designs.

### Layout ControlNet Example
Keep the room layout consistent, re-furnish the room.
<table style="table-layout: fixed;">
  <tr>
    <td style="text-align: center; vertical-align: middle; width: 50%"> <img src="https://huggingface.co./kujiale-ai/controlnet-layout/resolve/main/examples/layout_input.jpg"  alt="Input" width="100%" style="display: block; margin-left: auto; margin-right: auto;"></td>
    <td style="text-align: center; vertical-align: middle; width: 50%"> <img src="https://huggingface.co./kujiale-ai/controlnet-layout/resolve/main/examples/layout_output.jpg" alt="Output" width="100%" style="display: block; margin-left: auto; margin-right: auto;"></td>
   </tr> 
   <tr>
      <td style="text-align: center; vertical-align: middle; width: 50%">Input</td>
      <td style="text-align: center; vertical-align: middle; width: 50%">Output</td>
  </td>
  </tr>
</table>

## News🔥🔥🔥

* May.30, 2024. Our checkpoint Layout-ControlNet are publicly available on [HuggingFace Repo](https://huggingface.co./kujiale-ai/controlnet-layout).

<!-- ## Try our Hugging Face demos:  -->

## Checkpoints
* `control_v1_sd15_layout_fp16`: Layout ControlNet checkpoint, for SD15 models.

## Using in 🧨 diffusers
### Layout ControlNet
```python
import torch
from diffusers.utils import load_image
import numpy as np
from diffusers import ControlNetModel, StableDiffusionControlNetPipeline, UniPCMultistepScheduler

controlnet_checkpoint = "kujiale-ai/controlnet-layout"
# Load original image
image = load_image("https://huggingface.co./kujiale-ai/controlnet-layout/resolve/main/examples/layout_input.jpg")
depth_image = load_image("https://huggingface.co./kujiale-ai/controlnet-layout/resolve/main/examples/layout_depth.jpg").convert("L")
normal_image = load_image("https://huggingface.co./kujiale-ai/controlnet-layout/resolve/main/examples/layout_normal.jpg")
segm_image = load_image("https://huggingface.co./kujiale-ai/controlnet-layout/resolve/main/examples/layout_segm.jpg")
W, H = image.size
depth_image = depth_image.resize((W, H))
normal_image = normal_image.resize((W, H))
segm_image = segm_image.resize((W, H))
# Prepare Layout Control Image
depth_image = np.array(depth_image, dtype=np.float32) / 255.0
depth_image = torch.from_numpy(depth_image[:, :, None])[None].permute(0, 3, 1, 2)
normal_image = np.array(normal_image, dtype=np.float32)
normal_image = normal_image / 127.5 - 1.0
normal_image = torch.from_numpy(normal_image)[None].permute(0, 3, 1, 2)
segm_image = np.array(segm_image, dtype=np.float32) / 255.0
segm_image = torch.from_numpy(segm_image)[None].permute(0, 3, 1, 2)
control_image = torch.cat([depth_image, normal_image, segm_image], dim=1)
# Initialize pipeline
controlnet = ControlNetModel.from_pretrained(controlnet_checkpoint, subfolder="control_v1_sd15_layout_fp16", torch_dtype=torch.float16)
pipe = StableDiffusionControlNetPipeline.from_pretrained("runwayml/stable-diffusion-v1-5", controlnet=controlnet, torch_dtype=torch.float16).to("cuda")
pipe.scheduler = UniPCMultistepScheduler.from_config(pipe.scheduler.config)
image = pipe("A modern bedroom,best quality", num_inference_steps=30, image=control_image, guidance_scale=7).images[0]
image.save('layout_output.jpg')
```