Georgiy Grigorev commited on
Commit
dc4db28
1 Parent(s): c7d52e7

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +37 -0
README.md CHANGED
@@ -1,3 +1,40 @@
1
  ---
2
  license: openrail
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: openrail
3
  ---
4
+
5
+ Converted Canny SD 2.1-base model from https://huggingface.co/thibaud/controlnet-sd21/ to diffusers format.
6
+
7
+ Saved only ControlNet weights
8
+
9
+ Usage:
10
+ ```
11
+ from diffusers import StableDiffusionControlNetPipeline, ControlNetModel, DEISMultistepScheduler
12
+ import cv2
13
+ from PIL import Image
14
+ import numpy as np
15
+
16
+ pipe = StableDiffusionControlNetPipeline.from_pretrained(
17
+ "stabilityai/stable-diffusion-2-1-base",
18
+ safety_checker=None,
19
+ # revision='fp16',
20
+ # torch_dtype=torch.float16,
21
+ controlnet=ControlNetModel.from_pretrained("thepowefuldeez/sd21-controlnet-canny")
22
+ ).to('cuda')
23
+ pipe.scheduler = DEISMultistepScheduler.from_config(pipe.scheduler.config)
24
+
25
+ image = np.array(Image.open("10.png"))
26
+
27
+ low_threshold = 100
28
+ high_threshold = 200
29
+
30
+ image = cv2.Canny(image, low_threshold, high_threshold)
31
+ image = image[:, :, None]
32
+ image = np.concatenate([image, image, image], axis=2)
33
+ canny_image = Image.fromarray(image)
34
+
35
+ im = pipe(
36
+ "beautiful woman", image=canny_image, num_inference_steps=30,
37
+ negative_prompt="ugly, blurry, bad, deformed, bad anatomy",
38
+ generator=torch.manual_seed(42)
39
+ ).images[0]
40
+ ```