IDKiro commited on
Commit
6089b7d
1 Parent(s): 727e1e2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +66 -1
README.md CHANGED
@@ -1,3 +1,68 @@
1
  ---
2
- license: openrail
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ license: openrail++
3
+ tags:
4
+ - text-to-image
5
+ - stable-diffusion
6
+ library_name: diffusers
7
+ inference: false
8
  ---
9
+
10
+ # SDXS-512-DreamShaper
11
+
12
+ SDXS is a model that can generate high-resolution images in real-time based on prompt texts, trained using score distillation and feature matching.
13
+ For more information, please refer to our research paper: [SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions](https://arxiv.org/abs/2403.16627).
14
+ We open-source the model as part of the research.
15
+
16
+ SDXS-512-DreamShaper is the version we trained specifically for share.
17
+ The model is trained without focusing on FID, and sacrifices diversity for consistent image generation quality.
18
+ In order to avoid some possible risks, the SDXS-512-1.0 and SDXS-1024-1.0 will not be available shortly.
19
+ Watch [our repo](https://github.com/IDKiro/sdxs) for any updates.
20
+
21
+ Model Information:
22
+ - Teacher DM: [dreamshaper-8-lcm](https://huggingface.co/Lykon/dreamshaper-8-lcm)
23
+ - Offline DM: [dreamshaper-8](https://huggingface.co/Lykon/dreamshaper-8)
24
+ - VAE: [TAESD](https://huggingface.co/madebyollin/taesd)
25
+
26
+ Similar to SDXS-512-0.9, since our image decoder is not compatible with diffusers, we use TAESD.
27
+ Currently, our pull request has been merged in to reduce the gap between TAESD and our proprietary image decoder.
28
+ In the next diffusers release update, we may replace the image decoder.
29
+
30
+ ## Diffusers Usage
31
+
32
+ ![](output.png)
33
+
34
+ ```python
35
+ import torch
36
+ from diffusers import StableDiffusionPipeline, AutoencoderKL
37
+
38
+ repo = "IDKiro/sdxs-512-dreamshaper"
39
+ seed = 42
40
+ weight_type = torch.float16 # or float32
41
+
42
+ # Load model.
43
+ pipe = StableDiffusionPipeline.from_pretrained(repo, torch_dtype=weight_type)
44
+ pipe.to("cuda")
45
+
46
+ prompt = "a close-up picture of an old man standing in the rain"
47
+
48
+ # Ensure using 1 inference step and CFG set to 0.
49
+ image = pipe(
50
+ prompt,
51
+ num_inference_steps=1,
52
+ guidance_scale=0,
53
+ generator=torch.Generator(device="cuda").manual_seed(seed)
54
+ ).images[0]
55
+
56
+ image.save("output.png")
57
+ ```
58
+
59
+ ## Cite Our Work
60
+
61
+ ```
62
+ @article{song2024sdxs,
63
+ author = {Yuda Song, Zehao Sun, Xuanwu Yin},
64
+ title = {SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions},
65
+ journal = {arxiv},
66
+ year = {2024},
67
+ }
68
+ ```