Update README.md
Browse files
README.md
CHANGED
@@ -1,3 +1,68 @@
|
|
1 |
---
|
2 |
-
license: openrail
|
|
|
|
|
|
|
|
|
|
|
3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
+
license: openrail++
|
3 |
+
tags:
|
4 |
+
- text-to-image
|
5 |
+
- stable-diffusion
|
6 |
+
library_name: diffusers
|
7 |
+
inference: false
|
8 |
---
|
9 |
+
|
10 |
+
# SDXS-512-DreamShaper
|
11 |
+
|
12 |
+
SDXS is a model that can generate high-resolution images in real-time based on prompt texts, trained using score distillation and feature matching.
|
13 |
+
For more information, please refer to our research paper: [SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions](https://arxiv.org/abs/2403.16627).
|
14 |
+
We open-source the model as part of the research.
|
15 |
+
|
16 |
+
SDXS-512-DreamShaper is the version we trained specifically for share.
|
17 |
+
The model is trained without focusing on FID, and sacrifices diversity for consistent image generation quality.
|
18 |
+
In order to avoid some possible risks, the SDXS-512-1.0 and SDXS-1024-1.0 will not be available shortly.
|
19 |
+
Watch [our repo](https://github.com/IDKiro/sdxs) for any updates.
|
20 |
+
|
21 |
+
Model Information:
|
22 |
+
- Teacher DM: [dreamshaper-8-lcm](https://huggingface.co/Lykon/dreamshaper-8-lcm)
|
23 |
+
- Offline DM: [dreamshaper-8](https://huggingface.co/Lykon/dreamshaper-8)
|
24 |
+
- VAE: [TAESD](https://huggingface.co/madebyollin/taesd)
|
25 |
+
|
26 |
+
Similar to SDXS-512-0.9, since our image decoder is not compatible with diffusers, we use TAESD.
|
27 |
+
Currently, our pull request has been merged in to reduce the gap between TAESD and our proprietary image decoder.
|
28 |
+
In the next diffusers release update, we may replace the image decoder.
|
29 |
+
|
30 |
+
## Diffusers Usage
|
31 |
+
|
32 |
+
![](output.png)
|
33 |
+
|
34 |
+
```python
|
35 |
+
import torch
|
36 |
+
from diffusers import StableDiffusionPipeline, AutoencoderKL
|
37 |
+
|
38 |
+
repo = "IDKiro/sdxs-512-dreamshaper"
|
39 |
+
seed = 42
|
40 |
+
weight_type = torch.float16 # or float32
|
41 |
+
|
42 |
+
# Load model.
|
43 |
+
pipe = StableDiffusionPipeline.from_pretrained(repo, torch_dtype=weight_type)
|
44 |
+
pipe.to("cuda")
|
45 |
+
|
46 |
+
prompt = "a close-up picture of an old man standing in the rain"
|
47 |
+
|
48 |
+
# Ensure using 1 inference step and CFG set to 0.
|
49 |
+
image = pipe(
|
50 |
+
prompt,
|
51 |
+
num_inference_steps=1,
|
52 |
+
guidance_scale=0,
|
53 |
+
generator=torch.Generator(device="cuda").manual_seed(seed)
|
54 |
+
).images[0]
|
55 |
+
|
56 |
+
image.save("output.png")
|
57 |
+
```
|
58 |
+
|
59 |
+
## Cite Our Work
|
60 |
+
|
61 |
+
```
|
62 |
+
@article{song2024sdxs,
|
63 |
+
author = {Yuda Song, Zehao Sun, Xuanwu Yin},
|
64 |
+
title = {SDXS: Real-Time One-Step Latent Diffusion Models with Image Conditions},
|
65 |
+
journal = {arxiv},
|
66 |
+
year = {2024},
|
67 |
+
}
|
68 |
+
```
|