Commit
·
2133b8e
1
Parent(s):
d945009
Update README.md
Browse files
README.md
CHANGED
|
@@ -37,7 +37,7 @@ extra_gated_heading: Please read the LICENSE to access this model
|
|
| 37 |
|
| 38 |
# BK-SDM-2M Model Card
|
| 39 |
|
| 40 |
-
BK-SDM-{**Base-2M
|
| 41 |
- Block-removed Knowledge-distilled Stable Diffusion Model (BK-SDM) is an architecturally compressed SDM for efficient text-to-image synthesis.
|
| 42 |
- The previous BK-SDM-{[Base](https://huggingface.co/nota-ai/bk-sdm-base), [Small](https://huggingface.co/nota-ai/bk-sdm-small), [Tiny](https://huggingface.co/nota-ai/bk-sdm-tiny)} were obtained via distillation pretraining on 0.22M LAION pairs.
|
| 43 |
- Resources for more information: [Paper](https://arxiv.org/abs/2305.15798), [GitHub](https://github.com/Nota-NetsPresso/BK-SDM), [Demo]( https://huggingface.co/spaces/nota-ai/compressed-stable-diffusion).
|
|
@@ -75,13 +75,20 @@ Adhering to the [U-Net architecture](https://huggingface.co/nota-ai/bk-sdm-small
|
|
| 75 |
|
| 76 |
## Experimental Results
|
| 77 |
|
| 78 |
-
The following table shows the zero-shot results on 30K samples from the MS-COCO validation split. After generating 512×512 images with the PNDM scheduler and 25 denoising steps, we downsampled them to 256×256 for evaluating generation scores.
|
|
|
|
|
|
|
| 79 |
|
| 80 |
| Model | FID↓ | IS↑ | CLIP Score↑<br>(ViT-g/14) | # Params,<br>U-Net | # Params,<br>Whole SDM |
|
| 81 |
|:---:|:---:|:---:|:---:|:---:|:---:|
|
| 82 |
| [Stable Diffusion v1.4](https://huggingface.co/CompVis/stable-diffusion-v1-4) | 13.05 | 36.76 | 0.2958 | 0.86B | 1.04B |
|
|
|
|
|
|
|
| 83 |
| [BK-SDM-Small](https://huggingface.co/nota-ai/bk-sdm-small) (Ours) | 16.98 | 31.68 | 0.2677 | 0.49B | 0.66B |
|
| 84 |
-
| [
|
|
|
|
|
|
|
|
|
|
| 85 |
|
| 86 |
|
| 87 |
### Effect of Different Data Sizes for Training BK-SDM-Small
|
|
@@ -97,6 +104,12 @@ Furthermore, with the growth in data volume, visual results become more favorabl
|
|
| 97 |
<img alt="Visual results with different data sizes" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/assets-bk-sdm/fig_results_data_size.png" width="100%">
|
| 98 |
</center>
|
| 99 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 100 |
|
| 101 |
# Uses
|
| 102 |
Follow [the usage guidelines of Stable Diffusion v1](https://huggingface.co/CompVis/stable-diffusion-v1-4#uses).
|
|
@@ -128,4 +141,4 @@ Follow [the usage guidelines of Stable Diffusion v1](https://huggingface.co/Comp
|
|
| 128 |
}
|
| 129 |
```
|
| 130 |
|
| 131 |
-
*This model card was written by Bo-Kyeong Kim and is based on the [Stable Diffusion v1 model card]( https://huggingface.co/CompVis/stable-diffusion-v1-4).*
|
|
|
|
| 37 |
|
| 38 |
# BK-SDM-2M Model Card
|
| 39 |
|
| 40 |
+
BK-SDM-{[**Base-2M**](https://huggingface.co/nota-ai/bk-sdm-base-2m), [**Small-2M**](https://huggingface.co/nota-ai/bk-sdm-small-2m), [**Tiny-2M**](https://huggingface.co/nota-ai/bk-sdm-tiny-2m)} are pretrained with **10× more data** (2.3M LAION image-text pairs) compared to our previous release.
|
| 41 |
- Block-removed Knowledge-distilled Stable Diffusion Model (BK-SDM) is an architecturally compressed SDM for efficient text-to-image synthesis.
|
| 42 |
- The previous BK-SDM-{[Base](https://huggingface.co/nota-ai/bk-sdm-base), [Small](https://huggingface.co/nota-ai/bk-sdm-small), [Tiny](https://huggingface.co/nota-ai/bk-sdm-tiny)} were obtained via distillation pretraining on 0.22M LAION pairs.
|
| 43 |
- Resources for more information: [Paper](https://arxiv.org/abs/2305.15798), [GitHub](https://github.com/Nota-NetsPresso/BK-SDM), [Demo]( https://huggingface.co/spaces/nota-ai/compressed-stable-diffusion).
|
|
|
|
| 75 |
|
| 76 |
## Experimental Results
|
| 77 |
|
| 78 |
+
The following table shows the zero-shot results on 30K samples from the MS-COCO validation split. After generating 512×512 images with the PNDM scheduler and 25 denoising steps, we downsampled them to 256×256 for evaluating generation scores.
|
| 79 |
+
|
| 80 |
+
- Our models were drawn at the 50K-th training iteration.
|
| 81 |
|
| 82 |
| Model | FID↓ | IS↑ | CLIP Score↑<br>(ViT-g/14) | # Params,<br>U-Net | # Params,<br>Whole SDM |
|
| 83 |
|:---:|:---:|:---:|:---:|:---:|:---:|
|
| 84 |
| [Stable Diffusion v1.4](https://huggingface.co/CompVis/stable-diffusion-v1-4) | 13.05 | 36.76 | 0.2958 | 0.86B | 1.04B |
|
| 85 |
+
| [BK-SDM-Base](https://huggingface.co/nota-ai/bk-sdm-base) (Ours) | 15.76 | 33.79 | 0.2878 | 0.58B | 0.76B |
|
| 86 |
+
| [BK-SDM-Base-2M](https://huggingface.co/nota-ai/bk-sdm-base-2m) (Ours) | 14.81 | 34.17 | 0.2883 | 0.58B | 0.76B |
|
| 87 |
| [BK-SDM-Small](https://huggingface.co/nota-ai/bk-sdm-small) (Ours) | 16.98 | 31.68 | 0.2677 | 0.49B | 0.66B |
|
| 88 |
+
| [BK-SDM-Small-2M](https://huggingface.co/nota-ai/bk-sdm-small-2m) (Ours) | 17.05 | 33.10 | 0.2734 | 0.49B | 0.66B |
|
| 89 |
+
| [BK-SDM-Tiny](https://huggingface.co/nota-ai/bk-sdm-tiny) (Ours) | 17.12 | 30.09 | 0.2653 | 0.33B | 0.50B |
|
| 90 |
+
| [BK-SDM-Tiny-2M](https://huggingface.co/nota-ai/bk-sdm-tiny-2m) (Ours) | 17.53 | 31.32 | 0.2690 | 0.33B | 0.50B |
|
| 91 |
+
|
| 92 |
|
| 93 |
|
| 94 |
### Effect of Different Data Sizes for Training BK-SDM-Small
|
|
|
|
| 104 |
<img alt="Visual results with different data sizes" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/assets-bk-sdm/fig_results_data_size.png" width="100%">
|
| 105 |
</center>
|
| 106 |
|
| 107 |
+
### Additional Visual Examples
|
| 108 |
+
|
| 109 |
+
<center>
|
| 110 |
+
<img alt="additional visual examples" img src="https://netspresso-research-code-release.s3.us-east-2.amazonaws.com/assets-bk-sdm/fig_results_models_2m.png" width="100%">
|
| 111 |
+
</center>
|
| 112 |
+
|
| 113 |
|
| 114 |
# Uses
|
| 115 |
Follow [the usage guidelines of Stable Diffusion v1](https://huggingface.co/CompVis/stable-diffusion-v1-4#uses).
|
|
|
|
| 141 |
}
|
| 142 |
```
|
| 143 |
|
| 144 |
+
*This model card was written by Bo-Kyeong Kim and is based on the [Stable Diffusion v1 model card]( https://huggingface.co/CompVis/stable-diffusion-v1-4).*
|