File size: 9,061 Bytes
b0554a3 1a99301 71e3c81 fb2272c 4cf3db2 d7c5680 a9dd4c2 e5f6606 71e3c81 a9dd4c2 71e3c81 484dc44 71e3c81 484dc44 71e3c81 484dc44 71e3c81 484dc44 71e3c81 484dc44 71e3c81 e84534f 71e3c81 d7c5680 71e3c81 484dc44 71e3c81 a9dd4c2 e84534f a9dd4c2 484dc44 d7c5680 e5f6606 71e3c81 484dc44 d7c5680 71e3c81 d7c5680 71e3c81 b0554a3 c4c0be8 b0554a3 1a99301 b0554a3 c66d0f1 cae9940 c66d0f1 cae9940 b0554a3 c864364 b0554a3 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 |
---
license: other
license_name: fair-ai-public-license-1.0-sd
license_link: https://freedevproject.org/faipl-1.0-sd/
datasets:
- KBlueLeaf/danbooru2023-webp-4Mpixel
- KBlueLeaf/danbooru2023-sqlite
language:
- en
library_name: diffusers
pipeline_tag: text-to-image
---
# Kohaku XL Zeta
join us: https://discord.gg/tPBsKDyRR5
<style>
.image-viewer {position: relative;width: 100%;margin: 0 auto;display: flex;flex-flow: wrap;align-items: center;justify-content: center;}
.image-viewer input[type="radio"] {display: none;}
.image-viewer label {border-radius: 10%;padding: 20px;background-color: #B398F5;background-size: cover;background-position: center;cursor: pointer;color: black;margin: 8px;}
.image-viewer label:hover {background-color: #4C88F5;padding: 24px;margin: 4px;}
.image-viewer input[type="radio"]:checked + label {background-color: #6296F5;padding: 28px;margin: 0px;}
.image-container {position: relative;width: 100%;height: 50vh;margin: 1rem 1rem 0 0;}
.inner-container {position: absolute;width:100%;height: 100%;display: flex;align-items: center;justify-content: center;}
.inner-container img {border-radius: 10px;max-height: 100%;max-width: 100%;height: 0;width: 0;opacity: 0;transition: opacity 0.5s ease, height 0.25s ease, width 0.25s ease;}
#image1:checked ~ .image-container img:nth-child(1),#image2:checked ~ .image-container img:nth-child(2),#image3:checked ~ .image-container img:nth-child(3),#image4:checked ~ .image-container img:nth-child(4),#image5:checked ~ .image-container img:nth-child(5),#image6:checked ~ .image-container img:nth-child(6),#image7:checked ~ .image-container img:nth-child(7),#image8:checked ~ .image-container img:nth-child(8),#image9:checked ~ .image-container img:nth-child(9),#image10:checked ~ .image-container img:nth-child(10),#image11:checked ~ .image-container img:nth-child(11),#image12:checked ~ .image-container img:nth-child(12) {height: auto; width:auto; opacity: 1;}
#image1l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/01_2085.jpg");}
#image2l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/02_02084.jpg");}
#image3l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/03_02086.jpg");}
#image4l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/04_02081.jpg");}
#image5l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/05_00015-3807569455.jpg");}
#image6l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/05_00096-1093286410.jpg");}
#image7l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/05_00117-2417076749.jpg");}
#image8l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/05_00118-2417076750.jpg");}
#image9l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/05_00123-2659559372.jpg");}
#image10l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/06_02082.jpg");}
#image11l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/06_02088.jpg");}
#image12l{background-image: url("/KBlueLeaf/Kohaku-XL-Zeta/resolve/main/sample-images/06_02091.jpg");}
</style>
<div class="image-viewer">
<input type="radio" id="image1" name="image-switcher" checked>
<label for="image1" id="image1l"></label>
<input type="radio" id="image2" name="image-switcher">
<label for="image2" id="image2l"></label>
<input type="radio" id="image3" name="image-switcher">
<label for="image3" id="image3l"></label>
<input type="radio" id="image4" name="image-switcher">
<label for="image4" id="image4l"></label>
<input type="radio" id="image5" name="image-switcher">
<label for="image5" id="image5l"></label>
<input type="radio" id="image6" name="image-switcher">
<label for="image6" id="image6l"></label>
<input type="radio" id="image7" name="image-switcher">
<label for="image7" id="image7l"></label>
<input type="radio" id="image8" name="image-switcher">
<label for="image8" id="image8l"></label>
<input type="radio" id="image9" name="image-switcher">
<label for="image9" id="image9l"></label>
<input type="radio" id="image10" name="image-switcher">
<label for="image10" id="image10l"></label>
<input type="radio" id="image11" name="image-switcher">
<label for="image11" id="image11l"></label>
<input type="radio" id="image12" name="image-switcher">
<label for="image12" id="image12l"></label>
<div class="image-container">
<div class="inner-container">
<img src="sample-images/01_2085.jpg" alt="image1" />
<img src="sample-images/02_02084.jpg" alt="image2" />
<img src="sample-images/03_02086.jpg" alt="image3" />
<img src="sample-images/04_02081.jpg" alt="image4" />
<img src="sample-images/05_00015-3807569455.jpg" alt="image5" />
<img src="sample-images/05_00096-1093286410.jpg" alt="image6" />
<img src="sample-images/05_00117-2417076749.jpg" alt="image7" />
<img src="sample-images/05_00118-2417076750.jpg" alt="image8" />
<img src="sample-images/05_00123-2659559372.jpg" alt="image9" />
<img src="sample-images/06_02082.jpg" alt="image10" />
<img src="sample-images/06_02088.jpg" alt="image11" />
<img src="sample-images/06_02091.jpg" alt="image12" />
</div>
</div>
</div>
---
## Highlights
- Resume from Kohaku-XL-Epsilon rev2
- More stable, long/detailed prompt is not a requirement now.
- Better fidelity on style and character, support more style.
- CCIP metric surpass Sanae XL anime. have over 2200 character with CCIP score > 0.9 in 3700 character set.
- Trained on both danbooru tags and natural language, better ability on nl caption.
- Trained on combined dataset, not only danbooru
- danbooru (7.6M images, last id 7832883, 2024/07/10)
- pixiv (filtered from 2.6M special set, will release the url set)
- pvc figure (around 30k images, internal source)
- realbooru (around 90k images, for regularization)
- 8.46M images in total
- Since the model is trained on both kind of caption, the ctx length limit is extended to 300.
## Usage (PLEASE READ THIS SECTION)
### Recommended Generation Settings
- resolution: 1024x1024 or similar pixel count
- cfg scale: 3.5~6.5
- sampler/scheduler:
- Euler (A) / any scheduler
- DPM++ series / exponential scheduler
- for other sampler, I personally recommend exponential scheduler.
- step: 12~50
### Prompt Gen
DTG series prompt gen can still be used on KXL zeta.
A brand new prompt gen for cooperating both tag and nl caption is under developing.
|||
|-|-|
### Prompt Format
As same as Kohaku XL Epsilon or Delta, but you can replace "general tags" with "natural language caption".
You can also put both together.
### Special Tags
- Quality tags: masterpiece, best quality, great quality, good quality, normal quality, low quality, worst quality
- Rating tags: safe, sensitive, nsfw, explicit
- Date tags: newest, recent, mid, early, old
#### Rating tags
General: safe
Sensitive: sensitive
Questionable: nsfw
Explicit: nsfw, explicit
## Dataset
For better ability on some certain concepts, I use full danbooru dataset instead of filterd one.
Than use crawled Pixiv dataset (from 3~5 tag with popularity sort) as addon dataset.
Since Pixiv's search system only allow 5000 page per tag so there is not much meaningful image, and some of them are duplicated with danbooru set(but since I want to reinforce these concept I directly ignore the duplication)
As same as kxl eps rev2, I add realbooru and pvc figure images for more flexibility on concept/style.
## Training
- Hardware: Quad RTX 3090s
- Dataset
- Num Images: 8,468,798
- Resolution: 1024x1024
- Min Bucket Resolution: 256
- Max Bucket Resolution: 4096
- Caption Tag Dropout: 0.2
- Caption Group Dropout: 0.2 (for dropping tag/nl caption entirely)
- Training
- Batch Size: 4
- Grad Accumulation Step: 32
- Equivalent Batch Size: 512
- Total Epoch: 1
- Total Steps: 16548
- Training Time: 430 hours (wall time)
- Mixed Precision: FP16
- Optimizer
- Optimizer: Lion8bit
- Learning Rate: 1e-5 for UNet / TE training disabled
- LR Scheduler: Constant (with warmup)
- Warmup Steps: 100
- Weight Decay: 0.1
- Betas: 0.9, 0.95
- Diffusion
- Min SNR Gamma: 5
- Debiased Estimation Loss: Enabled
- IP Noise Gamma: 0.05
## Why do you still use SDXL but not any Brand New DiT-Based Models?
Unless any one give me reasonable compute resources or any team release efficient enough DiT or I will not train any DiT-based anime base model. <br>
But if you give me 8xH100 for an year, I can even train lot of DiT from scratch (If you want)
## License:
Fair-AI-public-1.0-sd |