Javier23VK commited on
Commit
f41fbf1
1 Parent(s): 4621659

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -215
README.md CHANGED
@@ -1,215 +1 @@
1
- ---
2
- license: openrail++
3
- tags:
4
- - text-to-image
5
- - stable-diffusion
6
- ---
7
- # SD-XL 1.0-base Model Card
8
- ![row01](01.png)
9
-
10
- ## Model
11
-
12
- ![pipeline](pipeline.png)
13
-
14
- [SDXL](https://arxiv.org/abs/2307.01952) consists of an [ensemble of experts](https://arxiv.org/abs/2211.01324) pipeline for latent diffusion:
15
- In a first step, the base model is used to generate (noisy) latents,
16
- which are then further processed with a refinement model (available here: https://huggingface.co/stabilityai/stable-diffusion-xl-refiner-1.0/) specialized for the final denoising steps.
17
- Note that the base model can be used as a standalone module.
18
-
19
- Alternatively, we can use a two-stage pipeline as follows:
20
- First, the base model is used to generate latents of the desired output size.
21
- In the second step, we use a specialized high-resolution model and apply a technique called SDEdit (https://arxiv.org/abs/2108.01073, also known as "img2img")
22
- to the latents generated in the first step, using the same prompt. This technique is slightly slower than the first one, as it requires more function evaluations.
23
-
24
- Source code is available at https://github.com/Stability-AI/generative-models .
25
-
26
- ### Model Description
27
-
28
- - **Developed by:** Stability AI
29
- - **Model type:** Diffusion-based text-to-image generative model
30
- - **License:** [CreativeML Open RAIL++-M License](https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/blob/main/LICENSE.md)
31
- - **Model Description:** This is a model that can be used to generate and modify images based on text prompts. It is a [Latent Diffusion Model](https://arxiv.org/abs/2112.10752) that uses two fixed, pretrained text encoders ([OpenCLIP-ViT/G](https://github.com/mlfoundations/open_clip) and [CLIP-ViT/L](https://github.com/openai/CLIP/tree/main)).
32
- - **Resources for more information:** Check out our [GitHub Repository](https://github.com/Stability-AI/generative-models) and the [SDXL report on arXiv](https://arxiv.org/abs/2307.01952).
33
-
34
- ### Model Sources
35
-
36
- For research purposes, we recommend our `generative-models` Github repository (https://github.com/Stability-AI/generative-models), which implements the most popular diffusion frameworks (both training and inference) and for which new functionalities like distillation will be added over time.
37
- [Clipdrop](https://clipdrop.co/stable-diffusion) provides free SDXL inference.
38
-
39
- - **Repository:** https://github.com/Stability-AI/generative-models
40
- - **Demo:** https://clipdrop.co/stable-diffusion
41
-
42
-
43
- ## Evaluation
44
- ![comparison](comparison.png)
45
- The chart above evaluates user preference for SDXL (with and without refinement) over SDXL 0.9 and Stable Diffusion 1.5 and 2.1.
46
- The SDXL base model performs significantly better than the previous variants, and the model combined with the refinement module achieves the best overall performance.
47
-
48
-
49
- ### 🧨 Diffusers
50
-
51
- Make sure to upgrade diffusers to >= 0.19.0:
52
- ```
53
- pip install diffusers --upgrade
54
- ```
55
-
56
- In addition make sure to install `transformers`, `safetensors`, `accelerate` as well as the invisible watermark:
57
- ```
58
- pip install invisible_watermark transformers accelerate safetensors
59
- ```
60
-
61
- To just use the base model, you can run:
62
-
63
- ```py
64
- from diffusers import DiffusionPipeline
65
- import torch
66
-
67
- pipe = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, use_safetensors=True, variant="fp16")
68
- pipe.to("cuda")
69
-
70
- # if using torch < 2.0
71
- # pipe.enable_xformers_memory_efficient_attention()
72
-
73
- prompt = "An astronaut riding a green horse"
74
-
75
- images = pipe(prompt=prompt).images[0]
76
- ```
77
-
78
- To use the whole base + refiner pipeline as an ensemble of experts you can run:
79
-
80
- ```py
81
- from diffusers import DiffusionPipeline
82
- import torch
83
-
84
- # load both base & refiner
85
- base = DiffusionPipeline.from_pretrained(
86
- "stabilityai/stable-diffusion-xl-base-1.0", torch_dtype=torch.float16, variant="fp16", use_safetensors=True
87
- )
88
- base.to("cuda")
89
- refiner = DiffusionPipeline.from_pretrained(
90
- "stabilityai/stable-diffusion-xl-refiner-1.0",
91
- text_encoder_2=base.text_encoder_2,
92
- vae=base.vae,
93
- torch_dtype=torch.float16,
94
- use_safetensors=True,
95
- variant="fp16",
96
- )
97
- refiner.to("cuda")
98
-
99
- # Define how many steps and what % of steps to be run on each experts (80/20) here
100
- n_steps = 40
101
- high_noise_frac = 0.8
102
-
103
- prompt = "A majestic lion jumping from a big stone at night"
104
-
105
- # run both experts
106
- image = base(
107
- prompt=prompt,
108
- num_inference_steps=n_steps,
109
- denoising_end=high_noise_frac,
110
- output_type="latent",
111
- ).images
112
- image = refiner(
113
- prompt=prompt,
114
- num_inference_steps=n_steps,
115
- denoising_start=high_noise_frac,
116
- image=image,
117
- ).images[0]
118
- ```
119
-
120
- When using `torch >= 2.0`, you can improve the inference speed by 20-30% with torch.compile. Simple wrap the unet with torch compile before running the pipeline:
121
- ```py
122
- pipe.unet = torch.compile(pipe.unet, mode="reduce-overhead", fullgraph=True)
123
- ```
124
-
125
- If you are limited by GPU VRAM, you can enable *cpu offloading* by calling `pipe.enable_model_cpu_offload`
126
- instead of `.to("cuda")`:
127
-
128
- ```diff
129
- - pipe.to("cuda")
130
- + pipe.enable_model_cpu_offload()
131
- ```
132
-
133
- For more information on how to use Stable Diffusion XL with `diffusers`, please have a look at [the Stable Diffusion XL Docs](https://huggingface.co/docs/diffusers/api/pipelines/stable_diffusion/stable_diffusion_xl).
134
-
135
- ### Optimum
136
- [Optimum](https://github.com/huggingface/optimum) provides a Stable Diffusion pipeline compatible with both [OpenVINO](https://docs.openvino.ai/latest/index.html) and [ONNX Runtime](https://onnxruntime.ai/).
137
-
138
- #### OpenVINO
139
-
140
- To install Optimum with the dependencies required for OpenVINO :
141
-
142
- ```bash
143
- pip install optimum[openvino]
144
- ```
145
-
146
- To load an OpenVINO model and run inference with OpenVINO Runtime, you need to replace `StableDiffusionXLPipeline` with Optimum `OVStableDiffusionXLPipeline`. In case you want to load a PyTorch model and convert it to the OpenVINO format on-the-fly, you can set `export=True`.
147
-
148
- ```diff
149
- - from diffusers import StableDiffusionXLPipeline
150
- + from optimum.intel import OVStableDiffusionXLPipeline
151
-
152
- model_id = "stabilityai/stable-diffusion-xl-base-1.0"
153
- - pipeline = StableDiffusionXLPipeline.from_pretrained(model_id)
154
- + pipeline = OVStableDiffusionXLPipeline.from_pretrained(model_id)
155
- prompt = "A majestic lion jumping from a big stone at night"
156
- image = pipeline(prompt).images[0]
157
- ```
158
-
159
- You can find more examples (such as static reshaping and model compilation) in optimum [documentation](https://huggingface.co/docs/optimum/main/en/intel/inference#stable-diffusion-xl).
160
-
161
-
162
- #### ONNX
163
-
164
- To install Optimum with the dependencies required for ONNX Runtime inference :
165
-
166
- ```bash
167
- pip install optimum[onnxruntime]
168
- ```
169
-
170
- To load an ONNX model and run inference with ONNX Runtime, you need to replace `StableDiffusionXLPipeline` with Optimum `ORTStableDiffusionXLPipeline`. In case you want to load a PyTorch model and convert it to the ONNX format on-the-fly, you can set `export=True`.
171
-
172
- ```diff
173
- - from diffusers import StableDiffusionXLPipeline
174
- + from optimum.onnxruntime import ORTStableDiffusionXLPipeline
175
-
176
- model_id = "stabilityai/stable-diffusion-xl-base-1.0"
177
- - pipeline = StableDiffusionXLPipeline.from_pretrained(model_id)
178
- + pipeline = ORTStableDiffusionXLPipeline.from_pretrained(model_id)
179
- prompt = "A majestic lion jumping from a big stone at night"
180
- image = pipeline(prompt).images[0]
181
- ```
182
-
183
- You can find more examples in optimum [documentation](https://huggingface.co/docs/optimum/main/en/onnxruntime/usage_guides/models#stable-diffusion-xl).
184
-
185
-
186
- ## Uses
187
-
188
- ### Direct Use
189
-
190
- The model is intended for research purposes only. Possible research areas and tasks include
191
-
192
- - Generation of artworks and use in design and other artistic processes.
193
- - Applications in educational or creative tools.
194
- - Research on generative models.
195
- - Safe deployment of models which have the potential to generate harmful content.
196
- - Probing and understanding the limitations and biases of generative models.
197
-
198
- Excluded uses are described below.
199
-
200
- ### Out-of-Scope Use
201
-
202
- The model was not trained to be factual or true representations of people or events, and therefore using the model to generate such content is out-of-scope for the abilities of this model.
203
-
204
- ## Limitations and Bias
205
-
206
- ### Limitations
207
-
208
- - The model does not achieve perfect photorealism
209
- - The model cannot render legible text
210
- - The model struggles with more difficult tasks which involve compositionality, such as rendering an image corresponding to “A red cube on top of a blue sphere”
211
- - Faces and people in general may not be generated properly.
212
- - The autoencoding part of the model is lossy.
213
-
214
- ### Bias
215
- While the capabilities of image generation models are impressive, they can also reinforce or exacerbate social biases.
 
1
+ Crea un póster promocional para un cortometraje titulado "BRIGTH ". El diseño debe ser [Animación 3D], usando una paleta de colores [Blanco, rojo, naranja, amarillo, negro]. Incluir una imagen clave de [Una vela con forma humana animado en 3D]. El título debe ser prominente con un lema debajo que diga "Bright". Agregar la fecha y lugar del estreno en la parte inferior, así como los créditos principales.