fofr commited on
Commit
32d594a
1 Parent(s): 104f6ef

Upload folder using huggingface_hub

Browse files
Files changed (4) hide show
  1. README.md +77 -0
  2. embeddings.pti +0 -0
  3. lora.safetensors +3 -0
  4. special_params.json +1 -0
README.md ADDED
@@ -0,0 +1,77 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: creativeml-openrail-m
3
+ tags:
4
+ - text-to-image
5
+ - stable-diffusion
6
+ - lora
7
+ - diffusers
8
+ base_model: stabilityai/stable-diffusion-xl-base-1.0
9
+ pivotal_tuning: true
10
+ textual_embeddings: embeddings.pti
11
+ instance_prompt: <s0><s1>
12
+ inference: false
13
+ ---
14
+ # sdxl-black-light LoRA by [fofr](https://replicate.com/fofr)
15
+ ### SDXL fine-tuned on black light imagery
16
+
17
+ ![lora_image](https://pbxt.replicate.delivery/hbLNDuXFSPZLGRR5oUwQjpPXldpIuOycNnxVBVZjyfaIEfyRA/out-0.png)
18
+ >
19
+
20
+ ## Inference with Replicate API
21
+ Grab your replicate token [here](https://replicate.com/account)
22
+ ```bash
23
+ pip install replicate
24
+ export REPLICATE_API_TOKEN=r8_*************************************
25
+ ```
26
+
27
+ ```py
28
+ import replicate
29
+
30
+ output = replicate.run(
31
+ "sdxl-black-light@sha256:0b682d5744e86e988216141edd6a99be821941fd1a49a64786ad47fa48c33a95",
32
+ input={"prompt": "A closeup neon portrait photo of a spooky skeleton in the style of TOK"}
33
+ )
34
+ print(output)
35
+ ```
36
+ You may also do inference via the API with Node.js or curl, and locally with COG and Docker, [check out the Replicate API page for this model](https://replicate.com/fofr/sdxl-black-light/api)
37
+
38
+ ## Inference with 🧨 diffusers
39
+ Replicate SDXL LoRAs are trained with Pivotal Tuning, which combines training a concept via Dreambooth LoRA with training a new token with Textual Inversion.
40
+ As `diffusers` doesn't yet support textual inversion for SDXL, we will use cog-sdxl `TokenEmbeddingsHandler` class.
41
+
42
+ The trigger tokens for your prompt will be `<s0><s1>`
43
+
44
+ ```shell
45
+ pip install diffusers transformers accelerate safetensors huggingface_hub
46
+ git clone https://github.com/replicate/cog-sdxl cog_sdxl
47
+ ```
48
+
49
+ ```py
50
+ import torch
51
+ from huggingface_hub import hf_hub_download
52
+ from diffusers import DiffusionPipeline
53
+ from cog_sdxl.dataset_and_utils import TokenEmbeddingsHandler
54
+ from diffusers.models import AutoencoderKL
55
+
56
+ pipe = DiffusionPipeline.from_pretrained(
57
+ "stabilityai/stable-diffusion-xl-base-1.0",
58
+ torch_dtype=torch.float16,
59
+ variant="fp16",
60
+ ).to("cuda")
61
+
62
+ pipe.load_lora_weights("fofr/sdxl-black-light", weight_name="lora.safetensors")
63
+
64
+ text_encoders = [pipe.text_encoder, pipe.text_encoder_2]
65
+ tokenizers = [pipe.tokenizer, pipe.tokenizer_2]
66
+
67
+ embedding_path = hf_hub_download(repo_id="fofr/sdxl-black-light", filename="embeddings.pti", repo_type="model")
68
+ embhandler = TokenEmbeddingsHandler(text_encoders, tokenizers)
69
+ embhandler.load_embeddings(embedding_path)
70
+ prompt="A closeup neon portrait photo of a spooky skeleton in the style of <s0><s1>"
71
+ images = pipe(
72
+ prompt,
73
+ cross_attention_kwargs={"scale": 0.8},
74
+ ).images
75
+ #your output image
76
+ images[0]
77
+ ```
embeddings.pti ADDED
lora.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:d032411bb8bc72f762bd7656c7462338f71cb13041bdd47bb3914770f1782bb0
3
+ size 185968776
special_params.json ADDED
@@ -0,0 +1 @@
 
 
1
+ {"TOK": "<s0><s1>"}