Update README.md
Browse files
README.md
CHANGED
|
@@ -69,7 +69,10 @@ Training logs are available on WandB [here](https://wandb.ai/diffusion-guidance/
|
|
| 69 |
|
| 70 |
## LoRA
|
| 71 |
|
| 72 |
-
We extracted a 64-rank LoRA from the finetuned checkpoint (script here). This LoRA can be used to emulate the same kind of effect:
|
|
|
|
|
|
|
|
|
|
| 73 |
|
| 74 |
```py
|
| 75 |
from diffusers import DiffusionPipeline
|
|
@@ -93,4 +96,62 @@ video = pipeline(
|
|
| 93 |
num_inference_steps=50
|
| 94 |
).frames[0]
|
| 95 |
export_to_video(video, "output_lora.mp4", fps=25)
|
| 96 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 69 |
|
| 70 |
## LoRA
|
| 71 |
|
| 72 |
+
We extracted a 64-rank LoRA from the finetuned checkpoint (script [here](./create_lora.py)). [This LoRA](./extracted_cakeify_lora_64.safetensors) can be used to emulate the same kind of effect:
|
| 73 |
+
|
| 74 |
+
<details>
|
| 75 |
+
<summary>Code</summary>
|
| 76 |
|
| 77 |
```py
|
| 78 |
from diffusers import DiffusionPipeline
|
|
|
|
| 96 |
num_inference_steps=50
|
| 97 |
).frames[0]
|
| 98 |
export_to_video(video, "output_lora.mp4", fps=25)
|
| 99 |
+
```
|
| 100 |
+
|
| 101 |
+
</details>
|
| 102 |
+
|
| 103 |
+
Below is a comparison between the LoRA and non-LoRA outputs (under same settings and seed):
|
| 104 |
+
|
| 105 |
+
<table>
|
| 106 |
+
<thead>
|
| 107 |
+
<tr>
|
| 108 |
+
<th>Full finetune</th>
|
| 109 |
+
<th>LoRA</th>
|
| 110 |
+
</tr>
|
| 111 |
+
</thead>
|
| 112 |
+
<tbody>
|
| 113 |
+
<tr>
|
| 114 |
+
<td>
|
| 115 |
+
<video width="320" height="240" controls>
|
| 116 |
+
<source src="https://huggingface.co/finetrainers/cakeify-v0/resolve/main/comparisons/original_output_0.mp4" type="video/mp4">
|
| 117 |
+
Your browser does not support the video tag.
|
| 118 |
+
</video>
|
| 119 |
+
</td>
|
| 120 |
+
<td>
|
| 121 |
+
<video width="320" height="240" controls>
|
| 122 |
+
<source src="https://huggingface.co/finetrainers/cakeify-v0/resolve/main/comparisons/output_0.mp4" type="video/mp4">
|
| 123 |
+
Your browser does not support the video tag.
|
| 124 |
+
</video>
|
| 125 |
+
</td>
|
| 126 |
+
</tr>
|
| 127 |
+
<tr>
|
| 128 |
+
<td>
|
| 129 |
+
<video width="320" height="240" controls>
|
| 130 |
+
<source src="https://huggingface.co/finetrainers/cakeify-v0/resolve/main/comparisons/original_output_1.mp4" type="video/mp4">
|
| 131 |
+
Your browser does not support the video tag.
|
| 132 |
+
</video>
|
| 133 |
+
</td>
|
| 134 |
+
<td>
|
| 135 |
+
<video width="320" height="240" controls>
|
| 136 |
+
<source src="https://huggingface.co/finetrainers/cakeify-v0/resolve/main/comparisons/output_1.mp4" type="video/mp4">
|
| 137 |
+
Your browser does not support the video tag.
|
| 138 |
+
</video>
|
| 139 |
+
</td>
|
| 140 |
+
</tr>
|
| 141 |
+
<tr>
|
| 142 |
+
<td>
|
| 143 |
+
<video width="320" height="240" controls>
|
| 144 |
+
<source src="https://huggingface.co/finetrainers/cakeify-v0/resolve/main/comparisons/original_output_2.mp4" type="video/mp4">
|
| 145 |
+
Your browser does not support the video tag.
|
| 146 |
+
</video>
|
| 147 |
+
</td>
|
| 148 |
+
<td>
|
| 149 |
+
<video width="320" height="240" controls>
|
| 150 |
+
<source src="https://huggingface.co/finetrainers/cakeify-v0/resolve/main/comparisons/output_2.mp4" type="video/mp4">
|
| 151 |
+
Your browser does not support the video tag.
|
| 152 |
+
</video>
|
| 153 |
+
</td>
|
| 154 |
+
</tr>
|
| 155 |
+
</tbody>
|
| 156 |
+
</table>
|
| 157 |
+
|