MysteryGuitarMan
commited on
Commit
•
2b7de13
1
Parent(s):
a47a1fd
Update README.md
Browse files
README.md
CHANGED
@@ -8,13 +8,9 @@ tags:
|
|
8 |
|
9 |
|
10 |
## Introduction
|
11 |
-
|
12 |
|
13 |
-
|
14 |
-
|
15 |
-
Integrating the strengths of both ControlNet and PEFT, this approach offers a more efficient and compact method to bring model control for a wider variety of consumer GPUs.
|
16 |
-
|
17 |
-
For each model below, you'll find `Rank 256` files (reducing the `~4.7GB` ControlNets to `~738MB`) and experimental, ultra-pruned `Rank 128` files (reducing to `~377MB`).
|
18 |
|
19 |
Each Control-LoRA has been trained on a diverse range of image concepts and aspect ratios.
|
20 |
### MiDaS and ClipDrop Depth
|
@@ -22,15 +18,15 @@ Each Control-LoRA has been trained on a diverse range of image concepts and aspe
|
|
22 |
|
23 |
Depth estimation is an image processing technique that determines the distance of objects in a scene, providing a depth map that highlights variations in proximity.
|
24 |
|
25 |
-
In the example above, we compare the depth results of MiDaS dpt_beit_large_512 with ClipDrop Depth for portraits, and their subsequent use in Depth Control-LoRa.
|
26 |
-
|
27 |
The Control-LoRA utilizes a grayscale depth map for guided generation.
|
28 |
|
|
|
|
|
29 |
### Canny Edge
|
30 |
![canny](samples/canny-sample.jpeg)
|
31 |
Canny Edge Detection is an image processing technique that identifies abrupt changes in intensity to highlight edges in an image.
|
32 |
|
33 |
-
This Control-LoRA uses the edges from an image to
|
34 |
|
35 |
### Photograph and Sketch Colorizer
|
36 |
![photograph colorizer](samples/colorizer-sample.jpeg)
|
|
|
8 |
|
9 |
|
10 |
## Introduction
|
11 |
+
By adding low-rank parameter efficient fine tuning to ControlNet, we introduce ***Control-LoRAs***. This approach offers a more efficient and compact method to bring model control to a wider variety of consumer GPUs.
|
12 |
|
13 |
+
For each model below, you'll find `Rank 256` files (reducing the `~4.7GB` ControlNets to `~738MB`) and experimental, `Rank 128` files (reduced to `~377MB`).
|
|
|
|
|
|
|
|
|
14 |
|
15 |
Each Control-LoRA has been trained on a diverse range of image concepts and aspect ratios.
|
16 |
### MiDaS and ClipDrop Depth
|
|
|
18 |
|
19 |
Depth estimation is an image processing technique that determines the distance of objects in a scene, providing a depth map that highlights variations in proximity.
|
20 |
|
|
|
|
|
21 |
The Control-LoRA utilizes a grayscale depth map for guided generation.
|
22 |
|
23 |
+
In the example above, we compare the depth results of `MiDaS dpt_beit_large_512` and the `Portrait Depth Estimation` (available in the [ClipDrop API by Stability AI](https://clipdrop.co/apis/docs/portrait-depth-estimation)).
|
24 |
+
|
25 |
### Canny Edge
|
26 |
![canny](samples/canny-sample.jpeg)
|
27 |
Canny Edge Detection is an image processing technique that identifies abrupt changes in intensity to highlight edges in an image.
|
28 |
|
29 |
+
This Control-LoRA uses the edges from an image to generate the final image.
|
30 |
|
31 |
### Photograph and Sketch Colorizer
|
32 |
![photograph colorizer](samples/colorizer-sample.jpeg)
|