Full Training Tutorial and Guide and Research For a FLUX Style
•
3
Furkan Gözükara
MonsterMMORPG
AI & ML interests
Check out my youtube page SECourses for Stable Diffusion tutorials. They will help you tremendously in every topic
Articles
Organizations
Posts
34
Post
785
How to Extract LoRA from FLUX Fine Tuning / DreamBooth Training Full Tutorial and Comparison Between Fine Tuning vs Extraction vs LoRA Training
Full article is here public post : https://www.patreon.com/posts/112335162
This was short on length so check out the full article - public post
Conclusions as below
Conclusions
With same training dataset (15 images used), same number of steps (all compared trainings are 150 epoch thus 2250 steps), almost same training duration, Fine Tuning / DreamBooth training of FLUX yields the very best results
So yes Fine Tuning is the much better than LoRA training itself
Amazing resemblance, quality with least amount of overfitting issue
Moreover, extracting a LoRA from Fine Tuned full checkpoint, yields way better results from LoRA training itself
Extracting LoRA from full trained checkpoints were yielding way better results in SD 1.5 and SDXL as well
Comparison of these 3 is made in Image 5 (check very top of the images to see)
640 Network Dimension (Rank) FP16 LoRA takes 6.1 GB disk space
You can also try 128 Network Dimension (Rank) FP16 and different LoRA strengths during inference to make it closer to Fine Tuned model
Moreover, you can try Resize LoRA feature of Kohya GUI but hopefully it will be my another research and article later
Image Raw Links
Image 1 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 2 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 3 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 4 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 5 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Full article is here public post : https://www.patreon.com/posts/112335162
This was short on length so check out the full article - public post
Conclusions as below
Conclusions
With same training dataset (15 images used), same number of steps (all compared trainings are 150 epoch thus 2250 steps), almost same training duration, Fine Tuning / DreamBooth training of FLUX yields the very best results
So yes Fine Tuning is the much better than LoRA training itself
Amazing resemblance, quality with least amount of overfitting issue
Moreover, extracting a LoRA from Fine Tuned full checkpoint, yields way better results from LoRA training itself
Extracting LoRA from full trained checkpoints were yielding way better results in SD 1.5 and SDXL as well
Comparison of these 3 is made in Image 5 (check very top of the images to see)
640 Network Dimension (Rank) FP16 LoRA takes 6.1 GB disk space
You can also try 128 Network Dimension (Rank) FP16 and different LoRA strengths during inference to make it closer to Fine Tuned model
Moreover, you can try Resize LoRA feature of Kohya GUI but hopefully it will be my another research and article later
Image Raw Links
Image 1 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 2 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 3 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 4 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Image 5 : MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Post
3172
Full Fine Tuning of FLUX yields way better results than LoRA training as expected, overfitting and bleeding reduced a lot
Configs and Full Experiments
Full configs and grid files shared here : https://www.patreon.com/posts/kohya-flux-fine-112099700
Details
I am still rigorously testing different hyperparameters and comparing impact of each one to find the best workflow
So far done 16 different full trainings and completing 8 more at the moment
I am using my poor overfit 15 images dataset for experimentation (4th image)
I have already proven that when I use a better dataset it becomes many times betters and generate expressions perfectly
Here example case : https://www.reddit.com/r/FluxAI/comments/1ffz9uc/tried_expressions_with_flux_lora_training_with_my/
Conclusions
When the results are analyzed, Fine Tuning is way lesser overfit and more generalized and better quality
In first 2 images, it is able to change hair color and add beard much better, means lesser overfit
In the third image, you will notice that the armor is much better, thus lesser overfit
I noticed that the environment and clothings are much lesser overfit and better quality
Disadvantages
Kohya still doesn’t have FP8 training, thus 24 GB GPUs gets a huge speed drop
Moreover, 48 GB GPUs has to use Fused Back Pass optimization, thus have some speed drop
16 GB GPUs gets way more aggressive speed drop due to lack of FP8
Clip-L and T5 trainings still not supported
Speeds
Rank 1 Fast Config — uses 27.5 GB VRAM, 6.28 second / it (LoRA is 4.85 second / it)
Rank 1 Slower Config — uses 23.1 GB VRAM, 14.12 second / it (LoRA is 4.85 second / it)
Rank 1 Slowest Config — uses 15.5 GB VRAM, 39 second / it (LoRA is 6.05 second / it)
Final Info
Saved checkpoints are FP16 and thus 23.8 GB (no Clip-L or T5 trained)
According to the Kohya, applied optimizations doesn’t change quality so all configs are ranked as Rank 1 at the moment
I am still testing whether these optimizations make any impact on quality or not
Configs and Full Experiments
Full configs and grid files shared here : https://www.patreon.com/posts/kohya-flux-fine-112099700
Details
I am still rigorously testing different hyperparameters and comparing impact of each one to find the best workflow
So far done 16 different full trainings and completing 8 more at the moment
I am using my poor overfit 15 images dataset for experimentation (4th image)
I have already proven that when I use a better dataset it becomes many times betters and generate expressions perfectly
Here example case : https://www.reddit.com/r/FluxAI/comments/1ffz9uc/tried_expressions_with_flux_lora_training_with_my/
Conclusions
When the results are analyzed, Fine Tuning is way lesser overfit and more generalized and better quality
In first 2 images, it is able to change hair color and add beard much better, means lesser overfit
In the third image, you will notice that the armor is much better, thus lesser overfit
I noticed that the environment and clothings are much lesser overfit and better quality
Disadvantages
Kohya still doesn’t have FP8 training, thus 24 GB GPUs gets a huge speed drop
Moreover, 48 GB GPUs has to use Fused Back Pass optimization, thus have some speed drop
16 GB GPUs gets way more aggressive speed drop due to lack of FP8
Clip-L and T5 trainings still not supported
Speeds
Rank 1 Fast Config — uses 27.5 GB VRAM, 6.28 second / it (LoRA is 4.85 second / it)
Rank 1 Slower Config — uses 23.1 GB VRAM, 14.12 second / it (LoRA is 4.85 second / it)
Rank 1 Slowest Config — uses 15.5 GB VRAM, 39 second / it (LoRA is 6.05 second / it)
Final Info
Saved checkpoints are FP16 and thus 23.8 GB (no Clip-L or T5 trained)
According to the Kohya, applied optimizations doesn’t change quality so all configs are ranked as Rank 1 at the moment
I am still testing whether these optimizations make any impact on quality or not
models
58
MonsterMMORPG/mich
Updated
MonsterMMORPG/FLUX-Fine-Tuning-Grid-Tests
Updated
MonsterMMORPG/YoloModels
Updated
MonsterMMORPG/Best_FLUX_LoRA_Checkpoints
Updated
MonsterMMORPG/256_Imgs_FLUX_LoRA_Trainings_Best
Updated
MonsterMMORPG/OneTrainer_FLUX_Training
Updated
MonsterMMORPG/256_images_tests
Updated
MonsterMMORPG/SECourses
Updated
•
38
MonsterMMORPG/FLUX_Kohya_SS_Massive_Research_Part6_T5_XXL
Updated
MonsterMMORPG/Generative-AI
Updated
•
3