radames/sdxl-turbo-DPO-LoRA

DPO LoRA Stable Diffusion XL Turbo

Model trained with LoRA implementation of Diffusion DPO Read more here

Base Model: https://huggingface.co./stabilityai/sdxl-turbo

Running with 🧨 diffusers library

from diffusers import DiffusionPipeline
from diffusers.utils import make_image_grid
import torch

pipe = DiffusionPipeline.from_pretrained(
    "stabilityai/sdxl-turbo",
    torch_dtype=torch.float16, variant="fp16"
)
pipe.to("cuda")
pipe.load_lora_weights("radames/sdxl-turbo-DPO-LoRA", adapter_name="dpo-lora-sdxl-turbo")
pipe.set_adapters(["dpo-lora-sdxl-turbo"], adapter_weights=[1.0]) # you can play with adapter_weights to increase the effect of the LoRA model
seed = 123123
prompt = " A photo of beautiful mountain with realistic sunset and blue lake, highly detailed, masterpiece"
negative_prompt = "3d render, cartoon, drawing, art, low light, blur, pixelated, low resolution, black and white, old photo, blurry faces"
generator = torch.Generator().manual_seed(seed)
images = pipe(
    prompt=prompt,
    negative_prompt=negative_prompt,
    width=512,
    height=512,
    num_inference_steps=2,
    generator=generator,
    guidance_scale=1.0,
    num_images_per_prompt=4
).images
make_image_grid(images, 1, 4)

Guidance Scale vs LoRA weights

Examples

Left Withoud DPO right with DPO LoRA

ComfyUI

https://huggingface.co./radames/sdxl-turbo-DPO-LoRA/raw/main/comfyui-workflow-sdxl-turbo-lora-dpo.json