Hugging Face's logo

AlekseyCalvin
/
StabledHSTorY_SD3.5_LoRA_V1

Text-to-Image
Diffusers
English
sd3
lora
template:sd-lora
StableDiffusion3
StableDiffusion3.5
ai-toolkit
Model card Files Community
StabledHSTorY_SD3.5_LoRA_V1 / README.md
AlekseyCalvin's picture
AlekseyCalvin
Update README.md
085002f verified
preview code
|
raw
history blame
9.9 kB
metadata
tags:
  - text-to-image
  - sd3
  - lora
  - diffusers
  - template:sd-lora
  - StableDiffusion3
  - StableDiffusion3.5
  - ai-toolkit
widget:
  - text: >-
      /@step 4000 weights:/ HST style autochrome photo of a drugged God
      Christ-insect in wooden pillbox being pinned needle-poked by a giant
      Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD
      realistic film in Yury Annenkov & David Lynch styles, detailed face
      worrying, full-height red-robed woman roams European city circa 1930
    output:
      url: samples/1729711624663__000004000_4.jpg
  - text: >-
      /@step 1600 weights:/HST style autochrome photo of a drugged God
      Christ-insect in wooden pillbox being pinned needle-poked by a giant
      Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD
      realistic film in Yury Annenkov & David Lynch styles, detailed face
      worrying, full-height red-robed woman roams European city circa 1930
    output:
      url: samples/1729700659095__000001600_4.jpg
  - text: >-
      /@step 1800 weights:/HST style autochrome photo of a drugged God
      Christ-insect in wooden pillbox being pinned needle-poked by a giant
      Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD
      realistic film in Yury Annenkov & David Lynch styles, detailed face
      worrying, full-height red-robed woman roams European city circa 1930
    output:
      url: samples/1729701571749__000001800_4.jpg
  - text: >-
      /@step 800 weights:/HHST style autochrome photo of a drugged God
      Christ-insect in wooden pillbox being pinned needle-poked by a giant
      Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD
      realistic film in Yury Annenkov & David Lynch styles, detailed face
      worrying, full-height red-robed woman roams European city circa 1930
    output:
      url: samples/1729696985671__000000800_4.jpg
  - text: >-
      /@step 1800 weights:/HST style communist poster with text "JOIN RCA!",
      over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916
      Zurich,  dancing with red feathered drunken dinosaur, an early conceptual
      artist. Lenin is full of contageous awe, his blemished skin flushing  with
      anxious excitement, his famous bald spot sweatily glistening under warm
      lights. In the back, Krupskaya and Inessa Armand laugh. 
    output:
      url: samples/1729701488064__000001800_0.jpg
  - text: >-
      /@step 4000 weights:/HHST style communist poster with text "JOIN RCA!",
      over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916
      Zurich,  dancing with red feathered drunken dinosaur, an early conceptual
      artist. Lenin is full of contageous awe, his blemished skin flushing  with
      anxious excitement, his famous bald spot sweatily glistening under warm
      lights. In the back, Krupskaya and Inessa Armand laugh. 
    output:
      url: samples/1729711540947__000004000_0.jpg
  - text: >-
      /@step 1600 weights:/ HST style communist poster with text "JOIN RCA!",
      over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916
      Zurich,  dancing with red feathered drunken dinosaur, an early conceptual
      artist. Lenin is full of contageous awe, his blemished skin flushing  with
      anxious excitement, his famous bald spot sweatily glistening under warm
      lights. In the back, Krupskaya and Inessa Armand laugh. 
    output:
      url: samples/1729700575417__000001600_0.jpg
  - text: >-
      /@step 1600 weights:/ HST autochrome style analog dslr award-winning 8k
      art photo showing a nurse battling a giant cell phone, above text caption
      of \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern
      American medical hospital 
    output:
      url: samples/1729700596338__000001600_1.jpg
  - text: >-
      /@step 1000 weights:/ HST autochrome style analog dslr award-winning 8k
      art photo showing a nurse battling a giant cell phone, above text caption
      of \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern
      American medical hospital 
    output:
      url: samples/1729697845170__000001000_1.jpg
  - text: >-
      /@step 3600 weights:/ HST style autochrome photograph of a dark CIA agent
      Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
      atmosphere, award-winning art photo
    output:
      url: samples/1729709764758__000003600_2.jpg
  - text: >-
      /@step 1200 weights:/ HST style autochrome photograph of a dark CIA agent
      Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
      atmosphere, award-winning art photo
    output:
      url: samples/1729698783304__000001200_2.jpg
  - text: >-
      /@step 1000 weights:/ HST style autochrome photograph of a dark CIA agent
      Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic
      atmosphere, award-winning art photo
    output:
      url: samples/1729697866093__000001000_2.jpg
  - text: >-
      /@step 4000 weights:/ HST style autochrome photo of realistic green-eyed
      black cat, with prominent regions of white fur, playing a piano and
      singing, amateur 2004 photograph shot on a cell phone in a Los Angeles
      apartment kitchen
    output:
      url: samples/1729711603742__000004000_3.jpg
  - text: >-
      /@step 3000 weights:/ HST style autochrome photo of realistic green-eyed
      black cat, with prominent regions of white fur, playing a piano and
      singing, amateur 2004 photograph shot on a cell phone in a Los Angeles
      apartment kitchen
    output:
      url: samples/1729707029633__000003000_3.jpg
  - text: >-
      /@step 1800 weights:/ HST style autochrome photo of realistic green-eyed
      black cat, with prominent regions of white fur, playing a piano and
      singing, amateur 2004 photograph shot on a cell phone in a Los Angeles
      apartment kitchen
    output:
      url: samples/1729701550832__000001800_3.jpg
  - text: >-
      /@step 1000 weights:/ HST style autochrome photo of realistic green-eyed
      black cat, with prominent regions of white fur, playing a piano and
      singing, amateur 2004 photograph shot on a cell phone in a Los Angeles
      apartment kitchen
    output:
      url: samples/1729697887015__000001000_3.jpg
base_model: stabilityai/stable-diffusion-3.5-large
license: creativeml-openrail-m
language:
  - en
pipeline_tag: text-to-image
library_name: diffusers

HSTsd3ii

Model trained with AI Toolkit by Ostris

Prompt
/@step 4000 weights:/ HST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930
Prompt
/@step 1600 weights:/HST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930
Prompt
/@step 1800 weights:/HST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930
Prompt
/@step 800 weights:/HHST style autochrome photo of a drugged God Christ-insect in wooden pillbox being pinned needle-poked by a giant Marina Tsvetaeva in a long red gown, arcane symbolism, colorful HD realistic film in Yury Annenkov & David Lynch styles, detailed face worrying, full-height red-robed woman roams European city circa 1930
Prompt
/@step 1800 weights:/HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh.
Prompt
/@step 4000 weights:/HHST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh.
Prompt
/@step 1600 weights:/ HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh.
Prompt
/@step 1600 weights:/ HST autochrome style analog dslr award-winning 8k art photo showing a nurse battling a giant cell phone, above text caption of \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American medical hospital
Prompt
/@step 1000 weights:/ HST autochrome style analog dslr award-winning 8k art photo showing a nurse battling a giant cell phone, above text caption of \ONCE WE WERE ALL HARRY GARDNER!\, in a highly realistic modern American medical hospital
Prompt
/@step 3600 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning art photo
Prompt
/@step 1200 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning art photo
Prompt
/@step 1000 weights:/ HST style autochrome photograph of a dark CIA agent Koala leaping at the throat of goth freedom-fighter Julian Assange, tragic atmosphere, award-winning art photo
Prompt
/@step 4000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen
Prompt
/@step 3000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen
Prompt
/@step 1800 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen
Prompt
/@step 1000 weights:/ HST style autochrome photo of realistic green-eyed black cat, with prominent regions of white fur, playing a piano and singing, amateur 2004 photograph shot on a cell phone in a Los Angeles apartment kitchen

Trigger words

'HST style autochrome photo'

Config Parameters

Dim:16 Alpha:32 Optimizer:Ademamix8bit LR:2e-4 More info below!
Fine-tuned using the Google Colab Notebook* Version of ai-toolkit.
I've used A100 via Colab Pro. However, training SD3.5 may potentially work with Free Colab or lower VRAM in general:
Especially if one were to use:
...Say, lower rank (try 4 or 8), dataset size (in terms of caching/bucketing/pre-loading impacts), 1 batch size, Adamw8bit optimizer, 512 resolution, maybe adding the /lowvram, true/ argument, and plausibly specifying alternate quantization variants.
Generally, VRAM expenditures for fine-tuning SD3.5 tend to be lower than for Flux during training.
So, try it!
To use on Colab, modify a Flux template Notebook from here with parameters from Ostris' example config for SD3.5 here!

job: extension
config:
  name: HSTsd3ii
  process:
  - type: sd_trainer
    training_folder: /content/drive/MyDrive/HSTsd3ii
    performance_log_every: 600
    device: cuda:0
    network:
      type: lora
      linear: 16
      linear_alpha: 32
    save:
      dtype: float16
      save_every: 250
      push_to_hub: true
      hf_repo_id: AlekseyCalvin/HSTsd3iii
      hf_private: true
      max_step_saves_to_keep: 16
    datasets:
    - folder_path: /content/dataset
      caption_ext: txt
      caption_dropout_rate: 0.0
      shuffle_tokens: false
      cache_latents_to_disk: true
      resolution:
      - 1024
    train:
      batch_size: 4
      steps: 4000
      gradient_accumulation_steps: 1
      train_unet: true
      train_text_encoder: false
      gradient_checkpointing: true
      noise_scheduler: flowmatch
      timestep_type: linear
      optimizer: ademamix8bit
      lr: 0.0002
      skip_first_sample: true
      ema_config:
        use_ema: true
        ema_decay: 0.8
      dtype: bf16
    model:
      name_or_path: stabilityai/stable-diffusion-3.5-large
      is_v3: true
      quantize: true

Plus validation settings:
Prompts like the above, at 1024, guidance scale 4, 25 steps, seed 42, no negatives.

Download model and use it with ComfyUI, AUTOMATIC1111, SD.Next, Invoke AI, etc.

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.

Use it with the 🧨 diffusers library 

from diffusers import AutoPipelineForText2Image
import torch

pipeline = AutoPipelineForText2Image.from_pretrained('stabilityai/stable-diffusion-3.5-large', torch_dtype=torch.float16).to('cuda')
pipeline.load_lora_weights('AlekseyCalvin/HSTsd3iii', weight_name='HSTsd3ii.safetensors')
image = pipeline('HST style communist poster with text "JOIN RCA!", over autochrome color photo of Vladimir Lenin at a Dada cabaret in 1916 Zurich, dancing with red feathered drunken dinosaur, an early conceptual artist. Lenin is full of contageous awe, his blemished skin flushing with anxious excitement, his famous bald spot sweatily glistening under warm lights. In the back, Krupskaya and Inessa Armand laugh. ').images[0]
image.save("my_image.png")

For more details, including weighting, merging and fusing LoRAs, check the documentation on loading LoRAs in diffusers