Extreme Training efficiency?
#4
by
aaronday3
- opened
Hi there, great model!
The thing that caught my attention is that you trained a 64 rank qlora for a 70B model on 4x4090s which would have around 96GB of VRAM.
This is extremely efficient and I was wondering if it's possible to do this in axolotl or whether qlora-pipe has special vram optimizations that axolotl doesn't. Is it possible to do this in axolotl?
I reviewed the config you posted in the other discussion and I have a question, did you make qlora-pipe specifically for VRAM efficiency? Is it more efficient than axolotl?