Fine tune the model with part of layers on GPU and rest on CPU
#11
by
vmirea
- opened
Hello, I love the accuracy of this model. I'm testing it on a RTX 4090, 24 GB using LM studio to load 30 layers on GPU an rest on CPU.
I found finetune/finetune_deepseekcoder.py that fine tunes the model but I get OOM errors because it tries to load it all on the GPU.
Is there any method to load some layers on the GPU and rest on the CPU during the fine tuning?
zqh11
changed discussion status to
closed