Fine tuning with Lora does not support this model
#4
by
fromjon
- opened
I attempt to run the lora.py trainer with this model but it fails to start
...mlx_lm/tuner/utils.py", line 129, in linear_to_lora_layers
raise ValueError(f"Lora does not support {model.model_type}")
ValueError: Lora does not support deepseek_v2
Can mlx_lm fine-tune models without LoRA?
If so, how?
The DeepSeek V2 is using MLA attention. I haven't tried LoRA yet, so it hasn't been added to MLX_LM's LoRA support. It might be better to open an issue on the mlx-example repo to see how to add LoRA support for it.
@fromjon This has been fixed by https://github.com/ml-explore/mlx-examples/pull/932. Let me know if it still doesnt work for you
mzbac
changed discussion status to
closed