how to finetune and quantize the qwen1.5 model with gguf
#5
by
huntz47
- opened
i am new in here. i tried finetuning the qwen model and and quantized it using llama factory and llama.cpp. but when i try to run the gguf file after quantizing, its getting error related to missing output.weight tensor file
It only happens to the 0.5B models which uses tie word embedings.
A fix has been merged: https://github.com/ggerganov/llama.cpp/pull/6738
jklj077
changed discussion status to
closed