Run guanaco model with llama.cpp, get gibberish output.
There's a bug in llama.cpp at the moment, where on Windows the new quant methods result in gibberish: https://github.com/ggerganov/llama.cpp/issues/1735
Please follow the above Issue for further details. You could maybe try going back to an older build of llama.cpp
There's a bug in llama.cpp at the moment, where on Windows the new quant methods result in gibberish: https://github.com/ggerganov/llama.cpp/issues/1735
Please follow the above Issue for further details. You could maybe try going back to an older build of llama.cpp
I'm running on ubuntu, does the bug described above only affect windows?
Thank you so much!
Everyone reporting it on that thread is using Windows. But your symptom is the same, and the developer does mention that theoretically the bug could exist in Linux also.
Could you please post in the bug report Issue and give your findings?
In any case there's nothing I can do about it, as the models themselves do work OK.