Running in llama.cpp
Note for any other 🤡 like me who wants to run this model in llama.cpp and it doesn't work: change in llama.cpp file
#define LLAMA_MAX_NODES 8192
to
#define LLAMA_MAX_NODES 16384
and compile.
Have fun!
Apologies for inconvenience.
Are there any specific issues that you face with this model?
Regards.
Yeah, it's incoherent, creates fake words and doesn't listen to the prompts. Not really worth its size, I did run it in Q3_K_L. Did you actually test the model before uploading? Or did you just glue it together and hoped that some poor fool would run it?
That sounds a bit odd to be happening.
We did test the model indeed before releasing it.
Allow us some time to look into the matter.
Thanks for bringing this to our attention.
Regards.
This model really seems to be broken to me. I've tried your saily_100b, and it doesn't have this kind of issues.
Thanks
@ChuckMcSneed
for bringing this to our attention.
We're looking into the issues with this model and will update the weights soon.
Regards.