Will you re-upload these quants?

#2
by yamikumods - opened

Will you re-upload these quants with BPE-pretokenization applied by new llama.cpp update?
I guess your quants with groups_merged.txt imatrix give better results comparing to wiki.train.raw especially in programming tasks.
So, I hope you update them.

Owner
β€’
edited May 8

I will, but my server is pretty busy ATM and the large models take a long time to quant, as I am doing it on a CPU-only server with barely enough RAM for these huge models...
Non-Plus is already running and should be done within a day or so. Plus will take a few days once I start it...

Ah, okay, I understand your situation.
thank you.

Thx a lot for re-uploading~

πŸš€

qwp4w3hyb changed discussion status to closed

Sign up or log in to comment