Improved quant

#2
by distantquant - opened

Here is a most likely improved quant that rotates the shapes better: https://huggingface.co./152334H/miqu-1-70b-sf

Why is this only 48Gb or less.

If it was full upscaled to full float16 wouldn't it be 140Gb?

Sign up or log in to comment