I'm using the IQ4_XS quant with 24k context on a A100, and it is GREAT. Thanks for providing the model!
· Sign up or log in to comment