Others quantization?

by slavczays - opened May 10

May 10

Any chance for models with lower quantization like q2_k (still usefull and better fit for mobile devices)?

Remek

Owner May 12

Hi,
I uploaded 2 and 3bit quentizations for you :)

May 14

Thanks a lot bro!;)

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment