Any chance your team is working on a 4-bit Llama-3.2-90B-Vision-Instruct-quantized.w4a16 version?

#1
by mrhendrey - opened

Love the work that you do. Hoping you are going to put out some of the 4-bit quantized versions in the near future. thank you

Neural Magic org

Appreciate it! Yes we are working on enabling quantization flows with calibration for VLMs

Sign up or log in to comment