mlx-community/Qwen2.5-Coder-32B-8bit · Is there any quantized version, with good performance, high accuracy that fits into 16GB

Is there any quantized version, with good performance, high accuracy that fits into 16GB

by bdutta - opened 22 days ago

MLX Community org 22 days ago

Looking for a quantized, MLX optimized flavour of Qwen2.5-Coder-32B that has good performance, has high accuracy and fits into 16GB memory. Is there something available ?

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment