Post
1332
A comprehensive matrix for which format should you use.
Read more on my blog post: https://huggingface.co./blog/ngxson/common-ai-model-formats
Read more on my blog post: https://huggingface.co./blog/ngxson/common-ai-model-formats
| Hardware | GGUF | PyTorch | Safetensors | ONNX |
|-----------------|-----------|------------------------|--------------------------|-------|
| CPU | ✅ (best) | 🟡 | 🟡 | ✅ |
| GPU | ✅ | ✅ | ✅ | ✅ |
| Mobile | ✅ | 🟡 (via executorch) | ❌ | ✅ |
| Apple silicon | ✅ | 🟡 | ✅ (via MLX framework) | ✅ |
- 1 reply
