Llama 3.1 GPTQ, AWQ, and BNB Quants - a hugging-quants Collection

hugging-quants 's Collections

Gemma2 AWQ Quants

Llama 3.2 3B & 1B GGUF Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

Llama 3.1 GPTQ, AWQ, and BNB Quants

updated Sep 26

Optimised Quants for high-throughput deployments! Compatible with Transformers, TGI & VLLM 🤗

hugging-quants/Meta-Llama-3.1-405B-Instruct-AWQ-INT4

Text Generation • Updated Sep 13 • 1.58k • 37
hugging-quants/Meta-Llama-3.1-405B-Instruct-BNB-NF4

Text Generation • Updated Sep 16 • 1.3k • 5
hugging-quants/Meta-Llama-3.1-405B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 1.13k • 16
hugging-quants/Meta-Llama-3.1-70B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 167k • 92
unsloth/Meta-Llama-3.1-70B-Instruct-bnb-4bit

Text Generation • Updated Nov 22 • 30.6k • 27
hugging-quants/Meta-Llama-3.1-70B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 206k • 22
hugging-quants/Meta-Llama-3.1-8B-Instruct-AWQ-INT4

Text Generation • Updated Aug 7 • 413k • 61
hugging-quants/Meta-Llama-3.1-8B-Instruct-BNB-NF4

Text Generation • Updated Aug 8 • 4.41k • 7
hugging-quants/Meta-Llama-3.1-8B-Instruct-GPTQ-INT4

Text Generation • Updated Aug 7 • 97.9k • 21