Hugging Face's logo
neuralmagic 's Collections
Granite 3.1 Quantization
Sparse-Llama-3.1-2of4
Vision Language Models Quantization
FP8 LLMs for vLLM
Llama-3.2 Quantization
Llama-3.1 Quantization
INT8 LLMs for vLLM
INT4 LLMs for vLLM
Sparse Foundational Llama 2 Models
Compression Papers
DeepSparse Sparse LLMs
Sparse Finetuning MPT
Compressed LLMs from the Community

INT4 LLMs for vLLM

Accurate INT4 quantized models by Neural Magic, ready for use with vLLM!

Upvote
8