Hugging Face's logo

kaitchup
/
Phi-4-mini-instruct-AutoRoundGPTQ-4bit

Text Generation
Transformers
Safetensors
English
phi3
auto-gptq
AutoRound
conversational
custom_code
text-generation-inference
Inference Endpoints
4-bit precision
gptq
Model card Files Community

Model Details

This is Phi-4-mini-instruct quantized with AutoRound (symmetric quantization) and serialized with the GPTQ format in 4-bit. The model has been created, tested, and evaluated by The Kaitchup.

image/png

Details on the quantization process and how to use the model here: The Kaitchup

  • Developed by: The Kaitchup
  • Language(s) (NLP): English
  • License: Apache 2.0 license

How to Support My Work

Subscribe to The Kaitchup! I release quantized models with the Apache 2.0 license.

Downloads last month
4
Safetensors
Model size
1.05B params
Tensor type
I32
·
BF16
·
FP16
·
Inference Providers NEW
Text Generation
This model is not currently available via any of the supported Inference Providers.

Collection including kaitchup/Phi-4-mini-instruct-AutoRoundGPTQ-4bit