leliuga
/

phi-2-bnb-4bit

Text Generation

arxiv:1910.09700

text-generation-inference

4-bit precision

Model card Files Files and versions Community

phi-2 - bnb 4bit

Model creator: Microsoft
Original model: phi-2

Description

This model is 4bit quantized version of phi-2 using bitsandbytes. It's designed for fine-tuning! The PAD token is set as "<|endoftext|>".

Downloads last month: 61

Safetensors

Model size

1.56B params

Tensor type

F32

·

FP16

·

U8

·

Inference Providers NEW

Text Generation

This model is not currently available via any of the supported Inference Providers.

The model cannot be deployed to the HF Inference API: The model authors have turned it off explicitly.

Model tree for leliuga/phi-2-bnb-4bit

Base model

microsoft/phi-2

Quantized

(26)

this model

Collection including leliuga/phi-2-bnb-4bit

bnb-4 bit

A 4-bit quantized version models designed for fine-tuning. Demo: https://colab.research.google.com/drive/19UpFUjtbJoLua-4DMb1JKKwJAcvMyMHb • 21 items • Updated Jun 28, 2024