leliuga
/

gemma-7b-it-bnb-4bit

Text Generation

arxiv:1910.09700

text-generation-inference

4-bit precision

Model card Files Files and versions Community

gemma-7b-it - bnb 4bit

Model creator: Google
Original model: gemma-7b-it

Description

This model is 4bit quantized version of gemma-7b-it using bitsandbytes. It's designed for fine-tuning! The PAD token is set as "".

Downloads last month: 16

Safetensors

Model size

4.78B params

Tensor type

F32

·

BF16

·

U8

·

Inference Examples

Text Generation

Inference API (serverless) has been turned off for this model.

Model tree for leliuga/gemma-7b-it-bnb-4bit

Base model

google/gemma-7b

Finetuned

google/gemma-7b-it

Quantized

(18)

this model

Collection including leliuga/gemma-7b-it-bnb-4bit

bnb-4 bit

A 4-bit quantized version models designed for fine-tuning. Demo: https://colab.research.google.com/drive/19UpFUjtbJoLua-4DMb1JKKwJAcvMyMHb • 21 items • Updated Jun 28