pbatra
/

TinyLlama-1.1B-Chat-v1.0-GGUF

Text Generation

Inference Endpoints

text-generation-inference

Model card Files Files and versions Community

Edit model card

TinyLlama-1.1B-Chat-v1.0

This repository contains quantized versions of the model from the original repository: TinyLlama/TinyLlama-1.1B-Chat-v1.0.

Name	Quantization Method	Size (GB)
tinyllama-1.1b-chat-v1.0.fp16.gguf	FP16	2.05
tinyllama-1.1b-chat-v1.0.fp32.gguf	FP32	4.10
tinyllama-1.1b-chat-v1.0.Q2_K.gguf	q2_k	0.40
tinyllama-1.1b-chat-v1.0.Q3_K_S.gguf	q3_k_s	0.47
tinyllama-1.1b-chat-v1.0.Q3_K_M.gguf	q3_k_m	0.51
tinyllama-1.1b-chat-v1.0.Q3_K_L.gguf	q3_k_l	0.55
tinyllama-1.1b-chat-v1.0.Q4_0.gguf	q4_0	0.59
tinyllama-1.1b-chat-v1.0.Q4_K_S.gguf	q4_k_s	0.60
tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf	q4_k_m	0.62
tinyllama-1.1b-chat-v1.0.Q5_0.gguf	q5_0	0.71
tinyllama-1.1b-chat-v1.0.Q5_K_S.gguf	q5_k_s	0.71
tinyllama-1.1b-chat-v1.0.Q5_K_M.gguf	q5_k_m	0.73
tinyllama-1.1b-chat-v1.0.Q6_K.gguf	q6_k	0.84
tinyllama-1.1b-chat-v1.0.Q8_0.gguf	q8_0	1.09

Downloads last month: 42

GGUF

Model size

1.1B params

Architecture

llama

2-bit

3-bit

4-bit

5-bit

6-bit

8-bit

16-bit

Inference Examples

Text Generation

Inference API (serverless) has been turned off for this model.

Model tree for pbatra/TinyLlama-1.1B-Chat-v1.0-GGUF

Base model

TinyLlama/TinyLlama-1.1B-Chat-v1.0

Quantized

(68)

this model

Datasets used to train pbatra/TinyLlama-1.1B-Chat-v1.0-GGUF