README.md

metadata

license: llama3.1
tags:
  - llama
  - conversational
  - text-generation-inference
  - facebook
  - meta
  - llama-3
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
library_name: transformers

NOTICE:
Llama-3.1 is licensed under Llama 3.1 Community License
A copy of this license is available at this repo, here

Original Model: meta-llama/Meta-Llama-3.1-8B-Instruct

How to Use: llama.cpp

Original Model License: Llama 3.1 Community License

Release Used: b3441

Quants

Name	Quant Type	Size
Meta-Llama-3.1-8B-Instruct-Q2_K.gguf	Q2_K	3.18 GB
Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf	Q3_K_S	3.66 GB
Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf	Q3_K_M	4.02 GB
Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf	Q3_K_L	4.32 GB
Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf	Q4_K_S	4.69 GB
Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf	Q4_K_M	4.92 GB
Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf	Q5_K_S	5.60 GB
Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf	Q5_K_M	5.73 GB
Meta-Llama-3.1-8B-Instruct-Q6_K.gguf	Q6_K	6.60 GB
Meta-Llama-3.1-8B-Instruct-Q8_0.gguf	Q8_0	8.54 GB