File size: 2,669 Bytes
62601d9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 |
---
license: llama3.1
tags:
- llama
- conversational
- text-generation-inference
- facebook
- meta
- llama-3
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct
library_name: transformers
---
> [!IMPORTANT]
> **NOTICE:**<br>
> Llama-3.1 is licensed under [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)<br>
> A copy of this license is available at this repo, [here](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/LICENSE)
**Original Model:** [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co./meta-llama/Meta-Llama-3.1-8B-Instruct)
**How to Use:** [llama.cpp](https://github.com/ggerganov/llama.cpp)
**Original Model License:** Llama 3.1 Community License
**Release Used:** [b3441](https://github.com/ggerganov/llama.cpp/releases/tag/b3441)
# Quants
| Name | Quant Type | Size |
| ---- | ---- | ---- |
| [Meta-Llama-3.1-8B-Instruct-Q2_K.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q2_K.gguf) | Q2_K | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf) | Q3_K_S | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf) | Q3_K_M | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf) | Q3_K_L | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf) | Q4_K_S | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf) | Q4_K_M | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf) | Q5_K_S | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf) | Q5_K_M | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q6_K.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q6_K.gguf) | Q6_K | ? GB |
| [Meta-Llama-3.1-8B-Instruct-Q8_0.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf) | Q8_0 | ? GB | |