|
--- |
|
license: llama3.1 |
|
tags: |
|
- llama |
|
- conversational |
|
- text-generation-inference |
|
- facebook |
|
- meta |
|
- llama-3 |
|
base_model: meta-llama/Meta-Llama-3.1-8B-Instruct |
|
library_name: transformers |
|
--- |
|
|
|
> [!IMPORTANT] |
|
> **NOTICE:**<br> |
|
> Llama-3.1 is licensed under [Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)<br> |
|
> A copy of this license is available at this repo, [here](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/LICENSE) |
|
|
|
**Original Model:** [meta-llama/Meta-Llama-3.1-8B-Instruct](https://huggingface.co./meta-llama/Meta-Llama-3.1-8B-Instruct) |
|
|
|
**How to Use:** [llama.cpp](https://github.com/ggerganov/llama.cpp) |
|
|
|
**Original Model License:** Llama 3.1 Community License |
|
|
|
**Release Used:** [b3441](https://github.com/ggerganov/llama.cpp/releases/tag/b3441) |
|
|
|
# Quants |
|
| Name | Quant Type | Size | |
|
| ---- | ---- | ---- | |
|
| [Meta-Llama-3.1-8B-Instruct-Q2_K.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q2_K.gguf) | Q2_K | 3.18 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_S.gguf) | Q3_K_S | 3.66 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_M.gguf) | Q3_K_M | 4.02 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q3_K_L.gguf) | Q3_K_L | 4.32 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q4_K_S.gguf) | Q4_K_S | 4.69 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q4_K_M.gguf) | Q4_K_M | 4.92 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q5_K_S.gguf) | Q5_K_S | 5.60 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q5_K_M.gguf) | Q5_K_M | 5.73 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q6_K.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q6_K.gguf) | Q6_K | 6.60 GB | |
|
| [Meta-Llama-3.1-8B-Instruct-Q8_0.gguf](https://huggingface.co./starble-dev/Meta-Llama-3.1-8B-Instruct-GGUF/blob/main/Meta-Llama-3.1-8B-Instruct-Q8_0.gguf) | Q8_0 | 8.54 GB | |
