|
--- |
|
license: mit |
|
license_link: >- |
|
https://huggingface.co./microsoft/Phi-3-mini-128k-instruct/resolve/main/LICENSE |
|
language: |
|
- en |
|
pipeline_tag: text-generation |
|
base_model: microsoft/Phi-3-mini-128k-instruct |
|
tags: |
|
- nlp |
|
- code |
|
--- |
|
|
|
## Model Summary |
|
|
|
This repo provides the GGUF format for the Phi-3-Mini-128K-Instruct. |
|
|
|
*For more details check out the original model at [microsoft/Phi-3-mini-128k-instruct](https://huggingface.co./microsoft/Phi-3-mini-128k-instruct).* |
|
|
|
|
|
The Phi-3-Mini-128K-Instruct is a 3.8 billion-parameter, lightweight, state-of-the-art open model trained using the Phi-3 datasets. This dataset includes both synthetic data and filtered publicly available website data, with an emphasis on high-quality and reasoning-dense properties. The model belongs to the Phi-3 family with the Mini version in two variants 4K and 128K which is the context length (in tokens) that it can support. |
|
|
|
After initial training, the model underwent a post-training process that involved supervised fine-tuning and direct preference optimization to enhance its ability to follow instructions and adhere to safety measures. When evaluated against benchmarks that test common sense, language understanding, mathematics, coding, long-term context, and logical reasoning, the Phi-3 Mini-128K-Instruct demonstrated robust and state-of-the-art performance among models with fewer than 13 billion parameters. Resources and Technical Documentation: |
|
|
|
Resources and Technical Documentation: |
|
|
|
+ [Phi-3 Microsoft Blog](https://aka.ms/phi3blog-april) |
|
+ [Phi-3 Technical Report](https://aka.ms/phi3-tech-report) |
|
+ [Phi-3 on Azure AI Studio](https://aka.ms/phi3-azure-ai) |
|
+ [Phi-3 on Hugging Face](https://aka.ms/phi3-hf) |
|
+ Phi-3 ONNX: [4K](https://aka.ms/phi3-mini-4k-instruct-onnx) and [128K](https://aka.ms/phi3-mini-128k-instruct-onnx) |
|
|
|
This repo provides GGUF files and Llamafiles ([`d228e01d`](https://github.com/Mozilla-Ocho/llamafile/tree/d228e01d70a7b91bf04dbf63428646f3f173b888)) for the Phi-3 Mini-128K-Instruct model. |
|
| Name | Quant method | Bits | Size | Use case | |
|
| ---- | ---- | ---- | ---- | ----- | |
|
| [Phi-3-mini-128k-instruct-Q4_K_M.gguf](https://huggingface.co./AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.gguf) | Q4_K_M | 4 | 2.39 GB | medium, balanced quality - recommended | |
|
| [Phi-3-mini-128k-instruct-Q4_K_M.llamafile](https://huggingface.co./AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-Q4_K_M.llamafile) | Q4_K_M | 4 | 2.4 GB | medium, balanced quality - recommended | |
|
| [Phi-3-mini-128k-instruct-f16.gguf](https://huggingface.co./AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.gguf) | None | 16 | 7.64 GB | minimal quality loss | |
|
| [Phi-3-mini-128k-instruct-f16.llamafile](https://huggingface.co./AlessandroW/Phi-3-mini-128k-instruct-gguf/blob/main/Phi-3-mini-128k-instruct-f16.llamafile) | None | 16 | 7.65 GB | minimal quality loss | |
|
|
|
*Note:* When using the llamafile version make sure to specify the context size, e.g., `./Phi-3-mini-128k-instruct-Q4_K_M.llamafile -c 0 -p "your prompt"`. |
|
|
|
### License |
|
|
|
The model is licensed under the [MIT license](https://huggingface.co./microsoft/phi-3-mini-128k/resolve/main/LICENSE). |
