qwp4w3hyb/Llama-3-11.5B-Instruct-V2-iMat-GGUF

Quant Infos

• quants done with an importance matrix for improved quantization loss
• ggufs & imatrix generated from bf16 for "optimal" accuracy loss
• Wide coverage of different gguf quant types from Q_8_0 down to IQ1_S
• Quantized with llama.cpp commit fabf30b4c4fca32e116009527180c252919ca922 (master as of 2024-05-20)
• Imatrix generated with this multi-purpose dataset.

  ./imatrix -c 512 -m $model_name-f16.gguf -f $llama_cpp_path/groups_merged.txt -o $out_path/imat-f16-gmerged.dat
  

Original Model Card:

Llama-3-11.5B-v2

Thank you to Meta for the weights for Meta-Llama-3-8B

This is an upscaling of the Meta-Llama-3-8B Ai using techniques created for chargoddard/mistral-11b-slimorca. This Ai model has been upscaled from 8b parameters to 11.5b parameters without any continuous pretraining or fine-tuning.

Unlike version 1 this model has no issues at fp16 or any quantizations.

The model that was used to create this one is linked below:

https://huggingface.co./meta-llama/Meta-Llama-3-8B