--- base_model: - 152334H/miqu-1-70b-sf - lizpreciatior/lzlv_70b_fp16_hf language: - en library_name: transformers quantized_by: mradermacher tags: - mergekit - merge --- ## About weighted/imatrix quants of https://huggingface.co./wolfram/miquliz-120b-v2.0 ## Usage If you are unsure how to use GGUF files, refer to one of [TheBloke's READMEs](https://huggingface.co./TheBloke/KafkaLM-70B-German-V0.1-GGUF) for more details, including on how to concatenate multi-part files. ## Provided Quants (sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants) | Link | Type | Size/GB | Notes | |:-----|:-----|--------:|:------| | [GGUF](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-IQ1_S.gguf) | i1-IQ1_S | 25.6 | | | [GGUF](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 32.1 | | | [GGUF](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-IQ2_XS.gguf) | i1-IQ2_XS | 35.7 | | | [GGUF](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q2_K.gguf) | i1-Q2_K | 44.5 | | | [GGUF](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 47.2 | fast, lower quality | | [PART 1](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_XS.gguf.split-aa) [PART 2](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_XS.gguf.split-ab) | i1-Q3_K_XS | 49.2 | IQ3-XXS probably better | | [PART 1](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_S.gguf.split-aa) [PART 2](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_S.gguf.split-ab) | i1-Q3_K_S | 52.1 | IQ3-XXS probably better | | [PART 1](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_M.gguf.split-aa) [PART 2](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_M.gguf.split-ab) | i1-Q3_K_M | 58.1 | lower quality | | [PART 1](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_L.gguf.split-aa) [PART 2](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q3_K_L.gguf.split-ab) | i1-Q3_K_L | 63.3 | | | [PART 1](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q4_K_S.gguf.split-aa) [PART 2](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q4_K_S.gguf.split-ab) | i1-Q4_K_S | 68.6 | fast, medium quality | | [PART 1](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q4_K_M.gguf.split-aa) [PART 2](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q4_K_M.gguf.split-ab) | i1-Q4_K_M | 72.5 | fast, medium quality | | [PART 1](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q5_K_M.gguf.part1of2) [PART 2](https://huggingface.co./mradermacher/miquliz-120b-v2.0-i1-GGUF/resolve/main/miquliz-120b-v2.0.i1-Q5_K_M.gguf.part2of2) | i1-Q5_K_M | 85.3 | best weighted quant | Here is a handy table comparing some lower-quality quant types: ![image.png](https://cdn-uploads.huggingface.co/production/uploads/645ce413a19f3e64bbeece31/dEiT6xDvxyANdetzVG1tX.png)