Commit
•
c3952e1
0
Parent(s):
Duplicate from mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF
Browse filesCo-authored-by: Michael Radermacher <[email protected]>
- .gitattributes +58 -0
- Midnight-Miqu-70B-v1.5.i1-IQ1_M.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ1_S.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ2_M.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ2_S.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ2_XS.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ2_XXS.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ3_M.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ3_S.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ3_XS.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ3_XXS.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-IQ4_XS.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q2_K.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q3_K_L.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q3_K_M.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q3_K_S.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q4_0.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q4_K_M.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q4_K_S.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q5_K_M.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q5_K_S.gguf +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part1of2 +3 -0
- Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part2of2 +3 -0
- README.md +71 -0
- imatrix.dat +3 -0
.gitattributes
ADDED
@@ -0,0 +1,58 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
*.7z filter=lfs diff=lfs merge=lfs -text
|
2 |
+
*.arrow filter=lfs diff=lfs merge=lfs -text
|
3 |
+
*.bin filter=lfs diff=lfs merge=lfs -text
|
4 |
+
*.bz2 filter=lfs diff=lfs merge=lfs -text
|
5 |
+
*.ckpt filter=lfs diff=lfs merge=lfs -text
|
6 |
+
*.ftz filter=lfs diff=lfs merge=lfs -text
|
7 |
+
*.gz filter=lfs diff=lfs merge=lfs -text
|
8 |
+
*.h5 filter=lfs diff=lfs merge=lfs -text
|
9 |
+
*.joblib filter=lfs diff=lfs merge=lfs -text
|
10 |
+
*.lfs.* filter=lfs diff=lfs merge=lfs -text
|
11 |
+
*.mlmodel filter=lfs diff=lfs merge=lfs -text
|
12 |
+
*.model filter=lfs diff=lfs merge=lfs -text
|
13 |
+
*.msgpack filter=lfs diff=lfs merge=lfs -text
|
14 |
+
*.npy filter=lfs diff=lfs merge=lfs -text
|
15 |
+
*.npz filter=lfs diff=lfs merge=lfs -text
|
16 |
+
*.onnx filter=lfs diff=lfs merge=lfs -text
|
17 |
+
*.ot filter=lfs diff=lfs merge=lfs -text
|
18 |
+
*.parquet filter=lfs diff=lfs merge=lfs -text
|
19 |
+
*.pb filter=lfs diff=lfs merge=lfs -text
|
20 |
+
*.pickle filter=lfs diff=lfs merge=lfs -text
|
21 |
+
*.pkl filter=lfs diff=lfs merge=lfs -text
|
22 |
+
*.pt filter=lfs diff=lfs merge=lfs -text
|
23 |
+
*.pth filter=lfs diff=lfs merge=lfs -text
|
24 |
+
*.rar filter=lfs diff=lfs merge=lfs -text
|
25 |
+
*.safetensors filter=lfs diff=lfs merge=lfs -text
|
26 |
+
saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
27 |
+
*.tar.* filter=lfs diff=lfs merge=lfs -text
|
28 |
+
*.tar filter=lfs diff=lfs merge=lfs -text
|
29 |
+
*.tflite filter=lfs diff=lfs merge=lfs -text
|
30 |
+
*.tgz filter=lfs diff=lfs merge=lfs -text
|
31 |
+
*.wasm filter=lfs diff=lfs merge=lfs -text
|
32 |
+
*.xz filter=lfs diff=lfs merge=lfs -text
|
33 |
+
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
+
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
+
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
imatrix.dat filter=lfs diff=lfs merge=lfs -text
|
37 |
+
Midnight-Miqu-70B-v1.5.i1-Q2_K.gguf filter=lfs diff=lfs merge=lfs -text
|
38 |
+
Midnight-Miqu-70B-v1.5.i1-Q4_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
39 |
+
Midnight-Miqu-70B-v1.5.i1-Q3_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
40 |
+
Midnight-Miqu-70B-v1.5.i1-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
41 |
+
Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part1of2 filter=lfs diff=lfs merge=lfs -text
|
42 |
+
Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part2of2 filter=lfs diff=lfs merge=lfs -text
|
43 |
+
Midnight-Miqu-70B-v1.5.i1-Q3_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
44 |
+
Midnight-Miqu-70B-v1.5.i1-Q3_K_L.gguf filter=lfs diff=lfs merge=lfs -text
|
45 |
+
Midnight-Miqu-70B-v1.5.i1-Q5_K_S.gguf filter=lfs diff=lfs merge=lfs -text
|
46 |
+
Midnight-Miqu-70B-v1.5.i1-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
|
47 |
+
Midnight-Miqu-70B-v1.5.i1-IQ2_M.gguf filter=lfs diff=lfs merge=lfs -text
|
48 |
+
Midnight-Miqu-70B-v1.5.i1-IQ3_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
49 |
+
Midnight-Miqu-70B-v1.5.i1-IQ3_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
50 |
+
Midnight-Miqu-70B-v1.5.i1-IQ3_S.gguf filter=lfs diff=lfs merge=lfs -text
|
51 |
+
Midnight-Miqu-70B-v1.5.i1-IQ3_M.gguf filter=lfs diff=lfs merge=lfs -text
|
52 |
+
Midnight-Miqu-70B-v1.5.i1-IQ1_S.gguf filter=lfs diff=lfs merge=lfs -text
|
53 |
+
Midnight-Miqu-70B-v1.5.i1-IQ2_XXS.gguf filter=lfs diff=lfs merge=lfs -text
|
54 |
+
Midnight-Miqu-70B-v1.5.i1-IQ2_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
55 |
+
Midnight-Miqu-70B-v1.5.i1-IQ2_S.gguf filter=lfs diff=lfs merge=lfs -text
|
56 |
+
Midnight-Miqu-70B-v1.5.i1-IQ4_XS.gguf filter=lfs diff=lfs merge=lfs -text
|
57 |
+
Midnight-Miqu-70B-v1.5.i1-IQ1_M.gguf filter=lfs diff=lfs merge=lfs -text
|
58 |
+
Midnight-Miqu-70B-v1.5.i1-Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
Midnight-Miqu-70B-v1.5.i1-IQ1_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ae8a6575a5abeada04c51dd47215cf406574d3a671d3c3156cf9992974c3a3ae
|
3 |
+
size 15943252096
|
Midnight-Miqu-70B-v1.5.i1-IQ1_S.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:984c9e819d65f293de4ab5c370b39c0633082a552243cc3f7ffa797a84ac4820
|
3 |
+
size 14879602240
|
Midnight-Miqu-70B-v1.5.i1-IQ2_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:4337edb97dc78ad80119e0e9b1d2ecc00e90da892bf4afde756355114184b3c8
|
3 |
+
size 23575328320
|
Midnight-Miqu-70B-v1.5.i1-IQ2_S.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:608526ed8316af4afbff28e0dcc24590e4242dffde6b12d1fae4a0f2faa45bde
|
3 |
+
size 21698377280
|
Midnight-Miqu-70B-v1.5.i1-IQ2_XS.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1025372a91649715c61b300abf3dae639320d2497fdbf74da2f6dc7a9f8d41cc
|
3 |
+
size 20678227520
|
Midnight-Miqu-70B-v1.5.i1-IQ2_XXS.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:dcc52524ecc79e9b04d52fd12d3b05f446d1f2e13208d1f293cb0ac8af7f8ed6
|
3 |
+
size 18633504320
|
Midnight-Miqu-70B-v1.5.i1-IQ3_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:dc6d1b954e0fe5a3562d7d6184491f1af67273523ab9e99159030fac4eba5f26
|
3 |
+
size 31253526080
|
Midnight-Miqu-70B-v1.5.i1-IQ3_S.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3680a674f1f4a06a0c08417f6d80a35df6e6bae0dd2eaa55fe1bc7e805c67c1c
|
3 |
+
size 30228543040
|
Midnight-Miqu-70B-v1.5.i1-IQ3_XS.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b35d063923f7fdf02883c32e58b4c2fec44016130e65500379255663b3ad318d
|
3 |
+
size 28451206720
|
Midnight-Miqu-70B-v1.5.i1-IQ3_XXS.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:bc87841adc76f748574d3a460615201d77e0b1dd9fba33316fc8226982fbb78c
|
3 |
+
size 26925528640
|
Midnight-Miqu-70B-v1.5.i1-IQ4_XS.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:73cf5c32e6068a839a999c6c290aa9fa043389b9f3485f8dd144c503e44f63b3
|
3 |
+
size 37139068480
|
Midnight-Miqu-70B-v1.5.i1-Q2_K.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:ea78969e7d629a38cc68459556680fdfa709602291f331999e04e06dce3c654c
|
3 |
+
size 25771685440
|
Midnight-Miqu-70B-v1.5.i1-Q3_K_L.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:10b0b320afddeb33754addee10c45ebdb91448cf0a6dcb55735c8b5da4e65d5d
|
3 |
+
size 36457084480
|
Midnight-Miqu-70B-v1.5.i1-Q3_K_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1d738177cb867a9d238d86a8de629828379524c2fae5eae64a13e192e768eb11
|
3 |
+
size 33583986240
|
Midnight-Miqu-70B-v1.5.i1-Q3_K_S.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:09993ca7d375ca6426c84a7f1bdfecc7fbbda092a72fdd7d386582f31d5e7cee
|
3 |
+
size 30228543040
|
Midnight-Miqu-70B-v1.5.i1-Q4_0.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:b5d79cfbcbfd5e00b6a6d98b9f4fbf7708515723529efe13a1ca08d635e29b47
|
3 |
+
size 39019051136
|
Midnight-Miqu-70B-v1.5.i1-Q4_K_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:29e175b9787eb3bec66a362e0cefa5508494a09789106814495ae9759c103921
|
3 |
+
size 41732159040
|
Midnight-Miqu-70B-v1.5.i1-Q4_K_S.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:370db9d9ce692e8f93a2e1c226e2da21af9fb25ab58be81af90ea1b1b909d5e6
|
3 |
+
size 39558985280
|
Midnight-Miqu-70B-v1.5.i1-Q5_K_M.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:e24670864a68e1f630301c80faec4b8ccfdc21c5d67f2ab43852159d646b4dcd
|
3 |
+
size 49063016000
|
Midnight-Miqu-70B-v1.5.i1-Q5_K_S.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1522701bb764a5fd1959dd3a50c802fe22c2c482df4551f0eca2683ac9f78945
|
3 |
+
size 47770646080
|
Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part1of2
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:577c45b763b3978f915a1e481b317f39418f1683e384ffe5399d2af1ed63f27c
|
3 |
+
size 28991029248
|
Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part2of2
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:399f2c52a3546d0023fefb1c407fb6639bb884c2be80b988889b29caf60b6152
|
3 |
+
size 27905586752
|
README.md
ADDED
@@ -0,0 +1,71 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
---
|
2 |
+
base_model: sophosympatheia/Midnight-Miqu-70B-v1.5
|
3 |
+
language:
|
4 |
+
- en
|
5 |
+
library_name: transformers
|
6 |
+
license: other
|
7 |
+
quantized_by: mradermacher
|
8 |
+
tags:
|
9 |
+
- mergekit
|
10 |
+
- merge
|
11 |
+
---
|
12 |
+
## About
|
13 |
+
|
14 |
+
weighted/imatrix quants of https://huggingface.co/sophosympatheia/Midnight-Miqu-70B-v1.5
|
15 |
+
|
16 |
+
<!-- provided-files -->
|
17 |
+
static quants are available at https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-GGUF
|
18 |
+
## Usage
|
19 |
+
|
20 |
+
If you are unsure how to use GGUF files, refer to one of [TheBloke's
|
21 |
+
READMEs](https://huggingface.co/TheBloke/KafkaLM-70B-German-V0.1-GGUF) for
|
22 |
+
more details, including on how to concatenate multi-part files.
|
23 |
+
|
24 |
+
## Provided Quants
|
25 |
+
|
26 |
+
(sorted by size, not necessarily quality. IQ-quants are often preferable over similar sized non-IQ quants)
|
27 |
+
|
28 |
+
| Link | Type | Size/GB | Notes |
|
29 |
+
|:-----|:-----|--------:|:------|
|
30 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ1_S.gguf) | i1-IQ1_S | 15.0 | for the desperate |
|
31 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ1_M.gguf) | i1-IQ1_M | 16.0 | mostly desperate |
|
32 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ2_XXS.gguf) | i1-IQ2_XXS | 18.7 | |
|
33 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ2_XS.gguf) | i1-IQ2_XS | 20.8 | |
|
34 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ2_S.gguf) | i1-IQ2_S | 21.8 | |
|
35 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ2_M.gguf) | i1-IQ2_M | 23.7 | |
|
36 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q2_K.gguf) | i1-Q2_K | 25.9 | IQ3_XXS probably better |
|
37 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ3_XXS.gguf) | i1-IQ3_XXS | 27.0 | lower quality |
|
38 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ3_XS.gguf) | i1-IQ3_XS | 28.6 | |
|
39 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ3_S.gguf) | i1-IQ3_S | 30.3 | beats Q3_K* |
|
40 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q3_K_S.gguf) | i1-Q3_K_S | 30.3 | IQ3_XS probably better |
|
41 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ3_M.gguf) | i1-IQ3_M | 31.4 | |
|
42 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q3_K_M.gguf) | i1-Q3_K_M | 33.7 | IQ3_S probably better |
|
43 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q3_K_L.gguf) | i1-Q3_K_L | 36.6 | IQ3_M probably better |
|
44 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-IQ4_XS.gguf) | i1-IQ4_XS | 37.2 | |
|
45 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q4_0.gguf) | i1-Q4_0 | 39.1 | fast, low quality |
|
46 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q4_K_S.gguf) | i1-Q4_K_S | 39.7 | optimal size/speed/quality |
|
47 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q4_K_M.gguf) | i1-Q4_K_M | 41.8 | fast, recommended |
|
48 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q5_K_S.gguf) | i1-Q5_K_S | 47.9 | |
|
49 |
+
| [GGUF](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q5_K_M.gguf) | i1-Q5_K_M | 49.2 | |
|
50 |
+
| [PART 1](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part1of2) [PART 2](https://huggingface.co/mradermacher/Midnight-Miqu-70B-v1.5-i1-GGUF/resolve/main/Midnight-Miqu-70B-v1.5.i1-Q6_K.gguf.part2of2) | i1-Q6_K | 57.0 | practically like static Q6_K |
|
51 |
+
|
52 |
+
Here is a handy graph by ikawrakow comparing some lower-quality quant
|
53 |
+
types (lower is better):
|
54 |
+
|
55 |
+
![image.png](https://www.nethype.de/huggingface_embed/quantpplgraph.png)
|
56 |
+
|
57 |
+
And here are Artefact2's thoughts on the matter:
|
58 |
+
https://gist.github.com/Artefact2/b5f810600771265fc1e39442288e8ec9
|
59 |
+
|
60 |
+
## FAQ / Model Request
|
61 |
+
|
62 |
+
See https://huggingface.co/mradermacher/model_requests for some answers to
|
63 |
+
questions you might have and/or if you want some other model quantized.
|
64 |
+
|
65 |
+
## Thanks
|
66 |
+
|
67 |
+
I thank my company, [nethype GmbH](https://www.nethype.de/), for letting
|
68 |
+
me use its servers and providing upgrades to my workstation to enable
|
69 |
+
this work in my free time.
|
70 |
+
|
71 |
+
<!-- end -->
|
imatrix.dat
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:3d370094d07f518f1eea5b6f5620b788fc53af3dd44c4dc9b1fc93051ed1523c
|
3 |
+
size 24922254
|