|
--- |
|
language: |
|
- en |
|
--- |
|
|
|
--- |
|
|
|
# To request a quant, open an new discussion in the Community tab (preferably with the url in the title) |
|
|
|
--- |
|
|
|
# Mini-FAQ |
|
|
|
## What does the "-i1" mean in "-i1-GGUF"? |
|
|
|
tbd |
|
|
|
## Why are you doing this? |
|
|
|
tbd |
|
|
|
## You have amazing hardware!?!?! |
|
|
|
tbd |
|
|
|
## Why don't you use gguf-split? |
|
|
|
TL;DR: I don't have the hardware/resources for that. |
|
|
|
Long answer: gguf-split requires a full copy for every quant. |
|
Unlike what many people think, my hardware is rather outdated and not very fast. The extra processing that gguf-split requires |
|
either runs out of space on my systems with fast disk, or takes a very long time and a lot of I/O bandwidth on the slower |
|
disks, all of which already run at their limits. Supporting gguf-split would mean |
|
|
|
While this is the blocking reason, I also find it less than ideal that yet another incompatible file format was created that |
|
requires special tools to manage, instead of supporting the tens of thousands of existing quants, of which the vast majority |
|
could just be mmapped together into memory from split files. That doesn't keep me from supporting it, but it would have |
|
been nice to look at the existing reality and/or consult the community before throwing yet another hard to support format out |
|
there without thinking. |
|
|
|
There are some developments to make this less of a pain, and I will revisit this issue from time to time to see if it has become feasible. |
|
|
