lmstudio-community/phi-4-GGUF
Text Generation
•
Updated
•
12.6k
•
25
LM Studio Community
community
AI & ML interests
Models quantized and uploaded by the LM Studio community, for the LM Studio community. Discord: https://discord.gg/aPQfnNkxGC
Recent Activity
View all activity
lmstudio-community's activity
bartowski
updated
a
model
about 23 hours ago
How do you load with oobabooga
1
#1 opened 1 day ago
by
spike4379
(Exit code: -1073740791). Unknown error. Try a different model and/or config
3
#1 opened 3 days ago
by
michabbb
yagilb
updated
a
model
3 days ago
bartowski
updated
a
model
3 days ago
bartowski
updated
a
model
5 days ago
llama.cpp error: 'error loading model architecture: unknown model architecture: 'deci''
1
#1 opened 5 days ago
by
iqbalk
Won't Work on LMStudio!
3
#1 opened 5 days ago
by
Lyte
Post
4410
Switching to
I posted a poll on twitter, and others have mentioned the interest in me using the convention of including the author name in the model path when I upload.
It has a couple advantages, first and foremost of course is ensuring clarity of who uploaded the original model (did Qwen upload Qwen2.6? Or did someone fine tune Qwen2.5 and named it 2.6 for fun?)
The second thing is that it avoids collisions, so if multiple people upload the same model and I try to quant them both, I would normally end up colliding and being unable to upload both
I'll be implementing the change next week, there are just two final details I'm unsure about:
First, should the files also inherit the author's name?
Second, what to do in the case that the author name + model name pushes us past the character limit?
Haven't yet decided how to handle either case, so feedback is welcome, but also just providing this as a "heads up"
author_model-nameI posted a poll on twitter, and others have mentioned the interest in me using the convention of including the author name in the model path when I upload.
It has a couple advantages, first and foremost of course is ensuring clarity of who uploaded the original model (did Qwen upload Qwen2.6? Or did someone fine tune Qwen2.5 and named it 2.6 for fun?)
The second thing is that it avoids collisions, so if multiple people upload the same model and I try to quant them both, I would normally end up colliding and being unable to upload both
I'll be implementing the change next week, there are just two final details I'm unsure about:
First, should the files also inherit the author's name?
Second, what to do in the case that the author name + model name pushes us past the character limit?
Haven't yet decided how to handle either case, so feedback is welcome, but also just providing this as a "heads up"
-
- 3 replies
bartowski
updated
5
models
5 days ago
lmstudio-community/Llama-3_1-Nemotron-51B-Instruct-GGUF
Text Generation
•
Updated
•
216
lmstudio-community/Qwen2-VL-2B-Instruct-GGUF
Text Generation
•
Updated
•
661
•
1
lmstudio-community/Qwen2-VL-7B-Instruct-GGUF
Image-Text-to-Text
•
Updated
•
1.88k
lmstudio-community/Qwen2-VL-72B-Instruct-GGUF
Image-Text-to-Text
•
Updated
•
221
lmstudio-community/QVQ-72B-Preview-GGUF
Image-Text-to-Text
•
Updated
•
279
bartowski
updated
2
models
19 days ago
Post
15569
Looks like Q4_0_N_M file types are going away
Before you panic, there's a new "preferred" method which is online (I prefer the term on-the-fly) repacking, so if you download Q4_0 and your setup can benefit from repacking the weights into interleaved rows (what Q4_0_4_4 was doing), it will do that automatically and give you similar performance (minor losses I think due to using intrinsics instead of assembly, but intrinsics are more maintainable)
You can see the reference PR here:
https://github.com/ggerganov/llama.cpp/pull/10446
So if you update your llama.cpp past that point, you won't be able to run Q4_0_4_4 (unless they add backwards compatibility back), but Q4_0 should be the same speeds (though it may currently be bugged on some platforms)
As such, I'll stop making those newer model formats soon, probably end of this week unless something changes, but you should be safe to download and Q4_0 quants and use those !
Also IQ4_NL supports repacking though not in as many shapes yet, but should get a respectable speed up on ARM chips, PR for that can be found here: https://github.com/ggerganov/llama.cpp/pull/10541
Remember, these are not meant for Apple silicon since those use the GPU and don't benefit from the repacking of weights
Before you panic, there's a new "preferred" method which is online (I prefer the term on-the-fly) repacking, so if you download Q4_0 and your setup can benefit from repacking the weights into interleaved rows (what Q4_0_4_4 was doing), it will do that automatically and give you similar performance (minor losses I think due to using intrinsics instead of assembly, but intrinsics are more maintainable)
You can see the reference PR here:
https://github.com/ggerganov/llama.cpp/pull/10446
So if you update your llama.cpp past that point, you won't be able to run Q4_0_4_4 (unless they add backwards compatibility back), but Q4_0 should be the same speeds (though it may currently be bugged on some platforms)
As such, I'll stop making those newer model formats soon, probably end of this week unless something changes, but you should be safe to download and Q4_0 quants and use those !
Also IQ4_NL supports repacking though not in as many shapes yet, but should get a respectable speed up on ARM chips, PR for that can be found here: https://github.com/ggerganov/llama.cpp/pull/10541
Remember, these are not meant for Apple silicon since those use the GPU and don't benefit from the repacking of weights
-
- 15 replies
Post
3922
VLMs are going through quite an open revolution AND on-device friendly sizes:
1. Google DeepMind w/ PaliGemma2 - 3B, 10B & 28B: google/paligemma-2-release-67500e1e1dbfdd4dee27ba48
2. OpenGVLabs w/ InternVL 2.5 - 1B, 2B, 4B, 8B, 26B, 38B & 78B: https://huggingface.co./collections/OpenGVLab/internvl-25-673e1019b66e2218f68d7c1c
3. Qwen w/ Qwen 2 VL - 2B, 7B & 72B: Qwen/qwen2-vl-66cee7455501d7126940800d
4. Microsoft w/ FlorenceVL - 3B & 8B: https://huggingface.co./jiuhai
5. Moondream2 w/ 0.5B: https://huggingface.co./vikhyatk/
What a time to be alive! 🔥
1. Google DeepMind w/ PaliGemma2 - 3B, 10B & 28B: google/paligemma-2-release-67500e1e1dbfdd4dee27ba48
2. OpenGVLabs w/ InternVL 2.5 - 1B, 2B, 4B, 8B, 26B, 38B & 78B: https://huggingface.co./collections/OpenGVLab/internvl-25-673e1019b66e2218f68d7c1c
3. Qwen w/ Qwen 2 VL - 2B, 7B & 72B: Qwen/qwen2-vl-66cee7455501d7126940800d
4. Microsoft w/ FlorenceVL - 3B & 8B: https://huggingface.co./jiuhai
5. Moondream2 w/ 0.5B: https://huggingface.co./vikhyatk/
What a time to be alive! 🔥
Post
16114
Old mixtral model quants may be broken!
Recently Slaren over on llama.cpp refactored the model loader - in a way that's super awesome and very powerful - but with it came breaking of support for "split tensor MoE models", which applies to older mixtral models
You may have seen my upload of one such older mixtral model, ondurbin/bagel-dpo-8x7b-v0.2, and with the newest changes it seems to be able to run without issue
If you happen to run into issues with any other old mixtral models, drop a link here and I'll try to remake them with the new changes so that we can continue enjoying them :)
Recently Slaren over on llama.cpp refactored the model loader - in a way that's super awesome and very powerful - but with it came breaking of support for "split tensor MoE models", which applies to older mixtral models
You may have seen my upload of one such older mixtral model, ondurbin/bagel-dpo-8x7b-v0.2, and with the newest changes it seems to be able to run without issue
If you happen to run into issues with any other old mixtral models, drop a link here and I'll try to remake them with the new changes so that we can continue enjoying them :)
-
- 2 replies