Rules and Requests [Status: Accepting Requests]
How to get your model tested
If you want me to manually test a model for you, make sure that:
- You are the author of said model (I do not accept requests from users).
- A version in GGUF Q8_0 must be available, and preferably linked to me.
- The model's size must NOT be above 13B. I will heavily prioritize smaller models.
- The instruct format must be ChatML or L3 instruct. Alpaca is tolerated but you'll have to specify the exact variant.
- If you have a favorite sampling preset, make sure to include it.
- Your model must behave correctly at 16K context length with KoboldCPP's automatic rope thingy
I'll test whenever I get the time to. Post your requests in this topic.
Could you please test my model? https://huggingface.co./mradermacher/Halu-8B-Llama3-v0.3-i1-GGUF, Thank you!
I think my Silver Sun merges would do good on these https://huggingface.co./ABX-AI/Silver-Sun-11B-GGUF-IQ-Imatrix :)
Already posted in the other thread with some results, I think I did the test exactly as instructed although I did not try all of the sampler presets provided
I think my Silver Sun merges would do good on these https://huggingface.co./ABX-AI/Silver-Sun-11B-GGUF-IQ-Imatrix :)
Already posted in the other thread with some results, I think I did the test exactly as instructed although I did not try all of the sampler presets provided
Sorry, I can't.
- Its a L2 model with max context length of 4096. Even rope extended, L2 models can't scale up to 16K properly, so I don't really have an use for it.
- It seems to be using Alpaca, for which I don't have made instruct files for, yet.
Feel free, to run it on your own.
SOLAR is not really L2, the base is L2 but all the layers are Mistral 7b + the brand new extra training they added, so it's Mistral with added training in practice. Otherwise, no problem, I am not actively looking to get benchmarks or anything, was just curious since you suggested this test in the coping thread.
Didn't know that was a thing. Now I'm sorta curious anyway. I'll probably check it even out of general curiosity (test or not i mean).
There's a very nice paper about it here - https://arxiv.org/abs/2312.15166 (which is where I learned this from)