https://huggingface.co./AuraIndustries/Aura-MoE-2x4B-v2
#497
by
jeiku
- opened
https://huggingface.co./AuraIndustries/Aura-MoE-2x4B-v2
Actually performed an SFT healing stage this time then the KTO. It seems to perform much better in my brief testing.
Probably won't be able to apply imatrix to this one either.
Thank you!
You never know...
mradermacher
changed discussion status to
closed
Wait did it actually work this time?
Wait, what? Really?
Yes, yes it did11!!
Serious question: did yo "just" take the wonderful eternal 4B from before, made an MoE and then some healing?
No, I used two older instruct 4B I did and then merged with a random gate, then healed with RP Data, then applied a FFT KTO over top.
Uhu, interesting...