https://huggingface.co./AuraIndustries/Aura-MoE-2x4B-v2

#497
by jeiku - opened

https://huggingface.co./AuraIndustries/Aura-MoE-2x4B-v2

Actually performed an SFT healing stage this time then the KTO. It seems to perform much better in my brief testing.

Probably won't be able to apply imatrix to this one either.

Thank you!

You never know...

mradermacher changed discussion status to closed

Wait did it actually work this time?

Wait, what? Really?

Yes, yes it did11!!

Serious question: did yo "just" take the wonderful eternal 4B from before, made an MoE and then some healing?

No, I used two older instruct 4B I did and then merged with a random gate, then healed with RP Data, then applied a FFT KTO over top.

Uhu, interesting...

Sign up or log in to comment