QuietImpostor/Rasphi-MoE-Instruct-Unfinetuned

Model Card for Rasphi

Rasphi (pronounced rasp-fee, though the name may be changed.) is a WIP architecture derrived from Microsoft's Phi 3.5 MoE / GRIN model. It aims to improve reasoning by having a dedicated reasoning stream in which half of all experts are allocated to it. Due to the experts being split directly in half, there is a high chance of instability / overall incoherence in both streams.

Model Details

Uses

Rasphi can be used for research purposes and or finetuning to gauge the performance of the new architecture. However, it is highly not recommended in its current state to be used for any user-facing applications. Or at all.