| license: mit | |
| base_model: | |
| - microsoft/Phi-3.5-MoE-instruct | |
| library_name: transformers | |
| # Model Card for Rasphi | |
| Rasphi (pronounced `rasp-fee`, though the name *may* be changed.) is a WIP architecture derrived from Microsoft's Phi 3.5 MoE / GRIN model. It aims to improve reasoning by having a dedicated reasoning stream in which half of all experts are allocated to it. Due to the experts being split directly in half, there is a high chance of instability / overall incoherence in both streams. | |
| ## Model Details | |
| ## Uses | |
| Rasphi can be used for research purposes and or finetuning to gauge the performance of the new architecture. However, it is __highly__ not recommended in its current state to be used for any user-facing applications. Or at all. |