Update README.md
Browse files
README.md
CHANGED
|
@@ -11,6 +11,13 @@ tags:
|
|
| 11 |
|
| 12 |
A frankenMoE not only using far better methodology and fundamental understanding of SMoE, but completely focused around intellectual roleplay. It may have a bit of a redundancy issue, to battle this, try to keep things fresh with the model by either introducing new concepts often, or through [drμgs](https://github.com/EGjoni/DRUGS). (no not that kind)
|
| 13 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 14 |
# "[What is a Mixture of Experts (MoE)?](https://huggingface.co/blog/moe)"
|
| 15 |
### (from the MistralAI papers...click the quoted question above to navigate to it directly.)
|
| 16 |
|
|
|
|
| 11 |
|
| 12 |
A frankenMoE not only using far better methodology and fundamental understanding of SMoE, but completely focused around intellectual roleplay. It may have a bit of a redundancy issue, to battle this, try to keep things fresh with the model by either introducing new concepts often, or through [drμgs](https://github.com/EGjoni/DRUGS). (no not that kind)
|
| 13 |
|
| 14 |
+
The config looks like this...(detailed version is in the files and versions):
|
| 15 |
+
- [mlabonne/Beagle14-7B](https://huggingface.co/mlabonne/Beagle14-7B) - base
|
| 16 |
+
- [fblgit/una-cybertron-7b-v3-OMA](https://huggingface.co/fblgit/una-cybertron-7b-v3-OMA) - expert #1
|
| 17 |
+
- [rwitz/go-bruins-v2](https://huggingface.co/rwitz/go-bruins-v2) - expert #2
|
| 18 |
+
- [mlabonne/Beagle14-7B](https://huggingface.co/mlabonne/Beagle14-7B) - expert #3
|
| 19 |
+
- [mlabonne/Beagle14-7B](https://huggingface.co/mlabonne/Beagle14-7B) - expert #4
|
| 20 |
+
|
| 21 |
# "[What is a Mixture of Experts (MoE)?](https://huggingface.co/blog/moe)"
|
| 22 |
### (from the MistralAI papers...click the quoted question above to navigate to it directly.)
|
| 23 |
|