File size: 1,150 Bytes
9fb3d4e ad80d21 9fb3d4e ad80d21 9fb3d4e 971c35d ad80d21 971c35d ad80d21 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 |
---
language:
- multilingual
thumbnail: url to a thumbnail used in social sharing
tags:
- coding
- moe
license: mit
base_model: ContextualAI/Contextual_KTO_Mistral_PairRM
pipeline_tag: text-generation
---
## Usage
NebulaNet-v2: An MOE of 4 7b expert models.
It is good at coding and multi language translation. It should be fluent at chat and math too.
The 4x7b merged model performs much better than the original Contextual_KTO_Mistral_PairRM on both coding and multilingual text generation in my observation.
## mergekit config
```
base_model: ContextualAI/Contextual_KTO_Mistral_PairRM
experts:
- source_model: ContextualAI/Contextual_KTO_Mistral_PairRM
positive_prompts:
- "chat"
- "assistant"
- "tell me"
- "explain"
- "I want"
- source_model: Nexusflow/Starling-LM-7B-beta
positive_prompts:
- "code"
- "python"
- "javascript"
- "programming"
- "algorithm"
- source_model: snorkelai/Snorkel-Mistral-PairRM-DPO
positive_prompts:
- ""
- source_model: mlabonne/NeuralDaredevil-7B
positive_prompts:
- "reason"
- "math"
- "mathematics"
- "solve"
- "count"
``` |