|
--- |
|
language: |
|
- multilingual |
|
thumbnail: url to a thumbnail used in social sharing |
|
tags: |
|
- coding |
|
- moe |
|
license: mit |
|
base_model: ContextualAI/Contextual_KTO_Mistral_PairRM |
|
pipeline_tag: text-generation |
|
--- |
|
|
|
## Usage |
|
NebulaNet-v2: An MOE of 4 7b expert models. |
|
It is good at coding and multi language translation. It should be fluent at chat and math too. |
|
|
|
The 4x7b merged model performs much better than the original Contextual_KTO_Mistral_PairRM on both coding and multilingual text generation in my observation. |
|
|
|
## mergekit config |
|
``` |
|
base_model: ContextualAI/Contextual_KTO_Mistral_PairRM |
|
experts: |
|
- source_model: ContextualAI/Contextual_KTO_Mistral_PairRM |
|
positive_prompts: |
|
- "chat" |
|
- "assistant" |
|
- "tell me" |
|
- "explain" |
|
- "I want" |
|
- source_model: Nexusflow/Starling-LM-7B-beta |
|
positive_prompts: |
|
- "code" |
|
- "python" |
|
- "javascript" |
|
- "programming" |
|
- "algorithm" |
|
- source_model: snorkelai/Snorkel-Mistral-PairRM-DPO |
|
positive_prompts: |
|
- "" |
|
- source_model: mlabonne/NeuralDaredevil-7B |
|
positive_prompts: |
|
- "reason" |
|
- "math" |
|
- "mathematics" |
|
- "solve" |
|
- "count" |
|
``` |
