megatron_1.1_MoE_2x7B / mergekit_moe_config.yml
Eurdem's picture
Upload folder using huggingface_hub
e9fe6c4 verified
raw
history blame
No virus
461 Bytes
base_model: openchat/openchat-3.5-0106
gate_mode: hidden
dtype: bfloat16
experts:
- source_model: fblgit/UNA-TheBeagle-7b-v1
positive_prompts:
- "Mathematics"
- "Physics"
negative_prompts:
- "History"
- "Philosophy"
- source_model: berkeley-nest/Starling-LM-7B-alpha
positive_prompts:
- "Earth Sciences (Geology, Meteorology, Oceanography)"
- "Environmental Science"
negative_prompts:
- "Education"
- "Law"