File size: 323 Bytes
9aab7ae b9633b2 |
1 2 3 4 5 6 7 8 9 |
---
license: mit
language:
- en
base_model: meta-llama/Meta-Llama-3-8B-Instruct
---
# Llama 3 8b Instruct MOE
Llama 3 8b Instruct base model converted to MOE style by randomly partitioning the FFN layers of each decoder layer into 8 experts of the same size. Weights are directly taken from the llama3 instruct base model. |