|
--- |
|
library_name: transformers |
|
tags: |
|
- mergekit |
|
- merge |
|
|
|
--- |
|
|
|
 |
|
|
|
niro is an improvement over the excellent [WizardLM-Evol-V2-Unfiltered](https://huggingface.co./trollek/danube2-1.8b-WizardLM-Evol-V2-Unfiltered) model, which at the time of writting is one of the best 1.8 billion parameters models. Keep in mind, nero is an un-trained merge, further improvements are to be released later. |
|
|
|
#### benchmarks |
|
|
|
zero-shot evaluations performed on current sota ~1.8b models. |
|
|
|
| Parameters | Model | MMLU | ARC | HellaSwag | PIQA | Winogrande | Average | |
|
| -----------|--------------------------------|-------|-------|-----------|--------|------------|---------| |
|
| 1.7b | smollm |27.65|**46.26**|65.74|76.06|60.93| 55.33| |
|
| 1.8B | danube |40.29|40.02|**72.40**|77.04|64.25| 58.80| |
|
| 1.8B | wizard |40.79|40.87|71.85|**78.02**|64.33| 59.17| |
|
| 1.8B | niro |**41.75**|40.96|72.07|77.97|**65.51**|**59.65**| |
|
|
