File size: 1,750 Bytes
0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 cb96f87 0f2ddb3 89b403e 0f2ddb3 cb96f87 0f2ddb3 8b6574d 0f2ddb3 cb96f87 89b403e b63e307 89b403e cb96f87 0f2ddb3 cb96f87 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 |
---
license: apache-2.0
---
<style>
img{
user-select: none;
transition: all 0.2s ease;
border-radius: .5rem;
}
img:hover{
transform: rotate(2deg);
filter: invert(100%);
}
@import url('https://fonts.googleapis.com/css2?family=Vollkorn:ital,wght@0,400..900;1,400..900&display=swap');
</style>
<div style="background-color: transparent; border-radius: .5rem; padding: 2rem; font-family: monospace; font-size: .85rem; text-align: justify;">
cubby consistently outperforms every sota model below 600m parameters, outperforms base 1b models and is competitive with the best ones.
#### benchmarks
zero-shot evaluations performed on current sota ~0.5b models and palmer-004.
| Parameters | Model | MMLU | ARC-C | HellaSwag | PIQA | Winogrande | Average |
| -----------|--------------------------------|-------|-------|-----------|--------|------------|---------|
| 0.5b | qwen2 |**0.4413**| 0.2892| 0.4905 | 0.6931 | 0.5699 | 0.4968 |
| 0.5b | palmer-004-turbo |0.2736|0.3558|0.6179|0.7367 | 0.6117 |0.5191|
| 1.1b | palmer-004 | 0.2661| 0.3490| 0.6173 |**0.7481**|**0.6417** |0.5244|
| 0.5b | cubby |0.2617|**0.3729**|**0.6288**|0.7437| 0.6227 |**0.5260**|
#### supporters
<a href="https://ko-fi.com/appvoid" target="_blank"><img src="https://cdn.buymeacoffee.com/buttons/v2/default-yellow.png" alt="Buy Me A Coffee" style="height: 34px !important; margin-top: -4px;width: 128px !important; filter: contrast(2) grayscale(100%) brightness(100%);" ></a>
</div> |