Crystalcareai
commited on
Commit
•
74927a7
1
Parent(s):
b27a11c
Update README.md
Browse files
README.md
CHANGED
@@ -114,18 +114,6 @@ The model was trained using a state-of-the-art distillation pipeline and an inst
|
|
114 |
|
115 |
Llama-3.1-SuperNova-Lite excels in both benchmark performance and real-world applications, providing the power of large-scale models in a more compact, efficient form ideal for organizations seeking high performance with reduced resource requirements.
|
116 |
|
117 |
-
# Evaluations
|
118 |
-
Here are our internal benchmarks using the main branch of lm evaluation harness:
|
119 |
-
|
120 |
-
| Benchmark | SuperNova-Lite | Llama-3.1-8b-Instruct |
|
121 |
-
|-------------|----------------|----------------------|
|
122 |
-
| IF_Eval | 81.1 | 77.4 |
|
123 |
-
| MMLU Pro | 38.7 | 37.7 |
|
124 |
-
| TruthfulQA | 64.4 | 55.0 |
|
125 |
-
| BBH | 51.1 | 50.6 |
|
126 |
-
| GPQA | 31.2 | 29.02 |
|
127 |
-
|
128 |
-
The script used for evaluation can be found inside this repository under /eval.sh, or click [here](https://huggingface.co/arcee-ai/Llama-3.1-SuperNova-Lite/blob/main/eval).
|
129 |
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
130 |
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_arcee-ai__Llama-3.1-SuperNova-Lite)
|
131 |
|
|
|
114 |
|
115 |
Llama-3.1-SuperNova-Lite excels in both benchmark performance and real-world applications, providing the power of large-scale models in a more compact, efficient form ideal for organizations seeking high performance with reduced resource requirements.
|
116 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
117 |
# [Open LLM Leaderboard Evaluation Results](https://huggingface.co/spaces/open-llm-leaderboard/open_llm_leaderboard)
|
118 |
Detailed results can be found [here](https://huggingface.co/datasets/open-llm-leaderboard/details_arcee-ai__Llama-3.1-SuperNova-Lite)
|
119 |
|