Text Generation
Transformers
Safetensors
English
falcon_mamba
Eval Results
Inference Endpoints
IChahed commited on
Commit
7f04837
·
verified ·
1 Parent(s): 9bc71f5

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -2
README.md CHANGED
@@ -198,7 +198,7 @@ We evaluate our model on all benchmarks of the new leaderboard's version using t
198
  | `model name` |`IFEval`| `BBH` |`MATH LvL5`| `GPQA`| `MUSR`|`MMLU-PRO`|`Average`|
199
  |:--------------------------|:------:|:-----:|:---------:|:-----:|:-----:|:--------:|:-------:|
200
  | ***Pure SSM models*** | | | | | | | |
201
- | `FalconMamba-7B` |**33.36**| 19.88 | 3.63 |**8.05**|10.86| 14.47 |**15.04**|
202
  | `TRI-ML/mamba-7b-rw`<sup>*</sup>| 22.46 | 6.71 | 0.45 | 1.12 | 5.51 | 1.69 | 6.25 |
203
  |***Hybrid SSM-attention models*** | | | | | | |
204
  |`recurrentgemma-9b` | 30.76 | 14.80 | 4.83 | 4.70 | 6.60 | 17.88 | 13.20 |
@@ -218,7 +218,7 @@ Also, we evaluate our model on the benchmarks of the first leaderboard using `li
218
  | `model name` |`ARC`|`HellaSwag` |`MMLU` |`Winogrande`|`TruthfulQA`|`GSM8K`|`Average` |
219
  |:-----------------------------|:------:|:---------:|:-----:|:----------:|:----------:|:-----:|:----------------:|
220
  | ***Pure SSM models*** | | | | | | | |
221
- | `FalconMamba-7B`<sup>*</sup> |**62.03**| 80.82 | 62.11 | 73.64 |**53.42**| 52.54 | **64.09** |
222
  | `TRI-ML/mamba-7b-rw`<sup>*</sup> | 51.25 | 80.85 | 33.41 | 71.11 | 32.08 | 4.70 | 45.52 |
223
  |***Hybrid SSM-attention models***| | | | | | | |
224
  | `recurrentgemma-9b`<sup>**</sup> |52.00 | 80.40 | 60.50 | 73.60 | 38.60 | 42.60 | 57.95 |
 
198
  | `model name` |`IFEval`| `BBH` |`MATH LvL5`| `GPQA`| `MUSR`|`MMLU-PRO`|`Average`|
199
  |:--------------------------|:------:|:-----:|:---------:|:-----:|:-----:|:--------:|:-------:|
200
  | ***Pure SSM models*** | | | | | | | |
201
+ | `FalconMamba-7B` | 33.36 | 19.88 | 3.63 |8.05 |10.86 | 14.47 |**15.04**|
202
  | `TRI-ML/mamba-7b-rw`<sup>*</sup>| 22.46 | 6.71 | 0.45 | 1.12 | 5.51 | 1.69 | 6.25 |
203
  |***Hybrid SSM-attention models*** | | | | | | |
204
  |`recurrentgemma-9b` | 30.76 | 14.80 | 4.83 | 4.70 | 6.60 | 17.88 | 13.20 |
 
218
  | `model name` |`ARC`|`HellaSwag` |`MMLU` |`Winogrande`|`TruthfulQA`|`GSM8K`|`Average` |
219
  |:-----------------------------|:------:|:---------:|:-----:|:----------:|:----------:|:-----:|:----------------:|
220
  | ***Pure SSM models*** | | | | | | | |
221
+ | `FalconMamba-7B`<sup>*</sup> | 62.03 | 80.82 | 62.11 | 73.64 | 53.42 | 52.54 | **64.09** |
222
  | `TRI-ML/mamba-7b-rw`<sup>*</sup> | 51.25 | 80.85 | 33.41 | 71.11 | 32.08 | 4.70 | 45.52 |
223
  |***Hybrid SSM-attention models***| | | | | | | |
224
  | `recurrentgemma-9b`<sup>**</sup> |52.00 | 80.40 | 60.50 | 73.60 | 38.60 | 42.60 | 57.95 |