Update README.md
Browse files
README.md
CHANGED
@@ -15,20 +15,6 @@ Ma-layala-mba is a state-of-the-art S6 SSM model specifically crafted for Indic
|
|
15 |
- **License**: GNU General Public License v3.0
|
16 |
- **Training Precision**: bfloat16
|
17 |
|
18 |
-
## Benchmark Results
|
19 |
-
|
20 |
-
Benchmarking was performed using LLM-Autoeval on an RTX 3090 on Runpod.
|
21 |
-
|
22 |
-
| Benchmark | Llama 2 Chat | Tamil Llama v0.2 Instruct | Telugu Llama Instruct | Ma-layala-mba |
|
23 |
-
|---------------------|--------------|---------------------------|-----------------------|---------------|
|
24 |
-
| ARC Challenge (25-shot) | 52.9 | 53.75 | 52.47 | 54.20 |
|
25 |
-
| TruthfulQA (0-shot) | 45.57 | 47.23 | 48.47 | 49.00 |
|
26 |
-
| Hellaswag (10-shot) | 78.55 | 76.11 | 76.13 | 77.50 |
|
27 |
-
| Winogrande (5-shot) | 71.74 | 73.95 | 71.74 | 74.00 |
|
28 |
-
| AGI Eval (0-shot) | 29.3 | 30.95 | 28.44 | 31.00 |
|
29 |
-
| BigBench (0-shot) | 32.6 | 33.08 | 32.99 | 33.50 |
|
30 |
-
| **Average** | 51.78 | 52.51 | 51.71 | 52.70 |
|
31 |
-
|
32 |
## Example Usage
|
33 |
|
34 |
Here's a quick example to get you started with the Ma-layala-mba model:
|
@@ -74,10 +60,4 @@ Please be aware that this model has not undergone comprehensive detoxification o
|
|
74 |
|
75 |
Get to know the creators behind Ma-layala-mba and follow their work:
|
76 |
|
77 |
-
- **Alosh Denny**
|
78 |
-
|
79 |
-
We hope Ma-layala-mba proves to be a valuable tool in your NLP projects and contributes significantly to the advancement of Indic language models.
|
80 |
-
|
81 |
-
---
|
82 |
-
|
83 |
-
Feel free to modify any sections to better suit your specific model details or additional features!
|
|
|
15 |
- **License**: GNU General Public License v3.0
|
16 |
- **Training Precision**: bfloat16
|
17 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
18 |
## Example Usage
|
19 |
|
20 |
Here's a quick example to get you started with the Ma-layala-mba model:
|
|
|
60 |
|
61 |
Get to know the creators behind Ma-layala-mba and follow their work:
|
62 |
|
63 |
+
- **Alosh Denny**
|
|
|
|
|
|
|
|
|
|
|
|