laurentiubp
commited on
Commit
•
bb7ac06
1
Parent(s):
9aca0ab
Update README.md
Browse files
README.md
CHANGED
@@ -41,13 +41,18 @@ The model shows improved proficiency with the Catalan language while performing
|
|
41 |
- *Summarization - both short form and long form*
|
42 |
- *Sentiment analysis*
|
43 |
|
44 |
-
|
45 |
**Model developers** [Laurentiu Petrea](https://www.linkedin.com/in/laurentiupetrea/) based on Llama-3 from Meta.
|
46 |
|
47 |
**Model Architecture** CataLlama is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and direct preference optimisation (DPO) to align with human preferences for helpfulness and safety.
|
48 |
|
49 |
**License** The model uses the llama-3 license available at: [https://llama.meta.com/llama3/license](https://llama.meta.com/llama3/license)
|
50 |
|
|
|
|
|
|
|
|
|
|
|
|
|
51 |
|
52 |
### Use with transformers
|
53 |
|
|
|
41 |
- *Summarization - both short form and long form*
|
42 |
- *Sentiment analysis*
|
43 |
|
|
|
44 |
**Model developers** [Laurentiu Petrea](https://www.linkedin.com/in/laurentiupetrea/) based on Llama-3 from Meta.
|
45 |
|
46 |
**Model Architecture** CataLlama is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and direct preference optimisation (DPO) to align with human preferences for helpfulness and safety.
|
47 |
|
48 |
**License** The model uses the llama-3 license available at: [https://llama.meta.com/llama3/license](https://llama.meta.com/llama3/license)
|
49 |
|
50 |
+
## Benchmarks
|
51 |
+
|
52 |
+
| Model | CataLlama-v0.1-Instruct-SFT | CataLlama-v0.2-Instruct-SFT |
|
53 |
+
| ------------------ | --------------------------- | ------------------------------- |
|
54 |
+
| MMLU 5 shot | 55.28 | **59.35** |
|
55 |
+
| GSM8K cot 8 shot | 51.63 | **76.04** |
|
56 |
|
57 |
### Use with transformers
|
58 |
|