laurentiubp commited on
Commit
bb7ac06
1 Parent(s): 9aca0ab

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -1
README.md CHANGED
@@ -41,13 +41,18 @@ The model shows improved proficiency with the Catalan language while performing
41
  - *Summarization - both short form and long form*
42
  - *Sentiment analysis*
43
 
44
-
45
  **Model developers** [Laurentiu Petrea](https://www.linkedin.com/in/laurentiupetrea/) based on Llama-3 from Meta.
46
 
47
  **Model Architecture** CataLlama is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and direct preference optimisation (DPO) to align with human preferences for helpfulness and safety.
48
 
49
  **License** The model uses the llama-3 license available at: [https://llama.meta.com/llama3/license](https://llama.meta.com/llama3/license)
50
 
 
 
 
 
 
 
51
 
52
  ### Use with transformers
53
 
 
41
  - *Summarization - both short form and long form*
42
  - *Sentiment analysis*
43
 
 
44
  **Model developers** [Laurentiu Petrea](https://www.linkedin.com/in/laurentiupetrea/) based on Llama-3 from Meta.
45
 
46
  **Model Architecture** CataLlama is an auto-regressive language model that uses an optimized transformer architecture. The tuned versions use supervised fine-tuning (SFT) and direct preference optimisation (DPO) to align with human preferences for helpfulness and safety.
47
 
48
  **License** The model uses the llama-3 license available at: [https://llama.meta.com/llama3/license](https://llama.meta.com/llama3/license)
49
 
50
+ ## Benchmarks
51
+
52
+ | Model | CataLlama-v0.1-Instruct-SFT | CataLlama-v0.2-Instruct-SFT |
53
+ | ------------------ | --------------------------- | ------------------------------- |
54
+ | MMLU 5 shot | 55.28 | **59.35** |
55
+ | GSM8K cot 8 shot | 51.63 | **76.04** |
56
 
57
  ### Use with transformers
58