Adding the Open Portuguese LLM Leaderboard Evaluation Results

README.md

@@ -1,8 +1,155 @@
 ---
-license: mit
 language: pt
+license: mit
 base_model: mistralai/Mistral-7B-Instruct-v0.1
 pipeline_tag: text-generation
+model-index:
+- name: bluearara-7B-instruct-v02
+  results:
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: ENEM Challenge (No Images)
+      type: eduagarcia/enem_challenge
+      split: train
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc
+      value: 28.48
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: BLUEX (No Images)
+      type: eduagarcia-temp/BLUEX_without_images
+      split: train
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc
+      value: 21.84
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: OAB Exams
+      type: eduagarcia/oab_exams
+      split: train
+      args:
+        num_few_shot: 3
+    metrics:
+    - type: acc
+      value: 27.84
+      name: accuracy
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Assin2 RTE
+      type: assin2
+      split: test
+      args:
+        num_few_shot: 15
+    metrics:
+    - type: f1_macro
+      value: 56.04
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: Assin2 STS
+      type: eduagarcia/portuguese_benchmark
+      split: test
+      args:
+        num_few_shot: 15
+    metrics:
+    - type: pearson
+      value: 21.3
+      name: pearson
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: FaQuAD NLI
+      type: ruanchaves/faquad-nli
+      split: test
+      args:
+        num_few_shot: 15
+    metrics:
+    - type: f1_macro
+      value: 45.33
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: HateBR Binary
+      type: ruanchaves/hatebr
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: f1_macro
+      value: 84.14
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: PT Hate Speech Binary
+      type: hate_speech_portuguese
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: f1_macro
+      value: 52.17
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
+  - task:
+      type: text-generation
+      name: Text Generation
+    dataset:
+      name: tweetSentBR
+      type: eduagarcia/tweetsentbr_fewshot
+      split: test
+      args:
+        num_few_shot: 25
+    metrics:
+    - type: f1_macro
+      value: 66.91
+      name: f1-macro
+    source:
+      url: https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard?query=fernandosola/bluearara-7B-instruct-v02
+      name: Open Portuguese LLM Leaderboard
 ---
 
 
@@ -201,4 +348,23 @@ llm.create_chat_completion(
 ## Carbon Footprint - Environmental Concerns
 A pegada de Carbono deste subprojeto consumiu 6 de horas de computação acumuladas em GPUs em hardware do tipo NVIDIA RTX 3070 e CPU Intel Xeon E5. Toda energia consumida foi de origem solar, com produção de 0.0g de CO2 ou gases de efeitos estufa.
 
-The Carbon footprint of this subproject consumed 6 hours of computing accumulated on hardware as NVIDIA RTX 3070 GPUs (100W-270W) and Intel Xeon E5 CPU (100W). All energy consumed was of solar origin, producing 0.0g of CO2 or greenhouse gases.
+The Carbon footprint of this subproject consumed 6 hours of computing accumulated on hardware as NVIDIA RTX 3070 GPUs (100W-270W) and Intel Xeon E5 CPU (100W). All energy consumed was of solar origin, producing 0.0g of CO2 or greenhouse gases.
+
+
+# Open Portuguese LLM Leaderboard Evaluation Results  
+
+Detailed results can be found [here](https://huggingface.co/datasets/eduagarcia-temp/llm_pt_leaderboard_raw_results/tree/main/fernandosola/bluearara-7B-instruct-v02) and on the [🚀 Open Portuguese LLM Leaderboard](https://huggingface.co/spaces/eduagarcia/open_pt_llm_leaderboard)
+
+|          Metric          |  Value  |
+|--------------------------|---------|
+|Average                   |**44.89**|
+|ENEM Challenge (No Images)|    28.48|
+|BLUEX (No Images)         |    21.84|
+|OAB Exams                 |    27.84|
+|Assin2 RTE                |    56.04|
+|Assin2 STS                |    21.30|
+|FaQuAD NLI                |    45.33|
+|HateBR Binary             |    84.14|
+|PT Hate Speech Binary     |    52.17|
+|tweetSentBR               |    66.91|
+