VAGOsolutions
/

Llama-3-SauerkrautLM-8b-Instruct

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

DavidGF commited on Apr 29

Commit

37127c4

•

1 Parent(s): 4a77598

Update README.md

Files changed (1) hide show

README.md +7 -6

README.md CHANGED Viewed

@@ -334,15 +334,16 @@ Llama-3-SauerkrautLM-8b-Instruct  7.65625
 ```
 **German RAG LLM Evaluation**
 ```
 |                         Task                         |Version|Metric|Value|   |Stderr|
 |------------------------------------------------------|------:|------|----:|---|-----:|
-|all                                                   |       |acc   |0.905|±  |0.0086|
-|community:german_rag_eval:_average:0                  |       |acc   |0.905|±  |0.0086|
-|community:german_rag_eval:choose_context_by_question:0|      0|acc   |0.896|±  |0.0097|
-|community:german_rag_eval:choose_question_by_context:0|      0|acc   |0.826|±  |0.0120|
-|community:german_rag_eval:context_question_match:0    |      0|acc   |0.987|±  |0.0036|
-|community:german_rag_eval:question_answer_match:0     |      0|acc   |0.911|±  |0.0090|
 ```
 ## Disclaimer

 ```
 **German RAG LLM Evaluation**
+corrected result after FIX: https://github.com/huggingface/lighteval/pull/171
 ```
 |                         Task                         |Version|Metric|Value|   |Stderr|
 |------------------------------------------------------|------:|------|----:|---|-----:|
+|all                                                   |       |acc   |0.910|±  |0.0084|
+|community:german_rag_eval:_average:0                  |       |acc   |0.910|±  |0.0084|
+|community:german_rag_eval:choose_context_by_question:0|      0|acc   |0.928|±  |0.0082|
+|community:german_rag_eval:choose_question_by_context:0|      0|acc   |0.824|±  |0.0120|
+|community:german_rag_eval:context_question_match:0    |      0|acc   |0.982|±  |0.0042|
+|community:german_rag_eval:question_answer_match:0     |      0|acc   |0.906|±  |0.0092|
 ```
 ## Disclaimer