hromi commited on
Commit
923c305
1 Parent(s): 30c268e

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +1 -1
README.md CHANGED
@@ -22,7 +22,7 @@ As You may notice, the dataset mostly consists of specially modified winogrande
22
 
23
  But before flagging this (or recommending this to be flagged), consider this:
24
 
25
- Subtle DPO-Contamination with modified Winogrande in such an extent that the average accuracy of all 5-non Winogrande metrics (e.g. including also MMLU and GSM8K) is 0.2% higher than the base model.
26
 
27
  | Model | ARC | HellaSwag | MMLU | Truthful QA | GSM8K | Average |
28
  | -----------------------------|------ | --------- | ---- | ----------- | ------| ------- |
 
22
 
23
  But before flagging this (or recommending this to be flagged), consider this:
24
 
25
+ Subtle DPO-Contamination with modified Winogrande causes the average accuracy of all 5-non Winogrande metrics (e.g. including also MMLU and GSM8K) to be 0.2% higher than the underlying model.
26
 
27
  | Model | ARC | HellaSwag | MMLU | Truthful QA | GSM8K | Average |
28
  | -----------------------------|------ | --------- | ---- | ----------- | ------| ------- |