Update README.md
Browse files
README.md
CHANGED
@@ -84,6 +84,26 @@ Evaluated on the SQuAD 2.0 dev set with the [official eval script](https://works
|
|
84 |
"NoAns_total": 5945
|
85 |
```
|
86 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
87 |
## Authors
|
88 |
**Branden Chan:** [email protected]
|
89 |
**Timo M枚ller:** [email protected]
|
|
|
84 |
"NoAns_total": 5945
|
85 |
```
|
86 |
|
87 |
+
Using the official [question answering notebook](https://github.com/huggingface/notebooks/blob/main/examples/question_answering.ipynb) from `transformers` yields:
|
88 |
+
|
89 |
+
```
|
90 |
+
{'HasAns_exact': 77.93522267206478,
|
91 |
+
'HasAns_f1': 83.93715663402219,
|
92 |
+
'HasAns_total': 5928,
|
93 |
+
'NoAns_exact': 81.90075693860386,
|
94 |
+
'NoAns_f1': 81.90075693860386,
|
95 |
+
'NoAns_total': 5945,
|
96 |
+
'best_exact': 79.92082877116145,
|
97 |
+
'best_exact_thresh': 0.0,
|
98 |
+
'best_f1': 82.91749890730902,
|
99 |
+
'best_f1_thresh': 0.0,
|
100 |
+
'exact': 79.92082877116145,
|
101 |
+
'f1': 82.91749890730917,
|
102 |
+
'total': 11873}
|
103 |
+
```
|
104 |
+
|
105 |
+
which is consistent with the officially reported results.
|
106 |
+
|
107 |
## Authors
|
108 |
**Branden Chan:** [email protected]
|
109 |
**Timo M枚ller:** [email protected]
|