Hugging Face's logo

junnyu
/
chinese_GAU-alpha-char_L-24_H-768

Fill-Mask
Transformers
PyTorch
Chinese
gau_alpha
gau alpha
torch
Model card Files Community
junnyu commited on
Commit
119c0c1
•
1 Parent(s): b28258a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +23 -10
README.md CHANGED
@@ -27,22 +27,35 @@ pip install gau_alpha
27
  | BERT | 60.06 | 56.80 | 72.41 | 79.56 | 73.93 | 78.62 | 83.93 |
28
  | RoBERTa | 60.64 | 58.06 | 74.05 | **81.24** | 76.00 | 87.50 | 84.50 |
29
  | RoFormer | 60.91 | 57.54 | 73.52 | 80.92 | 76.07 | 86.84 | 84.63 |
30
- | RoFormer-pytorch | 60.60 | 57.51 | 74.44 | 80.79 | 75.67 | 86.84 | 84.77 |
31
  | RoFormerV2<sup>*</sup> | 60.87 | 56.54 | 72.75 | 80.34 | 75.36 | 80.92 | 84.67 |
32
- | RoFormerV2<sup>*</sup>-pytorch | **62.87** | **59.03** | **76.20** | 80.85 | **79.73** | **87.82** | **91.87** |
33
  | GAU-α | 61.41 | 57.76 | 74.17 | 81.82 | 75.86 | 79.93 | 85.67 |
34
- | GAU-α-pytorch（Adafactor）(本仓库代码) | 61.18 | 57.52 | 73.42 | 80.91 | 75.69 | 80.59 | 85.5 |
35
- | GAU-α-pytorch（AdamW wd0.01 warm_decay0.1）(本仓库代码) | 60.68 | 57.95 | 73.08 | 81.02 | 75.36 | 81.25 | 83.93 |
 
36
 
37
-
38
- ### CLUE-1.0-test榜单分类任务结果，base版本。
39
 
40
  | | iflytek | tnews | afqmc | cmnli | ocnli | wsc | csl |
41
  | :-----: | :-----: | :---: | :---: | :---: | :---: | :---: | :---: |
42
- | RoFormer-pytorch | 59.54 | 57.34 | 74.46 | 80.23 | 73.67 | 80.69 | 84.57 |
43
- | RoFormerV2<sup>*</sup>-pytorch| 63.15 | 58.24 | 75.42 | 80.59 | 74.17 | 83.79 | 83.73 |
44
- | GAU-α-pytorch（Adafactor）(本仓库代码) | 61.38 | 57.08 | 74.05 | 80.37 | 73.53 | 74.83 | 85.6 |
45
- | GAU-α-pytorch（AdamW wd0.01 warm_decay0.1）(本仓库代码) | 60.54 | 57.67 | 72.44 | 80.32 | 72.97 | 76.55 | 84.13 |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
46
 
47
  # Usage
48
  ```python
 
27
  | BERT | 60.06 | 56.80 | 72.41 | 79.56 | 73.93 | 78.62 | 83.93 |
28
  | RoBERTa | 60.64 | 58.06 | 74.05 | **81.24** | 76.00 | 87.50 | 84.50 |
29
  | RoFormer | 60.91 | 57.54 | 73.52 | 80.92 | 76.07 | 86.84 | 84.63 |
 
30
  | RoFormerV2<sup>*</sup> | 60.87 | 56.54 | 72.75 | 80.34 | 75.36 | 80.92 | 84.67 |
 
31
  | GAU-α | 61.41 | 57.76 | 74.17 | 81.82 | 75.86 | 79.93 | 85.67 |
32
+ | RoFormerV2-pytorch| **62.87** | **59.03** | **76.20** | 80.85 | **79.73** | **87.82** | **91.87** |
33
+ | GAU-α-pytorch（Adafactor） | 61.18 | 57.52 | 73.42 | 80.91 | 75.69 | 80.59 | 85.5 |
34
+ | GAU-α-pytorch（AdamW wd0.01 warmup0.1） | 60.68 | 57.95 | 73.08 | 81.02 | 75.36 | 81.25 | 83.93 |
35
 
36
+ ### CLUE-test榜单分类任务结果，base版本。
 
37
 
38
  | | iflytek | tnews | afqmc | cmnli | ocnli | wsc | csl |
39
  | :-----: | :-----: | :---: | :---: | :---: | :---: | :---: | :---: |
40
+ | RoFormerV2-pytorch | **63.15** | **58.24** | **75.42** | **80.59** | **74.17** | **83.79** | 83.73 |
41
+ | GAU-α-pytorch（Adafactor） | 61.38 | 57.08 | 74.05 | 80.37 | 73.53 | 74.83 | **85.6** |
42
+ | GAU-α-pytorch（AdamW wd0.01 warmup0.1） | 60.54 | 57.67 | 72.44 | 80.32 | 72.97 | 76.55 | 84.13 |
43
+
44
+
45
+ ### CLUE-dev集榜单阅读理解和NER结果
46
+
47
+ | | cmrc2018 | c3 | chid | cluener |
48
+ | :-----: | :-----: | :---: | :---: | :---: |
49
+ | BERT | 56.17 | 60.54 | 85.69 | 79.45 |
50
+ | RoBERTa | 56.54 | 67.66 | 86.71 | 79.47 |
51
+ | RoFormer | 56.26 | 67.24 | 86.57 | 79.72 |
52
+ | RoFormerV2<sup>*</sup> | 57.91 | 64.62 | 85.09 | **81.08** |
53
+ | GAU-α | **58.09** | **68.24** | **87.91** | 80.01 |
54
+
55
+ ### 注：
56
+ - 其中RoFormerV2<sup>*</sup>表示的是未进行多任务学习的RoFormerV2模型，该模型苏神并未开源，感谢苏神的提醒。
57
+ - 其中不带有pytorch后缀结果都是从[GAU-alpha](https://github.com/ZhuiyiTechnology/GAU-alpha)仓库复制过来的。
58
+ - 其中带有pytorch后缀的结果都是自己训练得出的。
59
 
60
  # Usage
61
  ```python