dl-ru commited on
Commit
1bdd317
·
verified ·
1 Parent(s): bb40463

New version with explicit predicate marking

Browse files
Files changed (6) hide show
  1. README.md +72 -73
  2. config.json +4 -2
  3. model.safetensors +3 -0
  4. tokenizer.json +6 -1
  5. tokenizer_config.json +42 -0
  6. training_args.bin +2 -2
README.md CHANGED
@@ -13,69 +13,69 @@ should probably proofread and complete it, then remove this comment. -->
13
 
14
  # rubert-electra-srl
15
 
16
- This model is a fine-tuned version of [ai-forever/ruElectra-medium](https://huggingface.co/ai-forever/ruElectra-medium) on an unknown dataset.
17
  It achieves the following results on the evaluation set:
18
- - Loss: 0.1471
19
- - Addressee Precision: 0.9583
20
- - Addressee Recall: 0.9020
21
- - Addressee F1: 0.9293
22
- - Addressee Number: 51
23
- - Benefactive Precision: 0.8
24
- - Benefactive Recall: 0.25
25
- - Benefactive F1: 0.3810
26
- - Benefactive Number: 16
27
- - Causator Precision: 0.8971
28
- - Causator Recall: 0.8714
29
- - Causator F1: 0.8841
30
- - Causator Number: 70
31
- - Cause Precision: 0.6466
32
- - Cause Recall: 0.7353
33
- - Cause F1: 0.6881
34
- - Cause Number: 102
35
- - Contrsubject Precision: 0.832
36
- - Contrsubject Recall: 0.7879
37
- - Contrsubject F1: 0.8093
38
- - Contrsubject Number: 132
39
- - Deliberative Precision: 0.6269
40
- - Deliberative Recall: 0.84
41
- - Deliberative F1: 0.7179
42
- - Deliberative Number: 50
43
  - Destinative Precision: 1.0
44
- - Destinative Recall: 0.3871
45
- - Destinative F1: 0.5581
46
- - Destinative Number: 31
47
- - Directivefinal Precision: 0.5455
48
- - Directivefinal Recall: 0.6667
49
- - Directivefinal F1: 0.6
50
- - Directivefinal Number: 9
51
- - Experiencer Precision: 0.8669
52
- - Experiencer Recall: 0.8609
53
- - Experiencer F1: 0.8639
54
- - Experiencer Number: 726
55
- - Instrument Precision: 0.5
56
- - Instrument Recall: 0.3333
57
- - Instrument F1: 0.4
58
- - Instrument Number: 9
59
  - Limitative Precision: 0.0
60
  - Limitative Recall: 0.0
61
  - Limitative F1: 0.0
62
- - Limitative Number: 4
63
- - Object Precision: 0.8676
64
- - Object Recall: 0.8703
65
- - Object F1: 0.8689
66
- - Object Number: 1611
67
- - Overall Precision: 0.8515
68
- - Overall Recall: 0.8467
69
- - Overall F1: 0.8491
70
- - Overall Accuracy: 0.9687
71
- - Directiveinitial Recall: 0.0
72
- - Directiveinitial Number: 0.0
73
- - Directiveinitial Precision: 0.0
74
- - Directiveinitial F1: 0.0
75
- - Mediative Recall: 0.0
76
  - Mediative Number: 0.0
77
- - Mediative Precision: 0.0
78
  - Mediative F1: 0.0
 
 
 
 
 
 
79
 
80
  ## Model description
81
 
@@ -94,31 +94,30 @@ More information needed
94
  ### Training hyperparameters
95
 
96
  The following hyperparameters were used during training:
97
- - learning_rate: 0.000261433658985083
98
  - train_batch_size: 1
99
  - eval_batch_size: 1
100
- - seed: 510754
101
- - gradient_accumulation_steps: 8
102
- - total_train_batch_size: 8
103
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
104
  - lr_scheduler_type: linear
105
- - lr_scheduler_warmup_ratio: 0.3
106
- - num_epochs: 5
 
107
 
108
  ### Training results
109
 
110
- | Training Loss | Epoch | Step | Validation Loss | Addressee Precision | Addressee Recall | Addressee F1 | Addressee Number | Benefactive Precision | Benefactive Recall | Benefactive F1 | Benefactive Number | Causator Precision | Causator Recall | Causator F1 | Causator Number | Cause Precision | Cause Recall | Cause F1 | Cause Number | Contrsubject Precision | Contrsubject Recall | Contrsubject F1 | Contrsubject Number | Deliberative Precision | Deliberative Recall | Deliberative F1 | Deliberative Number | Destinative Precision | Destinative Recall | Destinative F1 | Destinative Number | Directivefinal Precision | Directivefinal Recall | Directivefinal F1 | Directivefinal Number | Experiencer Precision | Experiencer Recall | Experiencer F1 | Experiencer Number | Instrument Precision | Instrument Recall | Instrument F1 | Instrument Number | Limitative Precision | Limitative Recall | Limitative F1 | Limitative Number | Object Precision | Object Recall | Object F1 | Object Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | Directiveinitial Recall | Directiveinitial Number | Directiveinitial Precision | Directiveinitial F1 | Mediative Recall | Mediative Number | Mediative Precision | Mediative F1 |
111
- |:-------------:|:-----:|:----:|:---------------:|:-------------------:|:----------------:|:------------:|:----------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------------:|:---------------------:|:-----------------:|:---------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:----------------:|:-------------:|:---------:|:-------------:|:-----------------:|:--------------:|:----------:|:----------------:|:-----------------------:|:-----------------------:|:--------------------------:|:-------------------:|:----------------:|:----------------:|:-------------------:|:------------:|
112
- | 0.2154 | 1.0 | 763 | 0.2074 | 0.6842 | 0.5098 | 0.5843 | 51 | 0.0 | 0.0 | 0.0 | 16 | 0.1946 | 0.8286 | 0.3152 | 70 | 1.0 | 0.0098 | 0.0194 | 102 | 0.2 | 0.0076 | 0.0146 | 132 | 0.0 | 0.0 | 0.0 | 50 | 0.0 | 0.0 | 0.0 | 31 | 0.0 | 0.0 | 0.0 | 9 | 0.6747 | 0.7713 | 0.7198 | 726 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 4 | 0.8199 | 0.7263 | 0.7702 | 1611 | 0.6987 | 0.6460 | 0.6713 | 0.9433 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
113
- | 0.2294 | 2.0 | 1526 | 0.2028 | 0.7460 | 0.9216 | 0.8246 | 51 | 0.0 | 0.0 | 0.0 | 16 | 0.0 | 0.0 | 0.0 | 70 | 0.3333 | 0.0098 | 0.0190 | 102 | 0.7791 | 0.5076 | 0.6147 | 132 | 0.22 | 0.88 | 0.352 | 50 | 0.0 | 0.0 | 0.0 | 31 | 0.6667 | 0.6667 | 0.6667 | 9 | 0.8822 | 0.6708 | 0.7621 | 726 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 4 | 0.7332 | 0.7914 | 0.7612 | 1611 | 0.7255 | 0.6855 | 0.7050 | 0.9417 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
114
- | 0.132 | 3.0 | 2290 | 0.1485 | 0.7188 | 0.9020 | 0.8 | 51 | 0.0 | 0.0 | 0.0 | 16 | 0.6854 | 0.8714 | 0.7673 | 70 | 0.4079 | 0.3039 | 0.3483 | 102 | 0.6562 | 0.7955 | 0.7192 | 132 | 0.5263 | 0.4 | 0.4545 | 50 | 0.0 | 0.0 | 0.0 | 31 | 0.6 | 0.6667 | 0.6316 | 9 | 0.8289 | 0.8609 | 0.8446 | 726 | 0.0 | 0.0 | 0.0 | 9 | 0.0 | 0.0 | 0.0 | 4 | 0.8013 | 0.8610 | 0.8300 | 1611 | 0.7806 | 0.8115 | 0.7957 | 0.9574 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
115
- | 0.0748 | 4.0 | 3053 | 0.1382 | 0.9038 | 0.9216 | 0.9126 | 51 | 0.1905 | 0.25 | 0.2162 | 16 | 0.9104 | 0.8714 | 0.8905 | 70 | 0.5859 | 0.7353 | 0.6522 | 102 | 0.825 | 0.75 | 0.7857 | 132 | 0.4875 | 0.78 | 0.6 | 50 | 0.0 | 0.0 | 0.0 | 31 | 0.4615 | 0.6667 | 0.5455 | 9 | 0.9033 | 0.8237 | 0.8617 | 726 | 0.4 | 0.2222 | 0.2857 | 9 | 0.0 | 0.0 | 0.0 | 4 | 0.8468 | 0.8678 | 0.8571 | 1611 | 0.8321 | 0.8285 | 0.8303 | 0.9659 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
116
- | 0.0504 | 5.0 | 3815 | 0.1471 | 0.9583 | 0.9020 | 0.9293 | 51 | 0.8 | 0.25 | 0.3810 | 16 | 0.8971 | 0.8714 | 0.8841 | 70 | 0.6466 | 0.7353 | 0.6881 | 102 | 0.832 | 0.7879 | 0.8093 | 132 | 0.6269 | 0.84 | 0.7179 | 50 | 1.0 | 0.3871 | 0.5581 | 31 | 0.5455 | 0.6667 | 0.6 | 9 | 0.8669 | 0.8609 | 0.8639 | 726 | 0.5 | 0.3333 | 0.4 | 9 | 0.0 | 0.0 | 0.0 | 4 | 0.8676 | 0.8703 | 0.8689 | 1611 | 0.8515 | 0.8467 | 0.8491 | 0.9687 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
117
 
118
 
119
  ### Framework versions
120
 
121
- - Transformers 4.33.2
122
- - Pytorch 2.0.1+cu117
123
- - Datasets 2.14.5
124
- - Tokenizers 0.13.3
 
13
 
14
  # rubert-electra-srl
15
 
16
+ This model is a fine-tuned version of [ai-forever/ruElectra-medium](https://huggingface.co/ai-forever/ruElectra-medium) on the None dataset.
17
  It achieves the following results on the evaluation set:
18
+ - Loss: 0.0564
19
+ - Addressee Precision: 0.8710
20
+ - Addressee Recall: 0.9153
21
+ - Addressee F1: 0.8926
22
+ - Addressee Number: 59
23
+ - Benefactive Precision: 0.0
24
+ - Benefactive Recall: 0.0
25
+ - Benefactive F1: 0.0
26
+ - Benefactive Number: 8
27
+ - Causator Precision: 0.9007
28
+ - Causator Recall: 0.9379
29
+ - Causator F1: 0.9189
30
+ - Causator Number: 145
31
+ - Cause Precision: 0.8491
32
+ - Cause Recall: 0.7895
33
+ - Cause F1: 0.8182
34
+ - Cause Number: 114
35
+ - Contrsubject Precision: 0.872
36
+ - Contrsubject Recall: 0.9008
37
+ - Contrsubject F1: 0.8862
38
+ - Contrsubject Number: 121
39
+ - Deliberative Precision: 0.7439
40
+ - Deliberative Recall: 0.9385
41
+ - Deliberative F1: 0.8299
42
+ - Deliberative Number: 65
43
  - Destinative Precision: 1.0
44
+ - Destinative Recall: 0.5238
45
+ - Destinative F1: 0.6875
46
+ - Destinative Number: 21
47
+ - Directivefinal Precision: 1.0
48
+ - Directivefinal Recall: 0.7
49
+ - Directivefinal F1: 0.8235
50
+ - Directivefinal Number: 10
51
+ - Experiencer Precision: 0.9132
52
+ - Experiencer Recall: 0.9374
53
+ - Experiencer F1: 0.9252
54
+ - Experiencer Number: 1055
55
+ - Instrument Precision: 0.8409
56
+ - Instrument Recall: 0.7255
57
+ - Instrument F1: 0.7789
58
+ - Instrument Number: 51
59
  - Limitative Precision: 0.0
60
  - Limitative Recall: 0.0
61
  - Limitative F1: 0.0
62
+ - Limitative Number: 3
63
+ - Object Precision: 0.9449
64
+ - Object Recall: 0.9389
65
+ - Object F1: 0.9419
66
+ - Object Number: 1898
67
+ - Overall Precision: 0.9210
68
+ - Overall Recall: 0.9228
69
+ - Overall F1: 0.9219
70
+ - Overall Accuracy: 0.9855
 
 
 
 
 
71
  - Mediative Number: 0.0
 
72
  - Mediative F1: 0.0
73
+ - Mediative Precision: 0.0
74
+ - Mediative Recall: 0.0
75
+ - Directiveinitial Number: 0.0
76
+ - Directiveinitial F1: 0.0
77
+ - Directiveinitial Precision: 0.0
78
+ - Directiveinitial Recall: 0.0
79
 
80
  ## Model description
81
 
 
94
  ### Training hyperparameters
95
 
96
  The following hyperparameters were used during training:
97
+ - learning_rate: 0.00016666401556632117
98
  - train_batch_size: 1
99
  - eval_batch_size: 1
100
+ - seed: 708526
101
+ - gradient_accumulation_steps: 4
102
+ - total_train_batch_size: 4
103
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
104
  - lr_scheduler_type: linear
105
+ - lr_scheduler_warmup_ratio: 0.21
106
+ - num_epochs: 3
107
+ - mixed_precision_training: Native AMP
108
 
109
  ### Training results
110
 
111
+ | Training Loss | Epoch | Step | Validation Loss | Addressee Precision | Addressee Recall | Addressee F1 | Addressee Number | Benefactive Precision | Benefactive Recall | Benefactive F1 | Benefactive Number | Causator Precision | Causator Recall | Causator F1 | Causator Number | Cause Precision | Cause Recall | Cause F1 | Cause Number | Contrsubject Precision | Contrsubject Recall | Contrsubject F1 | Contrsubject Number | Deliberative Precision | Deliberative Recall | Deliberative F1 | Deliberative Number | Destinative Precision | Destinative Recall | Destinative F1 | Destinative Number | Directivefinal Precision | Directivefinal Recall | Directivefinal F1 | Directivefinal Number | Experiencer Precision | Experiencer Recall | Experiencer F1 | Experiencer Number | Instrument Precision | Instrument Recall | Instrument F1 | Instrument Number | Limitative Precision | Limitative Recall | Limitative F1 | Limitative Number | Object Precision | Object Recall | Object F1 | Object Number | Overall Precision | Overall Recall | Overall F1 | Overall Accuracy | Mediative Number | Mediative F1 | Mediative Precision | Mediative Recall | Directiveinitial Number | Directiveinitial F1 | Directiveinitial Precision | Directiveinitial Recall |
112
+ |:-------------:|:-----:|:----:|:---------------:|:-------------------:|:----------------:|:------------:|:----------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------:|:---------------:|:-----------:|:---------------:|:---------------:|:------------:|:--------:|:------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:----------------------:|:-------------------:|:---------------:|:-------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:------------------------:|:---------------------:|:-----------------:|:---------------------:|:---------------------:|:------------------:|:--------------:|:------------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:--------------------:|:-----------------:|:-------------:|:-----------------:|:----------------:|:-------------:|:---------:|:-------------:|:-----------------:|:--------------:|:----------:|:----------------:|:----------------:|:------------:|:-------------------:|:----------------:|:-----------------------:|:-------------------:|:--------------------------:|:-----------------------:|
113
+ | 0.574 | 1.0 | 2942 | 0.5853 | 0.0 | 0.0 | 0.0 | 59 | 0.0 | 0.0 | 0.0 | 8 | 0.0 | 0.0 | 0.0 | 145 | 0.0 | 0.0 | 0.0 | 114 | 0.0 | 0.0 | 0.0 | 121 | 0.0 | 0.0 | 0.0 | 65 | 0.0 | 0.0 | 0.0 | 21 | 0.0 | 0.0 | 0.0 | 10 | 0.0 | 0.0 | 0.0 | 1055 | 0.0 | 0.0 | 0.0 | 51 | 0.0 | 0.0 | 0.0 | 3 | 0.0 | 0.0 | 0.0 | 1898 | 0.0 | 0.0 | 0.0 | 0.8893 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
114
+ | 0.1625 | 2.0 | 5884 | 0.1573 | 0.5714 | 0.8136 | 0.6713 | 59 | 0.0 | 0.0 | 0.0 | 8 | 0.6966 | 0.8552 | 0.7678 | 145 | 0.3186 | 0.6316 | 0.4235 | 114 | 0.6875 | 0.4545 | 0.5473 | 121 | 0.0 | 0.0 | 0.0 | 65 | 0.0 | 0.0 | 0.0 | 21 | 0.0 | 0.0 | 0.0 | 10 | 0.8504 | 0.8246 | 0.8373 | 1055 | 0.4769 | 0.6078 | 0.5345 | 51 | 0.0 | 0.0 | 0.0 | 3 | 0.8923 | 0.8161 | 0.8525 | 1898 | 0.8104 | 0.7744 | 0.7920 | 0.9634 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
115
+ | 0.0838 | 3.0 | 8826 | 0.0564 | 0.8710 | 0.9153 | 0.8926 | 59 | 0.0 | 0.0 | 0.0 | 8 | 0.9007 | 0.9379 | 0.9189 | 145 | 0.8491 | 0.7895 | 0.8182 | 114 | 0.872 | 0.9008 | 0.8862 | 121 | 0.7439 | 0.9385 | 0.8299 | 65 | 1.0 | 0.5238 | 0.6875 | 21 | 1.0 | 0.7 | 0.8235 | 10 | 0.9132 | 0.9374 | 0.9252 | 1055 | 0.8409 | 0.7255 | 0.7789 | 51 | 0.0 | 0.0 | 0.0 | 3 | 0.9449 | 0.9389 | 0.9419 | 1898 | 0.9210 | 0.9228 | 0.9219 | 0.9855 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
 
 
116
 
117
 
118
  ### Framework versions
119
 
120
+ - Transformers 4.42.4
121
+ - Pytorch 2.3.1+cu121
122
+ - Datasets 2.20.0
123
+ - Tokenizers 0.19.1
config.json CHANGED
@@ -32,7 +32,8 @@
32
  "18": "B-DirectiveInitial",
33
  "19": "I-DirectiveInitial",
34
  "20": "I-Experiencer",
35
- "21": "I-Cause"
 
36
  },
37
  "initializer_range": 0.02,
38
  "intermediate_size": 2304,
@@ -51,6 +52,7 @@
51
  "B-Limitative": 14,
52
  "B-Mediative": 16,
53
  "B-Object": 1,
 
54
  "I-Cause": 21,
55
  "I-ContrSubject": 11,
56
  "I-Deliberative": 13,
@@ -72,7 +74,7 @@
72
  "summary_type": "first",
73
  "summary_use_proj": true,
74
  "torch_dtype": "float32",
75
- "transformers_version": "4.33.2",
76
  "type_vocab_size": 2,
77
  "use_cache": true,
78
  "vocab_size": 64000
 
32
  "18": "B-DirectiveInitial",
33
  "19": "I-DirectiveInitial",
34
  "20": "I-Experiencer",
35
+ "21": "I-Cause",
36
+ "22": "I-Causator"
37
  },
38
  "initializer_range": 0.02,
39
  "intermediate_size": 2304,
 
52
  "B-Limitative": 14,
53
  "B-Mediative": 16,
54
  "B-Object": 1,
55
+ "I-Causator": 22,
56
  "I-Cause": 21,
57
  "I-ContrSubject": 11,
58
  "I-Deliberative": 13,
 
74
  "summary_type": "first",
75
  "summary_use_proj": true,
76
  "torch_dtype": "float32",
77
+ "transformers_version": "4.42.4",
78
  "type_vocab_size": 2,
79
  "use_cache": true,
80
  "vocab_size": 64000
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8dc983e23ebf1a46a93b9ffb6bb8dfcc4f96b632c2282ab78edd817e53106b5c
3
+ size 340184276
tokenizer.json CHANGED
@@ -1,6 +1,11 @@
1
  {
2
  "version": "1.0",
3
- "truncation": null,
 
 
 
 
 
4
  "padding": null,
5
  "added_tokens": [
6
  {
 
1
  {
2
  "version": "1.0",
3
+ "truncation": {
4
+ "direction": "Right",
5
+ "max_length": 2048,
6
+ "strategy": "LongestFirst",
7
+ "stride": 0
8
+ },
9
  "padding": null,
10
  "added_tokens": [
11
  {
tokenizer_config.json CHANGED
@@ -1,4 +1,46 @@
1
  {
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
2
  "clean_up_tokenization_spaces": true,
3
  "cls_token": "[CLS]",
4
  "do_basic_tokenize": true,
 
1
  {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "4": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
  "clean_up_tokenization_spaces": true,
45
  "cls_token": "[CLS]",
46
  "do_basic_tokenize": true,
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:d6b76fd56499805942ca588f2f290c4c0a3e7c80b80ef2c2b659e065090c0acb
3
- size 4091
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:6e293be6da178e0dad7a21799fcc806409ec27abe94b264bcf829fab4f996651
3
+ size 5240