AngryBacteria commited on
Commit
eac5e5a
1 Parent(s): c5a0278

End of training

Browse files
README.md ADDED
@@ -0,0 +1,145 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: apache-2.0
3
+ base_model: GerMedBERT/medbert-512
4
+ tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - precision
8
+ - recall
9
+ - accuracy
10
+ model-index:
11
+ - name: GerMedBert_NORMTOP50_V02_BRONCO
12
+ results: []
13
+ ---
14
+
15
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
16
+ should probably proofread and complete it, then remove this comment. -->
17
+
18
+ # GerMedBert_NORMTOP50_V02_BRONCO
19
+
20
+ This model is a fine-tuned version of [GerMedBERT/medbert-512](https://huggingface.co/GerMedBERT/medbert-512) on the None dataset.
21
+ It achieves the following results on the evaluation set:
22
+ - Loss: 0.0166
23
+ - F1 Score: 0.8624
24
+ - Precision: 0.8939
25
+ - Recall: 0.8329
26
+ - Accuracy: 0.8594
27
+ - Num Input Tokens Seen: 15165836
28
+
29
+ ## Model description
30
+
31
+ More information needed
32
+
33
+ ## Intended uses & limitations
34
+
35
+ More information needed
36
+
37
+ ## Training and evaluation data
38
+
39
+ More information needed
40
+
41
+ ## Training procedure
42
+
43
+ ### Training hyperparameters
44
+
45
+ The following hyperparameters were used during training:
46
+ - learning_rate: 2e-05
47
+ - train_batch_size: 16
48
+ - eval_batch_size: 16
49
+ - seed: 42
50
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
51
+ - lr_scheduler_type: linear
52
+ - num_epochs: 20
53
+
54
+ ### Training results
55
+
56
+ | Training Loss | Epoch | Step | Validation Loss | F1 Score | Precision | Recall | Accuracy | Input Tokens Seen |
57
+ |:-------------:|:-----:|:----:|:---------------:|:--------:|:---------:|:------:|:--------:|:-----------------:|
58
+ | No log | 0.25 | 81 | 0.1649 | 0.0 | 1.0 | 0.0 | 0.5920 | 190512 |
59
+ | 0.2195 | 0.5 | 162 | 0.0988 | 0.0 | 1.0 | 0.0 | 0.5920 | 380384 |
60
+ | 0.2195 | 0.75 | 243 | 0.0825 | 0.0 | 1.0 | 0.0 | 0.5920 | 570256 |
61
+ | 0.0855 | 1.0 | 324 | 0.0768 | 0.0 | 1.0 | 0.0 | 0.5920 | 759687 |
62
+ | 0.0855 | 1.25 | 405 | 0.0744 | 0.0 | 1.0 | 0.0 | 0.5920 | 948599 |
63
+ | 0.0737 | 1.5 | 486 | 0.0688 | 0.0 | 1.0 | 0.0 | 0.5920 | 1138791 |
64
+ | 0.0737 | 1.75 | 567 | 0.0652 | 0.1979 | 0.94 | 0.1106 | 0.6059 | 1327703 |
65
+ | 0.0666 | 2.0 | 648 | 0.0600 | 0.0982 | 0.9565 | 0.0518 | 0.5972 | 1516814 |
66
+ | 0.0666 | 2.25 | 729 | 0.0553 | 0.1231 | 0.9333 | 0.0659 | 0.5972 | 1706686 |
67
+ | 0.0572 | 2.5 | 810 | 0.0521 | 0.4093 | 0.8394 | 0.2706 | 0.6458 | 1896878 |
68
+ | 0.0572 | 2.75 | 891 | 0.0490 | 0.4326 | 0.8777 | 0.2871 | 0.6510 | 2086110 |
69
+ | 0.0499 | 3.0 | 972 | 0.0456 | 0.2897 | 1.0 | 0.1694 | 0.6476 | 2275221 |
70
+ | 0.0499 | 3.25 | 1053 | 0.0423 | 0.5060 | 0.925 | 0.3482 | 0.6667 | 2465413 |
71
+ | 0.0441 | 3.5 | 1134 | 0.0406 | 0.5386 | 0.8913 | 0.3859 | 0.6736 | 2654965 |
72
+ | 0.0441 | 3.75 | 1215 | 0.0396 | 0.5979 | 0.8419 | 0.4635 | 0.6858 | 2845157 |
73
+ | 0.0393 | 4.0 | 1296 | 0.0377 | 0.6517 | 0.9004 | 0.5106 | 0.7083 | 3034556 |
74
+ | 0.0393 | 4.25 | 1377 | 0.0357 | 0.6319 | 0.9075 | 0.4847 | 0.6997 | 3224428 |
75
+ | 0.0361 | 4.5 | 1458 | 0.0346 | 0.6154 | 0.9245 | 0.4612 | 0.7066 | 3414620 |
76
+ | 0.0361 | 4.75 | 1539 | 0.0334 | 0.6258 | 0.8987 | 0.48 | 0.7101 | 3604172 |
77
+ | 0.032 | 5.0 | 1620 | 0.0321 | 0.6775 | 0.9124 | 0.5388 | 0.7292 | 3793603 |
78
+ | 0.032 | 5.25 | 1701 | 0.0306 | 0.7081 | 0.9176 | 0.5765 | 0.7378 | 3983155 |
79
+ | 0.0293 | 5.5 | 1782 | 0.0302 | 0.6928 | 0.9019 | 0.5624 | 0.7361 | 4172387 |
80
+ | 0.0293 | 5.75 | 1863 | 0.0292 | 0.6657 | 0.9247 | 0.52 | 0.7240 | 4362579 |
81
+ | 0.0273 | 6.0 | 1944 | 0.0287 | 0.7365 | 0.9253 | 0.6118 | 0.7691 | 4552330 |
82
+ | 0.0273 | 6.25 | 2025 | 0.0275 | 0.7215 | 0.9328 | 0.5882 | 0.7552 | 4741882 |
83
+ | 0.0258 | 6.5 | 2106 | 0.0272 | 0.7275 | 0.9024 | 0.6094 | 0.7517 | 4930794 |
84
+ | 0.0258 | 6.75 | 2187 | 0.0260 | 0.7451 | 0.9204 | 0.6259 | 0.7726 | 5120026 |
85
+ | 0.0228 | 7.0 | 2268 | 0.0260 | 0.7247 | 0.9203 | 0.5976 | 0.7656 | 5309137 |
86
+ | 0.0228 | 7.25 | 2349 | 0.0249 | 0.7867 | 0.9077 | 0.6941 | 0.7969 | 5499649 |
87
+ | 0.0218 | 7.5 | 2430 | 0.0246 | 0.7572 | 0.9079 | 0.6494 | 0.7778 | 5688881 |
88
+ | 0.0218 | 7.75 | 2511 | 0.0239 | 0.7779 | 0.9088 | 0.68 | 0.7882 | 5878113 |
89
+ | 0.02 | 8.0 | 2592 | 0.0239 | 0.7835 | 0.8994 | 0.6941 | 0.7899 | 6067224 |
90
+ | 0.02 | 8.25 | 2673 | 0.0229 | 0.7711 | 0.9159 | 0.6659 | 0.7917 | 6256456 |
91
+ | 0.0184 | 8.5 | 2754 | 0.0227 | 0.7705 | 0.8969 | 0.6753 | 0.7917 | 6446328 |
92
+ | 0.0184 | 8.75 | 2835 | 0.0226 | 0.7782 | 0.8671 | 0.7059 | 0.7899 | 6636520 |
93
+ | 0.0182 | 9.0 | 2916 | 0.0224 | 0.7937 | 0.8988 | 0.7106 | 0.8003 | 6825951 |
94
+ | 0.0182 | 9.25 | 2997 | 0.0217 | 0.7815 | 0.8939 | 0.6941 | 0.7951 | 7015183 |
95
+ | 0.0172 | 9.5 | 3078 | 0.0213 | 0.8156 | 0.9101 | 0.7388 | 0.8212 | 7205375 |
96
+ | 0.0172 | 9.75 | 3159 | 0.0211 | 0.8063 | 0.9086 | 0.7247 | 0.8142 | 7394927 |
97
+ | 0.0154 | 10.0 | 3240 | 0.0216 | 0.8246 | 0.8820 | 0.7741 | 0.8212 | 7583366 |
98
+ | 0.0154 | 10.25 | 3321 | 0.0204 | 0.7831 | 0.8943 | 0.6965 | 0.8021 | 7772598 |
99
+ | 0.0145 | 10.5 | 3402 | 0.0201 | 0.8185 | 0.9034 | 0.7482 | 0.8229 | 7962470 |
100
+ | 0.0145 | 10.75 | 3483 | 0.0200 | 0.8261 | 0.9048 | 0.76 | 0.8264 | 8152662 |
101
+ | 0.0143 | 11.0 | 3564 | 0.0198 | 0.8238 | 0.8929 | 0.7647 | 0.8281 | 8341773 |
102
+ | 0.0143 | 11.25 | 3645 | 0.0196 | 0.8229 | 0.8972 | 0.76 | 0.8264 | 8531645 |
103
+ | 0.0131 | 11.5 | 3726 | 0.0193 | 0.8231 | 0.8817 | 0.7718 | 0.8212 | 8720877 |
104
+ | 0.0131 | 11.75 | 3807 | 0.0195 | 0.8152 | 0.8822 | 0.7576 | 0.8177 | 8910109 |
105
+ | 0.0129 | 12.0 | 3888 | 0.0192 | 0.8263 | 0.9119 | 0.7553 | 0.8299 | 9099860 |
106
+ | 0.0129 | 12.25 | 3969 | 0.0188 | 0.8229 | 0.8972 | 0.76 | 0.8212 | 9289412 |
107
+ | 0.0116 | 12.5 | 4050 | 0.0191 | 0.8123 | 0.8883 | 0.7482 | 0.8247 | 9479284 |
108
+ | 0.0116 | 12.75 | 4131 | 0.0181 | 0.8417 | 0.9030 | 0.7882 | 0.8472 | 9669156 |
109
+ | 0.0115 | 13.0 | 4212 | 0.0180 | 0.8398 | 0.8895 | 0.7953 | 0.8420 | 9857947 |
110
+ | 0.0115 | 13.25 | 4293 | 0.0177 | 0.8445 | 0.9126 | 0.7859 | 0.8455 | 10045899 |
111
+ | 0.0108 | 13.5 | 4374 | 0.0179 | 0.8426 | 0.8901 | 0.8 | 0.8438 | 10236091 |
112
+ | 0.0108 | 13.75 | 4455 | 0.0179 | 0.8519 | 0.8961 | 0.8118 | 0.8524 | 10426283 |
113
+ | 0.0103 | 14.0 | 4536 | 0.0177 | 0.8392 | 0.9003 | 0.7859 | 0.8420 | 10615394 |
114
+ | 0.0103 | 14.25 | 4617 | 0.0176 | 0.8603 | 0.9062 | 0.8188 | 0.8594 | 10805906 |
115
+ | 0.0097 | 14.5 | 4698 | 0.0173 | 0.8475 | 0.904 | 0.7976 | 0.8507 | 10995458 |
116
+ | 0.0097 | 14.75 | 4779 | 0.0175 | 0.8511 | 0.9003 | 0.8071 | 0.8524 | 11185650 |
117
+ | 0.0095 | 15.0 | 4860 | 0.0173 | 0.8501 | 0.8979 | 0.8071 | 0.8490 | 11375081 |
118
+ | 0.0095 | 15.25 | 4941 | 0.0175 | 0.8451 | 0.8927 | 0.8024 | 0.8472 | 11564633 |
119
+ | 0.009 | 15.5 | 5022 | 0.0174 | 0.8483 | 0.8912 | 0.8094 | 0.8490 | 11754185 |
120
+ | 0.009 | 15.75 | 5103 | 0.0172 | 0.8490 | 0.8956 | 0.8071 | 0.8438 | 11943417 |
121
+ | 0.009 | 16.0 | 5184 | 0.0174 | 0.8547 | 0.8883 | 0.8235 | 0.8490 | 12132528 |
122
+ | 0.009 | 16.25 | 5265 | 0.0171 | 0.8519 | 0.8961 | 0.8118 | 0.8490 | 12322400 |
123
+ | 0.0085 | 16.5 | 5346 | 0.0169 | 0.8537 | 0.8861 | 0.8235 | 0.8524 | 12511632 |
124
+ | 0.0085 | 16.75 | 5427 | 0.0169 | 0.8498 | 0.8915 | 0.8118 | 0.8524 | 12701184 |
125
+ | 0.0084 | 17.0 | 5508 | 0.0169 | 0.8487 | 0.8892 | 0.8118 | 0.8507 | 12889655 |
126
+ | 0.0084 | 17.25 | 5589 | 0.0167 | 0.8634 | 0.8962 | 0.8329 | 0.8524 | 13079527 |
127
+ | 0.0079 | 17.5 | 5670 | 0.0168 | 0.8504 | 0.8958 | 0.8094 | 0.8472 | 13269399 |
128
+ | 0.0079 | 17.75 | 5751 | 0.0168 | 0.8606 | 0.8957 | 0.8282 | 0.8594 | 13458631 |
129
+ | 0.0081 | 18.0 | 5832 | 0.0167 | 0.8564 | 0.8949 | 0.8212 | 0.8542 | 13648382 |
130
+ | 0.0081 | 18.25 | 5913 | 0.0166 | 0.8620 | 0.8959 | 0.8306 | 0.8576 | 13838254 |
131
+ | 0.0078 | 18.5 | 5994 | 0.0166 | 0.8533 | 0.8964 | 0.8141 | 0.8542 | 14028126 |
132
+ | 0.0078 | 18.75 | 6075 | 0.0167 | 0.8610 | 0.8937 | 0.8306 | 0.8611 | 14217678 |
133
+ | 0.0078 | 19.0 | 6156 | 0.0166 | 0.8610 | 0.8937 | 0.8306 | 0.8594 | 14407109 |
134
+ | 0.0078 | 19.25 | 6237 | 0.0166 | 0.8637 | 0.8942 | 0.8353 | 0.8594 | 14596341 |
135
+ | 0.0072 | 19.5 | 6318 | 0.0166 | 0.8620 | 0.8959 | 0.8306 | 0.8594 | 14786213 |
136
+ | 0.0072 | 19.75 | 6399 | 0.0166 | 0.8624 | 0.8939 | 0.8329 | 0.8594 | 14976085 |
137
+ | 0.008 | 20.0 | 6480 | 0.0166 | 0.8624 | 0.8939 | 0.8329 | 0.8594 | 15165836 |
138
+
139
+
140
+ ### Framework versions
141
+
142
+ - Transformers 4.40.1
143
+ - Pytorch 2.3.0+cu121
144
+ - Datasets 2.19.0
145
+ - Tokenizers 0.19.1
config.json ADDED
@@ -0,0 +1,130 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "_name_or_path": "GerMedBERT/medbert-512",
3
+ "architectures": [
4
+ "BertForSequenceClassification"
5
+ ],
6
+ "attention_probs_dropout_prob": 0.1,
7
+ "classifier_dropout": null,
8
+ "hidden_act": "gelu",
9
+ "hidden_dropout_prob": 0.1,
10
+ "hidden_size": 768,
11
+ "id2label": {
12
+ "0": "1-442.0",
13
+ "1": "1-632.y",
14
+ "2": "3-03",
15
+ "3": "3-030",
16
+ "4": "3-20",
17
+ "5": "3-202",
18
+ "6": "3-206",
19
+ "7": "3-207",
20
+ "8": "3-80",
21
+ "9": "3-804",
22
+ "10": "5-501.5x",
23
+ "11": "5-501.y",
24
+ "12": "5-504.y",
25
+ "13": "6-001.1",
26
+ "14": "6-003.b",
27
+ "15": "6-008.m",
28
+ "16": "8-52",
29
+ "17": "8-52a",
30
+ "18": "8-530.a",
31
+ "19": "8-54",
32
+ "20": "8-541.6",
33
+ "21": "8-542",
34
+ "22": "8-547",
35
+ "23": "A02BC02",
36
+ "24": "B18.2",
37
+ "25": "C22.0",
38
+ "26": "C43.9",
39
+ "27": "C69.3",
40
+ "28": "C77.2",
41
+ "29": "C77.9",
42
+ "30": "C78.0",
43
+ "31": "C78.6",
44
+ "32": "C78.7",
45
+ "33": "C79.5",
46
+ "34": "C79.88",
47
+ "35": "C79.9",
48
+ "36": "D37.6",
49
+ "37": "D48.9",
50
+ "38": "I81",
51
+ "39": "I85.9",
52
+ "40": "K70.3",
53
+ "41": "K74.6",
54
+ "42": "L01AB02",
55
+ "43": "L01BC05",
56
+ "44": "L01XA01",
57
+ "45": "L01XC17",
58
+ "46": "L01XE05",
59
+ "47": "R18",
60
+ "48": "R52.9",
61
+ "49": "R53"
62
+ },
63
+ "initializer_range": 0.02,
64
+ "intermediate_size": 3072,
65
+ "label2id": {
66
+ "1-442.0": 0,
67
+ "1-632.y": 1,
68
+ "3-03": 2,
69
+ "3-030": 3,
70
+ "3-20": 4,
71
+ "3-202": 5,
72
+ "3-206": 6,
73
+ "3-207": 7,
74
+ "3-80": 8,
75
+ "3-804": 9,
76
+ "5-501.5x": 10,
77
+ "5-501.y": 11,
78
+ "5-504.y": 12,
79
+ "6-001.1": 13,
80
+ "6-003.b": 14,
81
+ "6-008.m": 15,
82
+ "8-52": 16,
83
+ "8-52a": 17,
84
+ "8-530.a": 18,
85
+ "8-54": 19,
86
+ "8-541.6": 20,
87
+ "8-542": 21,
88
+ "8-547": 22,
89
+ "A02BC02": 23,
90
+ "B18.2": 24,
91
+ "C22.0": 25,
92
+ "C43.9": 26,
93
+ "C69.3": 27,
94
+ "C77.2": 28,
95
+ "C77.9": 29,
96
+ "C78.0": 30,
97
+ "C78.6": 31,
98
+ "C78.7": 32,
99
+ "C79.5": 33,
100
+ "C79.88": 34,
101
+ "C79.9": 35,
102
+ "D37.6": 36,
103
+ "D48.9": 37,
104
+ "I81": 38,
105
+ "I85.9": 39,
106
+ "K70.3": 40,
107
+ "K74.6": 41,
108
+ "L01AB02": 42,
109
+ "L01BC05": 43,
110
+ "L01XA01": 44,
111
+ "L01XC17": 45,
112
+ "L01XE05": 46,
113
+ "R18": 47,
114
+ "R52.9": 48,
115
+ "R53": 49
116
+ },
117
+ "layer_norm_eps": 1e-12,
118
+ "max_position_embeddings": 512,
119
+ "model_type": "bert",
120
+ "num_attention_heads": 8,
121
+ "num_hidden_layers": 12,
122
+ "pad_token_id": 0,
123
+ "position_embedding_type": "absolute",
124
+ "problem_type": "multi_label_classification",
125
+ "torch_dtype": "float32",
126
+ "transformers_version": "4.40.1",
127
+ "type_vocab_size": 2,
128
+ "use_cache": true,
129
+ "vocab_size": 30000
130
+ }
model.safetensors ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:3adf2032b337cf64ce1a45fd434f4e63a5ee2331f72f2ff7b49ac633d7125ce4
3
+ size 436502704
special_tokens_map.json ADDED
@@ -0,0 +1,37 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "cls_token": {
3
+ "content": "[CLS]",
4
+ "lstrip": false,
5
+ "normalized": false,
6
+ "rstrip": false,
7
+ "single_word": false
8
+ },
9
+ "mask_token": {
10
+ "content": "[MASK]",
11
+ "lstrip": false,
12
+ "normalized": false,
13
+ "rstrip": false,
14
+ "single_word": false
15
+ },
16
+ "pad_token": {
17
+ "content": "[PAD]",
18
+ "lstrip": false,
19
+ "normalized": false,
20
+ "rstrip": false,
21
+ "single_word": false
22
+ },
23
+ "sep_token": {
24
+ "content": "[SEP]",
25
+ "lstrip": false,
26
+ "normalized": false,
27
+ "rstrip": false,
28
+ "single_word": false
29
+ },
30
+ "unk_token": {
31
+ "content": "[UNK]",
32
+ "lstrip": false,
33
+ "normalized": false,
34
+ "rstrip": false,
35
+ "single_word": false
36
+ }
37
+ }
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff
 
tokenizer_config.json ADDED
@@ -0,0 +1,55 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "added_tokens_decoder": {
3
+ "0": {
4
+ "content": "[PAD]",
5
+ "lstrip": false,
6
+ "normalized": false,
7
+ "rstrip": false,
8
+ "single_word": false,
9
+ "special": true
10
+ },
11
+ "1": {
12
+ "content": "[UNK]",
13
+ "lstrip": false,
14
+ "normalized": false,
15
+ "rstrip": false,
16
+ "single_word": false,
17
+ "special": true
18
+ },
19
+ "2": {
20
+ "content": "[CLS]",
21
+ "lstrip": false,
22
+ "normalized": false,
23
+ "rstrip": false,
24
+ "single_word": false,
25
+ "special": true
26
+ },
27
+ "3": {
28
+ "content": "[SEP]",
29
+ "lstrip": false,
30
+ "normalized": false,
31
+ "rstrip": false,
32
+ "single_word": false,
33
+ "special": true
34
+ },
35
+ "4": {
36
+ "content": "[MASK]",
37
+ "lstrip": false,
38
+ "normalized": false,
39
+ "rstrip": false,
40
+ "single_word": false,
41
+ "special": true
42
+ }
43
+ },
44
+ "clean_up_tokenization_spaces": true,
45
+ "cls_token": "[CLS]",
46
+ "do_lower_case": false,
47
+ "mask_token": "[MASK]",
48
+ "model_max_length": 512,
49
+ "pad_token": "[PAD]",
50
+ "sep_token": "[SEP]",
51
+ "strip_accents": null,
52
+ "tokenize_chinese_chars": true,
53
+ "tokenizer_class": "BertTokenizer",
54
+ "unk_token": "[UNK]"
55
+ }
training_args.bin ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:27a725604bcdf56a1dadd69cc60e25920d4431909bfb5bfe006bc77aabaee9f0
3
+ size 5112
vocab.txt ADDED
The diff for this file is too large to render. See raw diff