Hugging Face's logo

hrezaei
/
T5LAE

Text2Text Generation
Transformers
Safetensors
t5la
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
hrezaei commited on
Commit
bf1562c
·
1 Parent(s): 279bc95

Model save

Browse files
Files changed (2) hide show
  1. README.md +163 -0
  2. generation_config.json +7 -0
README.md ADDED
@@ -0,0 +1,163 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ library_name: transformers
3
+ tags:
4
+ - generated_from_trainer
5
+ metrics:
6
+ - accuracy
7
+ model-index:
8
+ - name: T5LAE
9
+ results: []
10
+ ---
11
+
12
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
13
+ should probably proofread and complete it, then remove this comment. -->
14
+
15
+ # T5LAE
16
+
17
+ This model is a fine-tuned version of [](https://huggingface.co/) on an unknown dataset.
18
+ It achieves the following results on the evaluation set:
19
+ - Loss: 6.3530
20
+ - Accuracy: 0.0323
21
+
22
+ ## Model description
23
+
24
+ More information needed
25
+
26
+ ## Intended uses & limitations
27
+
28
+ More information needed
29
+
30
+ ## Training and evaluation data
31
+
32
+ More information needed
33
+
34
+ ## Training procedure
35
+
36
+ ### Training hyperparameters
37
+
38
+ The following hyperparameters were used during training:
39
+ - learning_rate: 5e-05
40
+ - train_batch_size: 8
41
+ - eval_batch_size: 8
42
+ - seed: 42
43
+ - distributed_type: multi-GPU
44
+ - num_devices: 2
45
+ - total_train_batch_size: 16
46
+ - total_eval_batch_size: 16
47
+ - optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
48
+ - lr_scheduler_type: linear
49
+ - training_steps: 200000
50
+ - mixed_precision_training: Native AMP
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss | Accuracy |
55
+ |:-------------:|:------:|:------:|:---------------:|:--------:|
56
+ | 7.8503 | 0.01 | 2000 | 7.6894 | 0.0317 |
57
+ | 7.3885 | 0.02 | 4000 | 7.3045 | 0.0291 |
58
+ | 7.2248 | 0.03 | 6000 | 7.1483 | 0.0295 |
59
+ | 7.146 | 0.04 | 8000 | 7.0598 | 0.0298 |
60
+ | 7.098 | 0.05 | 10000 | 7.0069 | 0.0293 |
61
+ | 7.059 | 0.06 | 12000 | 6.9745 | 0.0304 |
62
+ | 7.036 | 0.07 | 14000 | 6.9492 | 0.0294 |
63
+ | 7.0083 | 0.08 | 16000 | 6.9298 | 0.0290 |
64
+ | 6.9703 | 0.09 | 18000 | 6.9145 | 0.0294 |
65
+ | 6.961 | 0.1 | 20000 | 6.9006 | 0.0303 |
66
+ | 6.9502 | 0.11 | 22000 | 6.8869 | 0.0302 |
67
+ | 6.9297 | 0.12 | 24000 | 6.8809 | 0.0282 |
68
+ | 6.9577 | 0.13 | 26000 | 6.8740 | 0.0288 |
69
+ | 6.9097 | 0.14 | 28000 | 6.8537 | 0.0290 |
70
+ | 6.9034 | 0.15 | 30000 | 6.8485 | 0.0293 |
71
+ | 6.9243 | 0.16 | 32000 | 6.8369 | 0.0292 |
72
+ | 6.8998 | 0.17 | 34000 | 6.8280 | 0.0297 |
73
+ | 6.8914 | 0.18 | 36000 | 6.8237 | 0.0289 |
74
+ | 6.8788 | 0.19 | 38000 | 6.8096 | 0.0306 |
75
+ | 6.8585 | 0.2 | 40000 | 6.8057 | 0.0295 |
76
+ | 6.8719 | 0.21 | 42000 | 6.7966 | 0.0313 |
77
+ | 6.8534 | 0.22 | 44000 | 6.7896 | 0.0297 |
78
+ | 6.8463 | 1.0067 | 46000 | 6.7795 | 0.0312 |
79
+ | 6.8588 | 1.0167 | 48000 | 6.7659 | 0.0304 |
80
+ | 6.8477 | 1.0267 | 50000 | 6.7667 | 0.0293 |
81
+ | 6.8268 | 1.0367 | 52000 | 6.7545 | 0.0301 |
82
+ | 6.8205 | 1.0467 | 54000 | 6.7439 | 0.0308 |
83
+ | 6.8035 | 1.0567 | 56000 | 6.7329 | 0.0297 |
84
+ | 6.7904 | 1.0667 | 58000 | 6.7233 | 0.0314 |
85
+ | 6.781 | 1.0767 | 60000 | 6.7235 | 0.0290 |
86
+ | 6.7722 | 1.0867 | 62000 | 6.7047 | 0.0311 |
87
+ | 6.7618 | 1.0967 | 64000 | 6.6947 | 0.0315 |
88
+ | 6.7821 | 1.1067 | 66000 | 6.6881 | 0.0309 |
89
+ | 6.7478 | 1.1167 | 68000 | 6.6781 | 0.0313 |
90
+ | 6.7544 | 1.1267 | 70000 | 6.6677 | 0.0292 |
91
+ | 6.7451 | 1.1367 | 72000 | 6.6529 | 0.0314 |
92
+ | 6.738 | 1.1467 | 74000 | 6.6436 | 0.0316 |
93
+ | 6.7223 | 1.1567 | 76000 | 6.6381 | 0.0312 |
94
+ | 6.7099 | 1.1667 | 78000 | 6.6245 | 0.0321 |
95
+ | 6.6851 | 1.1767 | 80000 | 6.6122 | 0.0311 |
96
+ | 6.6702 | 1.1867 | 82000 | 6.5993 | 0.0314 |
97
+ | 6.6761 | 1.1967 | 84000 | 6.5896 | 0.0317 |
98
+ | 6.6701 | 1.2067 | 86000 | 6.5855 | 0.0302 |
99
+ | 6.6696 | 1.2167 | 88000 | 6.5767 | 0.0313 |
100
+ | 6.6283 | 2.0035 | 90000 | 6.5673 | 0.0312 |
101
+ | 6.6662 | 2.0135 | 92000 | 6.5728 | 0.0307 |
102
+ | 6.6544 | 2.0235 | 94000 | 6.5492 | 0.0310 |
103
+ | 6.634 | 2.0335 | 96000 | 6.5433 | 0.0319 |
104
+ | 6.63 | 2.0435 | 98000 | 6.5395 | 0.0318 |
105
+ | 6.6022 | 2.0535 | 100000 | 6.5284 | 0.0318 |
106
+ | 6.5875 | 2.0635 | 102000 | 6.5209 | 0.0316 |
107
+ | 6.6115 | 2.0735 | 104000 | 6.5107 | 0.0320 |
108
+ | 6.5769 | 2.0835 | 106000 | 6.5118 | 0.0318 |
109
+ | 6.5941 | 2.0935 | 108000 | 6.4977 | 0.0312 |
110
+ | 6.5838 | 2.1035 | 110000 | 6.4884 | 0.0326 |
111
+ | 6.579 | 2.1135 | 112000 | 6.4919 | 0.0316 |
112
+ | 6.5642 | 2.1235 | 114000 | 6.4880 | 0.0318 |
113
+ | 6.5825 | 2.1335 | 116000 | 6.4747 | 0.0325 |
114
+ | 6.5625 | 2.1435 | 118000 | 6.4699 | 0.0310 |
115
+ | 6.5352 | 2.1535 | 120000 | 6.4664 | 0.0323 |
116
+ | 6.5174 | 2.1635 | 122000 | 6.4611 | 0.0320 |
117
+ | 6.5338 | 2.1735 | 124000 | 6.4618 | 0.0323 |
118
+ | 6.5264 | 2.1835 | 126000 | 6.4524 | 0.0320 |
119
+ | 6.533 | 2.1935 | 128000 | 6.4500 | 0.0315 |
120
+ | 6.5256 | 2.2035 | 130000 | 6.4433 | 0.0314 |
121
+ | 6.5293 | 2.2135 | 132000 | 6.4379 | 0.0316 |
122
+ | 6.5199 | 3.0002 | 134000 | 6.4395 | 0.0324 |
123
+ | 6.5356 | 3.0102 | 136000 | 6.4327 | 0.0321 |
124
+ | 6.4831 | 3.0202 | 138000 | 6.4207 | 0.0322 |
125
+ | 6.5051 | 3.0302 | 140000 | 6.4205 | 0.0311 |
126
+ | 6.5076 | 3.0402 | 142000 | 6.4148 | 0.0326 |
127
+ | 6.5085 | 3.0502 | 144000 | 6.4078 | 0.0323 |
128
+ | 6.5023 | 3.0602 | 146000 | 6.4070 | 0.0325 |
129
+ | 6.5019 | 3.0702 | 148000 | 6.4053 | 0.0331 |
130
+ | 6.4881 | 3.0802 | 150000 | 6.4011 | 0.0323 |
131
+ | 6.4642 | 3.0902 | 152000 | 6.4023 | 0.0316 |
132
+ | 6.4711 | 3.1002 | 154000 | 6.3948 | 0.0320 |
133
+ | 6.4713 | 3.1102 | 156000 | 6.3942 | 0.0323 |
134
+ | 6.461 | 3.1202 | 158000 | 6.3899 | 0.0319 |
135
+ | 6.4891 | 3.1302 | 160000 | 6.3877 | 0.0319 |
136
+ | 6.454 | 3.1402 | 162000 | 6.3834 | 0.0318 |
137
+ | 6.4456 | 3.1502 | 164000 | 6.3858 | 0.0319 |
138
+ | 6.4825 | 3.1602 | 166000 | 6.3827 | 0.0325 |
139
+ | 6.4563 | 3.1702 | 168000 | 6.3758 | 0.0321 |
140
+ | 6.4595 | 3.1802 | 170000 | 6.3755 | 0.0320 |
141
+ | 6.4525 | 3.1902 | 172000 | 6.3731 | 0.0319 |
142
+ | 6.4332 | 3.2002 | 174000 | 6.3691 | 0.0320 |
143
+ | 6.4656 | 3.2102 | 176000 | 6.3682 | 0.0318 |
144
+ | 6.4312 | 3.2202 | 178000 | 6.3672 | 0.0323 |
145
+ | 6.4439 | 4.0069 | 180000 | 6.3707 | 0.0317 |
146
+ | 6.4629 | 4.0169 | 182000 | 6.3619 | 0.0323 |
147
+ | 6.4505 | 4.0269 | 184000 | 6.3633 | 0.0324 |
148
+ | 6.4294 | 4.0369 | 186000 | 6.3594 | 0.0324 |
149
+ | 6.4427 | 4.0469 | 188000 | 6.3580 | 0.0319 |
150
+ | 6.4237 | 4.0569 | 190000 | 6.3600 | 0.0321 |
151
+ | 6.4201 | 4.0669 | 192000 | 6.3591 | 0.0322 |
152
+ | 6.4308 | 4.0769 | 194000 | 6.3554 | 0.0322 |
153
+ | 6.4349 | 4.0869 | 196000 | 6.3535 | 0.0323 |
154
+ | 6.4181 | 4.0969 | 198000 | 6.3542 | 0.0322 |
155
+ | 6.4385 | 4.1069 | 200000 | 6.3530 | 0.0323 |
156
+
157
+
158
+ ### Framework versions
159
+
160
+ - Transformers 4.49.0.dev0
161
+ - Pytorch 2.5.1+cu121
162
+ - Datasets 3.2.0
163
+ - Tokenizers 0.21.0
generation_config.json ADDED
@@ -0,0 +1,7 @@
 
 
 
 
 
 
 
 
1
+ {
2
+ "_from_model_config": true,
3
+ "decoder_start_token_id": 0,
4
+ "eos_token_id": 1,
5
+ "pad_token_id": 0,
6
+ "transformers_version": "4.49.0.dev0"
7
+ }