|
--- |
|
tags: |
|
- masked-auto-encoding |
|
- generated_from_trainer |
|
model-index: |
|
- name: pixel-small-wlr |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# pixel-small-wlr |
|
|
|
This model is a fine-tuned version of [](https://huggingface.co./) on the wikipedia + bookcorpus dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.8048 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0003 |
|
- train_batch_size: 64 |
|
- eval_batch_size: 8 |
|
- seed: 42 |
|
- distributed_type: multi-GPU |
|
- num_devices: 8 |
|
- total_train_batch_size: 512 |
|
- total_eval_batch_size: 64 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: cosine |
|
- lr_scheduler_warmup_ratio: 0.05 |
|
- training_steps: 500000 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:------:|:---------------:| |
|
| 0.7216 | 0.03 | 1000 | 0.9141 | |
|
| 0.7114 | 0.05 | 2000 | 0.9156 | |
|
| 0.711 | 0.08 | 3000 | 0.9157 | |
|
| 0.7102 | 0.1 | 4000 | 0.8821 | |
|
| 0.709 | 0.13 | 5000 | 0.8774 | |
|
| 0.7083 | 0.15 | 6000 | 0.8945 | |
|
| 0.6516 | 0.18 | 7000 | 0.8945 | |
|
| 0.6042 | 0.2 | 8000 | 0.8856 | |
|
| 0.5732 | 0.23 | 9000 | 0.8907 | |
|
| 0.5506 | 0.25 | 10000 | 0.8922 | |
|
| 0.5385 | 0.28 | 11000 | 0.8904 | |
|
| 0.5276 | 0.31 | 12000 | 0.8865 | |
|
| 0.517 | 0.33 | 13000 | 0.8837 | |
|
| 0.5077 | 0.36 | 14000 | 0.8863 | |
|
| 0.498 | 0.38 | 15000 | 0.8770 | |
|
| 0.4897 | 0.41 | 16000 | 0.8794 | |
|
| 0.4791 | 0.43 | 17000 | 0.8796 | |
|
| 0.4698 | 0.46 | 18000 | 0.8754 | |
|
| 0.4592 | 0.48 | 19000 | 0.8825 | |
|
| 0.4489 | 0.51 | 20000 | 0.8787 | |
|
| 0.439 | 0.54 | 21000 | 0.8742 | |
|
| 0.4292 | 0.56 | 22000 | 0.8849 | |
|
| 0.4212 | 0.59 | 23000 | 0.8842 | |
|
| 0.4142 | 0.61 | 24000 | 0.8812 | |
|
| 0.4076 | 0.64 | 25000 | 0.8659 | |
|
| 0.4017 | 0.66 | 26000 | 0.8744 | |
|
| 0.3958 | 0.69 | 27000 | 0.8822 | |
|
| 0.3907 | 0.71 | 28000 | 0.8762 | |
|
| 0.3863 | 0.74 | 29000 | 0.8758 | |
|
| 0.382 | 0.76 | 30000 | 0.8755 | |
|
| 0.378 | 0.79 | 31000 | 0.8781 | |
|
| 0.3748 | 0.82 | 32000 | 0.8815 | |
|
| 0.3716 | 0.84 | 33000 | 0.8689 | |
|
| 0.3689 | 0.87 | 34000 | 0.8759 | |
|
| 0.3665 | 0.89 | 35000 | 0.8690 | |
|
| 0.364 | 0.92 | 36000 | 0.8696 | |
|
| 0.3614 | 0.94 | 37000 | 0.8684 | |
|
| 0.3592 | 0.97 | 38000 | 0.8598 | |
|
| 0.3571 | 0.99 | 39000 | 0.8572 | |
|
| 0.3555 | 1.02 | 40000 | 0.8637 | |
|
| 0.3535 | 1.04 | 41000 | 0.8638 | |
|
| 0.3518 | 1.07 | 42000 | 0.8665 | |
|
| 0.3502 | 1.1 | 43000 | 0.8559 | |
|
| 0.3488 | 1.12 | 44000 | 0.8600 | |
|
| 0.3469 | 1.15 | 45000 | 0.8528 | |
|
| 0.3459 | 1.17 | 46000 | 0.8598 | |
|
| 0.3444 | 1.2 | 47000 | 0.8607 | |
|
| 0.3428 | 1.22 | 48000 | 0.8650 | |
|
| 0.342 | 1.25 | 49000 | 0.8640 | |
|
| 0.3408 | 1.27 | 50000 | 0.8549 | |
|
| 0.3398 | 1.3 | 51000 | 0.8630 | |
|
| 0.3387 | 1.33 | 52000 | 0.8541 | |
|
| 0.3373 | 1.35 | 53000 | 0.8588 | |
|
| 0.3368 | 1.38 | 54000 | 0.8639 | |
|
| 0.3357 | 1.4 | 55000 | 0.8546 | |
|
| 0.335 | 1.43 | 56000 | 0.8535 | |
|
| 0.334 | 1.45 | 57000 | 0.8511 | |
|
| 0.333 | 1.48 | 58000 | 0.8525 | |
|
| 0.3322 | 1.5 | 59000 | 0.8536 | |
|
| 0.3314 | 1.53 | 60000 | 0.8422 | |
|
| 0.3307 | 1.55 | 61000 | 0.8627 | |
|
| 0.3298 | 1.58 | 62000 | 0.8435 | |
|
| 0.3292 | 1.61 | 63000 | 0.8569 | |
|
| 0.3287 | 1.63 | 64000 | 0.8517 | |
|
| 0.3278 | 1.66 | 65000 | 0.8488 | |
|
| 0.3269 | 1.68 | 66000 | 0.8470 | |
|
| 0.3262 | 1.71 | 67000 | 0.8487 | |
|
| 0.3257 | 1.73 | 68000 | 0.8430 | |
|
| 0.325 | 1.76 | 69000 | 0.8382 | |
|
| 0.3248 | 1.78 | 70000 | 0.8480 | |
|
| 0.3238 | 1.81 | 71000 | 0.8432 | |
|
| 0.3235 | 1.83 | 72000 | 0.8572 | |
|
| 0.3227 | 1.86 | 73000 | 0.8466 | |
|
| 0.3224 | 1.89 | 74000 | 0.8524 | |
|
| 0.3217 | 1.91 | 75000 | 0.8451 | |
|
| 0.321 | 1.94 | 76000 | 0.8453 | |
|
| 0.3207 | 1.96 | 77000 | 0.8389 | |
|
| 0.3202 | 1.99 | 78000 | 0.8391 | |
|
| 0.3195 | 2.01 | 79000 | 0.8535 | |
|
| 0.3194 | 2.04 | 80000 | 0.8578 | |
|
| 0.3194 | 2.06 | 81000 | 0.8517 | |
|
| 0.3185 | 2.09 | 82000 | 0.8537 | |
|
| 0.3183 | 2.12 | 83000 | 0.8353 | |
|
| 0.3177 | 2.14 | 84000 | 0.8457 | |
|
| 0.3174 | 2.17 | 85000 | 0.8515 | |
|
| 0.3165 | 2.19 | 86000 | 0.8357 | |
|
| 0.3164 | 2.22 | 87000 | 0.8555 | |
|
| 0.3159 | 2.24 | 88000 | 0.8426 | |
|
| 0.3159 | 2.27 | 89000 | 0.8529 | |
|
| 0.3152 | 2.29 | 90000 | 0.8297 | |
|
| 0.3149 | 2.32 | 91000 | 0.8462 | |
|
| 0.3144 | 2.34 | 92000 | 0.8435 | |
|
| 0.3141 | 2.37 | 93000 | 0.8363 | |
|
| 0.3139 | 2.4 | 94000 | 0.8435 | |
|
| 0.3139 | 2.42 | 95000 | 0.8472 | |
|
| 0.3131 | 2.45 | 96000 | 0.8396 | |
|
| 0.3125 | 2.47 | 97000 | 0.8420 | |
|
| 0.3128 | 2.5 | 98000 | 0.8410 | |
|
| 0.3121 | 2.52 | 99000 | 0.8418 | |
|
| 0.3118 | 2.55 | 100000 | 0.8400 | |
|
| 0.3112 | 2.57 | 101000 | 0.8347 | |
|
| 0.3112 | 2.6 | 102000 | 0.8289 | |
|
| 0.3112 | 2.63 | 103000 | 0.8456 | |
|
| 0.3107 | 2.65 | 104000 | 0.8414 | |
|
| 0.3101 | 2.68 | 105000 | 0.8327 | |
|
| 0.3107 | 2.7 | 106000 | 0.8374 | |
|
| 0.3103 | 2.73 | 107000 | 0.8471 | |
|
| 0.3095 | 2.75 | 108000 | 0.8452 | |
|
| 0.3094 | 2.78 | 109000 | 0.8513 | |
|
| 0.3094 | 2.8 | 110000 | 0.8348 | |
|
| 0.3089 | 2.83 | 111000 | 0.8334 | |
|
| 0.3089 | 2.85 | 112000 | 0.8438 | |
|
| 0.3088 | 2.88 | 113000 | 0.8328 | |
|
| 0.3085 | 2.91 | 114000 | 0.8317 | |
|
| 0.3097 | 2.93 | 115000 | 0.8462 | |
|
| 0.3082 | 2.96 | 116000 | 0.8436 | |
|
| 0.3077 | 2.98 | 117000 | 0.8436 | |
|
| 0.3086 | 3.01 | 118000 | 0.8483 | |
|
| 0.3072 | 3.03 | 119000 | 0.8355 | |
|
| 0.3066 | 3.06 | 120000 | 0.8281 | |
|
| 0.3072 | 3.08 | 121000 | 0.8393 | |
|
| 0.3063 | 3.11 | 122000 | 0.8436 | |
|
| 0.3061 | 3.13 | 123000 | 0.8346 | |
|
| 0.3059 | 3.16 | 124000 | 0.8408 | |
|
| 0.3062 | 3.19 | 125000 | 0.8384 | |
|
| 0.307 | 3.21 | 126000 | 0.8374 | |
|
| 0.3056 | 3.24 | 127000 | 0.8240 | |
|
| 0.3049 | 3.26 | 128000 | 0.8263 | |
|
| 0.3068 | 3.29 | 129000 | 0.8301 | |
|
| 0.3053 | 3.31 | 130000 | 0.8350 | |
|
| 0.3048 | 3.34 | 131000 | 0.8295 | |
|
| 0.3051 | 3.36 | 132000 | 0.8297 | |
|
| 0.3045 | 3.39 | 133000 | 0.8295 | |
|
| 0.3043 | 3.42 | 134000 | 0.8245 | |
|
| 0.3037 | 3.44 | 135000 | 0.8189 | |
|
| 0.3042 | 3.47 | 136000 | 0.8286 | |
|
| 0.3038 | 3.49 | 137000 | 0.8326 | |
|
| 0.3033 | 3.52 | 138000 | 0.8184 | |
|
| 0.3035 | 3.54 | 139000 | 0.8136 | |
|
| 0.3027 | 3.57 | 140000 | 0.8287 | |
|
| 0.303 | 3.59 | 141000 | 0.8184 | |
|
| 0.3027 | 3.62 | 142000 | 0.8444 | |
|
| 0.3024 | 3.64 | 143000 | 0.8402 | |
|
| 0.3027 | 3.67 | 144000 | 0.8280 | |
|
| 0.3029 | 3.7 | 145000 | 0.8255 | |
|
| 0.3023 | 3.72 | 146000 | 0.8287 | |
|
| 0.3024 | 3.75 | 147000 | 0.8176 | |
|
| 0.302 | 3.77 | 148000 | 0.8372 | |
|
| 0.3019 | 3.8 | 149000 | 0.8221 | |
|
| 0.3016 | 3.82 | 150000 | 0.8251 | |
|
| 0.3014 | 3.85 | 151000 | 0.8370 | |
|
| 0.3012 | 3.87 | 152000 | 0.8285 | |
|
| 0.3012 | 3.9 | 153000 | 0.8453 | |
|
| 0.3007 | 3.92 | 154000 | 0.8195 | |
|
| 0.3009 | 3.95 | 155000 | 0.8309 | |
|
| 0.3007 | 3.98 | 156000 | 0.8357 | |
|
| 0.3003 | 4.0 | 157000 | 0.8225 | |
|
| 0.3014 | 4.03 | 158000 | 0.8343 | |
|
| 0.3005 | 4.05 | 159000 | 0.8267 | |
|
| 0.2994 | 4.08 | 160000 | 0.8258 | |
|
| 0.2996 | 4.1 | 161000 | 0.8267 | |
|
| 0.301 | 4.13 | 162000 | 0.8216 | |
|
| 0.2987 | 4.15 | 163000 | 0.8304 | |
|
| 0.2989 | 4.18 | 164000 | 0.8385 | |
|
| 0.2995 | 4.21 | 165000 | 0.8305 | |
|
| 0.2998 | 4.23 | 166000 | 0.8391 | |
|
| 0.2991 | 4.26 | 167000 | 0.8364 | |
|
| 0.2994 | 4.28 | 168000 | 0.8259 | |
|
| 0.2977 | 4.31 | 169000 | 0.8347 | |
|
| 0.2989 | 4.33 | 170000 | 0.8346 | |
|
| 0.2997 | 4.36 | 171000 | 0.8418 | |
|
| 0.2975 | 4.38 | 172000 | 0.8321 | |
|
| 0.2988 | 4.41 | 173000 | 0.8193 | |
|
| 0.2979 | 4.43 | 174000 | 0.8213 | |
|
| 0.2973 | 4.46 | 175000 | 0.8220 | |
|
| 0.2967 | 4.49 | 176000 | 0.8286 | |
|
| 0.2969 | 4.51 | 177000 | 0.8219 | |
|
| 0.2966 | 4.54 | 178000 | 0.8279 | |
|
| 0.2966 | 4.56 | 179000 | 0.8254 | |
|
| 0.2968 | 4.59 | 180000 | 0.8309 | |
|
| 0.2962 | 4.61 | 181000 | 0.8313 | |
|
| 0.2968 | 4.64 | 182000 | 0.8232 | |
|
| 0.2967 | 4.66 | 183000 | 0.8215 | |
|
| 0.2958 | 4.69 | 184000 | 0.8171 | |
|
| 0.2958 | 4.71 | 185000 | 0.8280 | |
|
| 0.2958 | 4.74 | 186000 | 0.8222 | |
|
| 0.2958 | 4.77 | 187000 | 0.8303 | |
|
| 0.2965 | 4.79 | 188000 | 0.8213 | |
|
| 0.2958 | 4.82 | 189000 | 0.8167 | |
|
| 0.297 | 4.84 | 190000 | 0.8272 | |
|
| 0.2959 | 4.87 | 191000 | 0.8258 | |
|
| 0.295 | 4.89 | 192000 | 0.8217 | |
|
| 0.295 | 4.92 | 193000 | 0.8130 | |
|
| 0.2968 | 4.94 | 194000 | 0.8097 | |
|
| 0.2947 | 4.97 | 195000 | 0.8070 | |
|
| 0.2941 | 5.0 | 196000 | 0.8227 | |
|
| 0.294 | 5.02 | 197000 | 0.8133 | |
|
| 0.2947 | 5.05 | 198000 | 0.8142 | |
|
| 0.2941 | 5.07 | 199000 | 0.8159 | |
|
| 0.294 | 5.1 | 200000 | 0.8274 | |
|
| 0.2941 | 5.12 | 201000 | 0.8195 | |
|
| 0.2938 | 5.15 | 202000 | 0.8285 | |
|
| 0.2934 | 5.17 | 203000 | 0.8159 | |
|
| 0.2932 | 5.2 | 204000 | 0.8073 | |
|
| 0.2946 | 5.22 | 205000 | 0.8255 | |
|
| 0.2939 | 5.25 | 206000 | 0.8250 | |
|
| 0.2933 | 5.28 | 207000 | 0.8215 | |
|
| 0.2927 | 5.3 | 208000 | 0.8153 | |
|
| 0.2931 | 5.33 | 209000 | 0.8284 | |
|
| 0.2928 | 5.35 | 210000 | 0.8204 | |
|
| 0.2923 | 5.38 | 211000 | 0.8265 | |
|
| 0.2925 | 5.4 | 212000 | 0.8269 | |
|
| 0.2926 | 5.43 | 213000 | 0.8337 | |
|
| 0.292 | 5.45 | 214000 | 0.8255 | |
|
| 0.292 | 5.48 | 215000 | 0.8224 | |
|
| 0.2915 | 5.5 | 216000 | 0.8217 | |
|
| 0.2916 | 5.53 | 217000 | 0.8251 | |
|
| 0.291 | 5.56 | 218000 | 0.8244 | |
|
| 0.2918 | 5.58 | 219000 | 0.8229 | |
|
| 0.2911 | 5.61 | 220000 | 0.8245 | |
|
| 0.2911 | 5.63 | 221000 | 0.8201 | |
|
| 0.2913 | 5.66 | 222000 | 0.8082 | |
|
| 0.2912 | 5.68 | 223000 | 0.8194 | |
|
| 0.2908 | 5.71 | 224000 | 0.8260 | |
|
| 0.291 | 5.73 | 225000 | 0.8226 | |
|
| 0.2908 | 5.76 | 226000 | 0.8231 | |
|
| 0.2903 | 5.79 | 227000 | 0.8101 | |
|
| 0.2917 | 5.81 | 228000 | 0.8148 | |
|
| 0.2915 | 5.84 | 229000 | 0.8212 | |
|
| 0.2901 | 5.86 | 230000 | 0.8126 | |
|
| 0.2898 | 5.89 | 231000 | 0.8182 | |
|
| 0.29 | 5.91 | 232000 | 0.8150 | |
|
| 0.2905 | 5.94 | 233000 | 0.8126 | |
|
| 0.2894 | 5.96 | 234000 | 0.8208 | |
|
| 0.2894 | 5.99 | 235000 | 0.8262 | |
|
| 0.2899 | 6.01 | 236000 | 0.8133 | |
|
| 0.2891 | 6.04 | 237000 | 0.8039 | |
|
| 0.2887 | 6.07 | 238000 | 0.8182 | |
|
| 0.2889 | 6.09 | 239000 | 0.8066 | |
|
| 0.2889 | 6.12 | 240000 | 0.8129 | |
|
| 0.2899 | 6.14 | 241000 | 0.8204 | |
|
| 0.2894 | 6.17 | 242000 | 0.8142 | |
|
| 0.2893 | 6.19 | 243000 | 0.8167 | |
|
| 0.2883 | 6.22 | 244000 | 0.8152 | |
|
| 0.2882 | 6.24 | 245000 | 0.8129 | |
|
| 0.2883 | 6.27 | 246000 | 0.8146 | |
|
| 0.2886 | 6.29 | 247000 | 0.8157 | |
|
| 0.2886 | 6.32 | 248000 | 0.8172 | |
|
| 0.2885 | 6.35 | 249000 | 0.8210 | |
|
| 0.2886 | 6.37 | 250000 | 0.8213 | |
|
| 0.2877 | 6.4 | 251000 | 0.8104 | |
|
| 0.2872 | 6.42 | 252000 | 0.8114 | |
|
| 0.2871 | 6.45 | 253000 | 0.8148 | |
|
| 0.2875 | 6.47 | 254000 | 0.8127 | |
|
| 0.287 | 6.5 | 255000 | 0.8201 | |
|
| 0.2869 | 6.52 | 256000 | 0.8101 | |
|
| 0.2868 | 6.55 | 257000 | 0.8142 | |
|
| 0.2869 | 6.58 | 258000 | 0.8158 | |
|
| 0.2868 | 6.6 | 259000 | 0.8125 | |
|
| 0.2865 | 6.63 | 260000 | 0.8167 | |
|
| 0.2871 | 6.65 | 261000 | 0.8194 | |
|
| 0.2863 | 6.68 | 262000 | 0.8059 | |
|
| 0.2864 | 6.7 | 263000 | 0.8195 | |
|
| 0.2863 | 6.73 | 264000 | 0.8099 | |
|
| 0.2868 | 6.75 | 265000 | 0.8127 | |
|
| 0.2863 | 6.78 | 266000 | 0.8069 | |
|
| 0.2854 | 6.8 | 267000 | 0.8033 | |
|
| 0.2855 | 6.83 | 268000 | 0.8097 | |
|
| 0.2864 | 6.86 | 269000 | 0.8096 | |
|
| 0.2865 | 6.88 | 270000 | 0.8194 | |
|
| 0.2852 | 6.91 | 271000 | 0.8104 | |
|
| 0.2852 | 6.93 | 272000 | 0.8214 | |
|
| 0.2848 | 6.96 | 273000 | 0.8105 | |
|
| 0.2857 | 6.98 | 274000 | 0.8124 | |
|
| 0.2849 | 7.01 | 275000 | 0.8164 | |
|
| 0.2848 | 7.03 | 276000 | 0.8183 | |
|
| 0.2847 | 7.06 | 277000 | 0.8188 | |
|
| 0.2846 | 7.08 | 278000 | 0.8136 | |
|
| 0.2847 | 7.11 | 279000 | 0.8129 | |
|
| 0.2845 | 7.14 | 280000 | 0.8166 | |
|
| 0.2836 | 7.16 | 281000 | 0.8175 | |
|
| 0.2839 | 7.19 | 282000 | 0.8130 | |
|
| 0.284 | 7.21 | 283000 | 0.8058 | |
|
| 0.2839 | 7.24 | 284000 | 0.8161 | |
|
| 0.2842 | 7.26 | 285000 | 0.8232 | |
|
| 0.2835 | 7.29 | 286000 | 0.8186 | |
|
| 0.2837 | 7.31 | 287000 | 0.8180 | |
|
| 0.2835 | 7.34 | 288000 | 0.8165 | |
|
| 0.2835 | 7.37 | 289000 | 0.8122 | |
|
| 0.2832 | 7.39 | 290000 | 0.8192 | |
|
| 0.2829 | 7.42 | 291000 | 0.8085 | |
|
| 0.2827 | 7.44 | 292000 | 0.8086 | |
|
| 0.2829 | 7.47 | 293000 | 0.8102 | |
|
| 0.2829 | 7.49 | 294000 | 0.8082 | |
|
| 0.2828 | 7.52 | 295000 | 0.8098 | |
|
| 0.2828 | 7.54 | 296000 | 0.8034 | |
|
| 0.2831 | 7.57 | 297000 | 0.8072 | |
|
| 0.2825 | 7.59 | 298000 | 0.8063 | |
|
| 0.282 | 7.62 | 299000 | 0.8125 | |
|
| 0.2823 | 7.65 | 300000 | 0.8154 | |
|
| 0.2818 | 7.67 | 301000 | 0.8139 | |
|
| 0.2818 | 7.7 | 302000 | 0.8098 | |
|
| 0.2826 | 7.72 | 303000 | 0.8181 | |
|
| 0.2825 | 7.75 | 304000 | 0.8146 | |
|
| 0.2813 | 7.77 | 305000 | 0.8216 | |
|
| 0.2814 | 7.8 | 306000 | 0.8134 | |
|
| 0.2808 | 7.82 | 307000 | 0.8111 | |
|
| 0.2808 | 7.85 | 308000 | 0.8111 | |
|
| 0.2811 | 7.88 | 309000 | 0.8077 | |
|
| 0.2812 | 7.9 | 310000 | 0.8111 | |
|
| 0.281 | 7.93 | 311000 | 0.8070 | |
|
| 0.2807 | 7.95 | 312000 | 0.8041 | |
|
| 0.2811 | 7.98 | 313000 | 0.8100 | |
|
| 0.2821 | 8.0 | 314000 | 0.8284 | |
|
| 0.2808 | 8.03 | 315000 | 0.8073 | |
|
| 0.2805 | 8.05 | 316000 | 0.8141 | |
|
| 0.2801 | 8.08 | 317000 | 0.8067 | |
|
| 0.28 | 8.1 | 318000 | 0.8123 | |
|
| 0.2802 | 8.13 | 319000 | 0.8078 | |
|
| 0.2799 | 8.16 | 320000 | 0.8211 | |
|
| 0.28 | 8.18 | 321000 | 0.8135 | |
|
| 0.2796 | 8.21 | 322000 | 0.8164 | |
|
| 0.2793 | 8.23 | 323000 | 0.8119 | |
|
| 0.2791 | 8.26 | 324000 | 0.8065 | |
|
| 0.2793 | 8.28 | 325000 | 0.8142 | |
|
| 0.2794 | 8.31 | 326000 | 0.8038 | |
|
| 0.2792 | 8.33 | 327000 | 0.8117 | |
|
| 0.2789 | 8.36 | 328000 | 0.8118 | |
|
| 0.2793 | 8.38 | 329000 | 0.8092 | |
|
| 0.279 | 8.41 | 330000 | 0.8081 | |
|
| 0.2792 | 8.44 | 331000 | 0.8179 | |
|
| 0.2788 | 8.46 | 332000 | 0.8141 | |
|
| 0.2785 | 8.49 | 333000 | 0.8112 | |
|
| 0.2786 | 8.51 | 334000 | 0.8080 | |
|
| 0.2788 | 8.54 | 335000 | 0.8106 | |
|
| 0.279 | 8.56 | 336000 | 0.8106 | |
|
| 0.2781 | 8.59 | 337000 | 0.8100 | |
|
| 0.278 | 8.61 | 338000 | 0.8252 | |
|
| 0.2777 | 8.64 | 339000 | 0.8137 | |
|
| 0.2778 | 8.67 | 340000 | 0.8187 | |
|
| 0.2773 | 8.69 | 341000 | 0.8103 | |
|
| 0.2779 | 8.72 | 342000 | 0.8094 | |
|
| 0.2777 | 8.74 | 343000 | 0.8024 | |
|
| 0.277 | 8.77 | 344000 | 0.8033 | |
|
| 0.2771 | 8.79 | 345000 | 0.8085 | |
|
| 0.2773 | 8.82 | 346000 | 0.8130 | |
|
| 0.2775 | 8.84 | 347000 | 0.8052 | |
|
| 0.2769 | 8.87 | 348000 | 0.8048 | |
|
| 0.2769 | 8.89 | 349000 | 0.8069 | |
|
| 0.2774 | 8.92 | 350000 | 0.8126 | |
|
| 0.2766 | 8.95 | 351000 | 0.8036 | |
|
| 0.2765 | 8.97 | 352000 | 0.8100 | |
|
| 0.2762 | 9.0 | 353000 | 0.8091 | |
|
| 0.2765 | 9.02 | 354000 | 0.8081 | |
|
| 0.2763 | 9.05 | 355000 | 0.8072 | |
|
| 0.2763 | 9.07 | 356000 | 0.8050 | |
|
| 0.2763 | 9.1 | 357000 | 0.8132 | |
|
| 0.2758 | 9.12 | 358000 | 0.8092 | |
|
| 0.2758 | 9.15 | 359000 | 0.8033 | |
|
| 0.2757 | 9.17 | 360000 | 0.8122 | |
|
| 0.2757 | 9.2 | 361000 | 0.8061 | |
|
| 0.2754 | 9.23 | 362000 | 0.8106 | |
|
| 0.2755 | 9.25 | 363000 | 0.8048 | |
|
| 0.2753 | 9.28 | 364000 | 0.8104 | |
|
| 0.2753 | 9.3 | 365000 | 0.8095 | |
|
| 0.2753 | 9.33 | 366000 | 0.8097 | |
|
| 0.2752 | 9.35 | 367000 | 0.8090 | |
|
| 0.2749 | 9.38 | 368000 | 0.8059 | |
|
| 0.2749 | 9.4 | 369000 | 0.8114 | |
|
| 0.2747 | 9.43 | 370000 | 0.8089 | |
|
| 0.2745 | 9.46 | 371000 | 0.8080 | |
|
| 0.2745 | 9.48 | 372000 | 0.8102 | |
|
| 0.2747 | 9.51 | 373000 | 0.8059 | |
|
| 0.2742 | 9.53 | 374000 | 0.8085 | |
|
| 0.2742 | 9.56 | 375000 | 0.8031 | |
|
| 0.274 | 9.58 | 376000 | 0.8067 | |
|
| 0.274 | 9.61 | 377000 | 0.8057 | |
|
| 0.274 | 9.63 | 378000 | 0.8031 | |
|
| 0.2738 | 9.66 | 379000 | 0.8067 | |
|
| 0.2737 | 9.68 | 380000 | 0.8090 | |
|
| 0.2736 | 9.71 | 381000 | 0.8044 | |
|
| 0.2739 | 9.74 | 382000 | 0.8078 | |
|
| 0.2729 | 9.76 | 383000 | 0.8075 | |
|
| 0.2735 | 9.79 | 384000 | 0.8107 | |
|
| 0.2729 | 9.81 | 385000 | 0.8120 | |
|
| 0.2731 | 9.84 | 386000 | 0.8059 | |
|
| 0.2727 | 9.86 | 387000 | 0.8082 | |
|
| 0.2726 | 9.89 | 388000 | 0.8090 | |
|
| 0.2727 | 9.91 | 389000 | 0.8020 | |
|
| 0.273 | 9.94 | 390000 | 0.8115 | |
|
| 0.2727 | 9.96 | 391000 | 0.8077 | |
|
| 0.2726 | 9.99 | 392000 | 0.8175 | |
|
| 0.2722 | 10.02 | 393000 | 0.8073 | |
|
| 0.2725 | 10.04 | 394000 | 0.8089 | |
|
| 0.2721 | 10.07 | 395000 | 0.8181 | |
|
| 0.2722 | 10.09 | 396000 | 0.8067 | |
|
| 0.2721 | 10.12 | 397000 | 0.8155 | |
|
| 0.2718 | 10.14 | 398000 | 0.8150 | |
|
| 0.272 | 10.17 | 399000 | 0.8131 | |
|
| 0.2721 | 10.19 | 400000 | 0.8092 | |
|
| 0.2715 | 10.22 | 401000 | 0.8083 | |
|
| 0.2717 | 10.25 | 402000 | 0.8100 | |
|
| 0.2715 | 10.27 | 403000 | 0.8108 | |
|
| 0.2715 | 10.3 | 404000 | 0.8090 | |
|
| 0.2716 | 10.32 | 405000 | 0.8160 | |
|
| 0.2712 | 10.35 | 406000 | 0.8142 | |
|
| 0.2712 | 10.37 | 407000 | 0.8071 | |
|
| 0.2712 | 10.4 | 408000 | 0.8115 | |
|
| 0.2709 | 10.42 | 409000 | 0.8093 | |
|
| 0.271 | 10.45 | 410000 | 0.8109 | |
|
| 0.2712 | 10.47 | 411000 | 0.8162 | |
|
| 0.2709 | 10.5 | 412000 | 0.8158 | |
|
| 0.2706 | 10.53 | 413000 | 0.8103 | |
|
| 0.2709 | 10.55 | 414000 | 0.8069 | |
|
| 0.2706 | 10.58 | 415000 | 0.8130 | |
|
| 0.2706 | 10.6 | 416000 | 0.8126 | |
|
| 0.2704 | 10.63 | 417000 | 0.8181 | |
|
| 0.2704 | 10.65 | 418000 | 0.8100 | |
|
| 0.2702 | 10.68 | 419000 | 0.8089 | |
|
| 0.2702 | 10.7 | 420000 | 0.8133 | |
|
| 0.2699 | 10.73 | 421000 | 0.8155 | |
|
| 0.2701 | 10.75 | 422000 | 0.8139 | |
|
| 0.2701 | 10.78 | 423000 | 0.8133 | |
|
| 0.2701 | 10.81 | 424000 | 0.8100 | |
|
| 0.2696 | 10.83 | 425000 | 0.8077 | |
|
| 0.2696 | 10.86 | 426000 | 0.8097 | |
|
| 0.2698 | 10.88 | 427000 | 0.8036 | |
|
| 0.2698 | 10.91 | 428000 | 0.8067 | |
|
| 0.2699 | 10.93 | 429000 | 0.8131 | |
|
| 0.2695 | 10.96 | 430000 | 0.8059 | |
|
| 0.2695 | 10.98 | 431000 | 0.8142 | |
|
| 0.2693 | 11.01 | 432000 | 0.8080 | |
|
| 0.2695 | 11.04 | 433000 | 0.8101 | |
|
| 0.2692 | 11.06 | 434000 | 0.8111 | |
|
| 0.2693 | 11.09 | 435000 | 0.8064 | |
|
| 0.2689 | 11.11 | 436000 | 0.8066 | |
|
| 0.2688 | 11.14 | 437000 | 0.8145 | |
|
| 0.2691 | 11.16 | 438000 | 0.8088 | |
|
| 0.2689 | 11.19 | 439000 | 0.8115 | |
|
| 0.2688 | 11.21 | 440000 | 0.8066 | |
|
| 0.2689 | 11.24 | 441000 | 0.8038 | |
|
| 0.2687 | 11.26 | 442000 | 0.8066 | |
|
| 0.2688 | 11.29 | 443000 | 0.8125 | |
|
| 0.2686 | 11.32 | 444000 | 0.8055 | |
|
| 0.2686 | 11.34 | 445000 | 0.8065 | |
|
| 0.2685 | 11.37 | 446000 | 0.8134 | |
|
| 0.2684 | 11.39 | 447000 | 0.8068 | |
|
| 0.2683 | 11.42 | 448000 | 0.8086 | |
|
| 0.2684 | 11.44 | 449000 | 0.8025 | |
|
| 0.2682 | 11.47 | 450000 | 0.8073 | |
|
| 0.2682 | 11.49 | 451000 | 0.8042 | |
|
| 0.2683 | 11.52 | 452000 | 0.8097 | |
|
| 0.2678 | 11.54 | 453000 | 0.8062 | |
|
| 0.2678 | 11.57 | 454000 | 0.8084 | |
|
| 0.2681 | 11.6 | 455000 | 0.8135 | |
|
| 0.2678 | 11.62 | 456000 | 0.8098 | |
|
| 0.2681 | 11.65 | 457000 | 0.8079 | |
|
| 0.2679 | 11.67 | 458000 | 0.8052 | |
|
| 0.268 | 11.7 | 459000 | 0.8038 | |
|
| 0.268 | 11.72 | 460000 | 0.8100 | |
|
| 0.2677 | 11.75 | 461000 | 0.8057 | |
|
| 0.2676 | 11.77 | 462000 | 0.8142 | |
|
| 0.2679 | 11.8 | 463000 | 0.8076 | |
|
| 0.2676 | 11.83 | 464000 | 0.8087 | |
|
| 0.2677 | 11.85 | 465000 | 0.8066 | |
|
| 0.2673 | 11.88 | 466000 | 0.8059 | |
|
| 0.2676 | 11.9 | 467000 | 0.8067 | |
|
| 0.2675 | 11.93 | 468000 | 0.8043 | |
|
| 0.2675 | 11.95 | 469000 | 0.8103 | |
|
| 0.2673 | 11.98 | 470000 | 0.8092 | |
|
| 0.2676 | 12.0 | 471000 | 0.8069 | |
|
| 0.2673 | 12.03 | 472000 | 0.8062 | |
|
| 0.2673 | 12.05 | 473000 | 0.8025 | |
|
| 0.2672 | 12.08 | 474000 | 0.8044 | |
|
| 0.2671 | 12.11 | 475000 | 0.8068 | |
|
| 0.2672 | 12.13 | 476000 | 0.8039 | |
|
| 0.2673 | 12.16 | 477000 | 0.8078 | |
|
| 0.2671 | 12.18 | 478000 | 0.8061 | |
|
| 0.2673 | 12.21 | 479000 | 0.8022 | |
|
| 0.267 | 12.23 | 480000 | 0.8065 | |
|
| 0.2672 | 12.26 | 481000 | 0.8035 | |
|
| 0.267 | 12.28 | 482000 | 0.8039 | |
|
| 0.2669 | 12.31 | 483000 | 0.8074 | |
|
| 0.267 | 12.33 | 484000 | 0.8040 | |
|
| 0.267 | 12.36 | 485000 | 0.8028 | |
|
| 0.2668 | 12.39 | 486000 | 0.8055 | |
|
| 0.2669 | 12.41 | 487000 | 0.8062 | |
|
| 0.2669 | 12.44 | 488000 | 0.8053 | |
|
| 0.267 | 12.46 | 489000 | 0.8089 | |
|
| 0.267 | 12.49 | 490000 | 0.8081 | |
|
| 0.267 | 12.51 | 491000 | 0.8053 | |
|
| 0.2668 | 12.54 | 492000 | 0.8053 | |
|
| 0.2671 | 12.56 | 493000 | 0.8097 | |
|
| 0.267 | 12.59 | 494000 | 0.8088 | |
|
| 0.2669 | 12.62 | 495000 | 0.8081 | |
|
| 0.2667 | 12.64 | 496000 | 0.8047 | |
|
| 0.2667 | 12.67 | 497000 | 0.8043 | |
|
| 0.2669 | 12.69 | 498000 | 0.8051 | |
|
| 0.2669 | 12.72 | 499000 | 0.8085 | |
|
| 0.2666 | 12.74 | 500000 | 0.8054 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.17.0 |
|
- Pytorch 1.11.0 |
|
- Datasets 2.5.0 |
|
- Tokenizers 0.12.1 |
|
|
