--- library_name: transformers base_model: aubmindlab/bert-base-arabertv02 tags: - generated_from_trainer model-index: - name: Arabic_FineTuningAraBERT_AugV4-trial2_k2_task1_organization_fold1 results: [] --- # Arabic_FineTuningAraBERT_AugV4-trial2_k2_task1_organization_fold1 This model is a fine-tuned version of [aubmindlab/bert-base-arabertv02](https://huggingface.co./aubmindlab/bert-base-arabertv02) on the None dataset. It achieves the following results on the evaluation set: - Loss: 0.6479 - Qwk: 0.7390 - Mse: 0.6479 - Rmse: 0.8049 ## Model description More information needed ## Intended uses & limitations More information needed ## Training and evaluation data More information needed ## Training procedure ### Training hyperparameters The following hyperparameters were used during training: - learning_rate: 2e-05 - train_batch_size: 8 - eval_batch_size: 8 - seed: 42 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 - lr_scheduler_type: linear - num_epochs: 10 ### Training results | Training Loss | Epoch | Step | Validation Loss | Qwk | Mse | Rmse | |:-------------:|:------:|:----:|:---------------:|:-------:|:------:|:------:| | No log | 0.0444 | 2 | 3.0936 | 0.0535 | 3.0936 | 1.7589 | | No log | 0.0889 | 4 | 1.5812 | -0.0672 | 1.5812 | 1.2575 | | No log | 0.1333 | 6 | 0.7505 | 0.1212 | 0.7505 | 0.8663 | | No log | 0.1778 | 8 | 0.7640 | 0.6051 | 0.7640 | 0.8741 | | No log | 0.2222 | 10 | 1.0786 | 0.2125 | 1.0786 | 1.0385 | | No log | 0.2667 | 12 | 1.0415 | 0.3563 | 1.0415 | 1.0205 | | No log | 0.3111 | 14 | 0.5944 | 0.5044 | 0.5944 | 0.7710 | | No log | 0.3556 | 16 | 0.7187 | 0.4201 | 0.7187 | 0.8478 | | No log | 0.4 | 18 | 0.5325 | 0.4474 | 0.5325 | 0.7297 | | No log | 0.4444 | 20 | 0.6558 | 0.51 | 0.6558 | 0.8098 | | No log | 0.4889 | 22 | 0.9806 | 0.4455 | 0.9806 | 0.9902 | | No log | 0.5333 | 24 | 1.0506 | 0.4368 | 1.0506 | 1.0250 | | No log | 0.5778 | 26 | 0.8678 | 0.4096 | 0.8678 | 0.9315 | | No log | 0.6222 | 28 | 0.6718 | 0.3897 | 0.6718 | 0.8197 | | No log | 0.6667 | 30 | 0.6200 | 0.4717 | 0.6200 | 0.7874 | | No log | 0.7111 | 32 | 0.5349 | 0.5532 | 0.5349 | 0.7314 | | No log | 0.7556 | 34 | 0.5505 | 0.5172 | 0.5505 | 0.7420 | | No log | 0.8 | 36 | 0.6261 | 0.5591 | 0.6261 | 0.7913 | | No log | 0.8444 | 38 | 0.9287 | 0.6851 | 0.9287 | 0.9637 | | No log | 0.8889 | 40 | 1.1843 | 0.7147 | 1.1843 | 1.0883 | | No log | 0.9333 | 42 | 0.8376 | 0.7552 | 0.8376 | 0.9152 | | No log | 0.9778 | 44 | 0.5065 | 0.6540 | 0.5065 | 0.7117 | | No log | 1.0222 | 46 | 0.4681 | 0.6062 | 0.4681 | 0.6842 | | No log | 1.0667 | 48 | 0.4454 | 0.6062 | 0.4454 | 0.6674 | | No log | 1.1111 | 50 | 0.5397 | 0.7372 | 0.5397 | 0.7347 | | No log | 1.1556 | 52 | 0.9865 | 0.7 | 0.9865 | 0.9932 | | No log | 1.2 | 54 | 1.2991 | 0.6839 | 1.2991 | 1.1398 | | No log | 1.2444 | 56 | 1.1028 | 0.7 | 1.1028 | 1.0501 | | No log | 1.2889 | 58 | 0.7718 | 0.7138 | 0.7718 | 0.8785 | | No log | 1.3333 | 60 | 0.5857 | 0.7009 | 0.5857 | 0.7653 | | No log | 1.3778 | 62 | 0.5043 | 0.6751 | 0.5043 | 0.7102 | | No log | 1.4222 | 64 | 0.5274 | 0.6751 | 0.5274 | 0.7262 | | No log | 1.4667 | 66 | 0.6278 | 0.6719 | 0.6278 | 0.7923 | | No log | 1.5111 | 68 | 0.7948 | 0.7279 | 0.7948 | 0.8915 | | No log | 1.5556 | 70 | 0.7247 | 0.6851 | 0.7247 | 0.8513 | | No log | 1.6 | 72 | 0.5944 | 0.6776 | 0.5944 | 0.7710 | | No log | 1.6444 | 74 | 0.6215 | 0.7111 | 0.6215 | 0.7884 | | No log | 1.6889 | 76 | 0.6426 | 0.6918 | 0.6426 | 0.8016 | | No log | 1.7333 | 78 | 0.6170 | 0.6789 | 0.6170 | 0.7855 | | No log | 1.7778 | 80 | 0.5731 | 0.6490 | 0.5731 | 0.7571 | | No log | 1.8222 | 82 | 0.5651 | 0.6167 | 0.5651 | 0.7517 | | No log | 1.8667 | 84 | 0.5892 | 0.7701 | 0.5892 | 0.7676 | | No log | 1.9111 | 86 | 0.5792 | 0.7407 | 0.5792 | 0.7610 | | No log | 1.9556 | 88 | 0.6997 | 0.7892 | 0.6997 | 0.8365 | | No log | 2.0 | 90 | 0.9112 | 0.6936 | 0.9112 | 0.9546 | | No log | 2.0444 | 92 | 1.1582 | 0.6557 | 1.1582 | 1.0762 | | No log | 2.0889 | 94 | 0.9782 | 0.6936 | 0.9782 | 0.9890 | | No log | 2.1333 | 96 | 0.6711 | 0.7308 | 0.6711 | 0.8192 | | No log | 2.1778 | 98 | 0.5175 | 0.7840 | 0.5175 | 0.7194 | | No log | 2.2222 | 100 | 0.4991 | 0.7840 | 0.4991 | 0.7065 | | No log | 2.2667 | 102 | 0.6537 | 0.6441 | 0.6537 | 0.8085 | | No log | 2.3111 | 104 | 1.0720 | 0.75 | 1.0720 | 1.0354 | | No log | 2.3556 | 106 | 1.2298 | 0.7388 | 1.2298 | 1.1090 | | No log | 2.4 | 108 | 1.0252 | 0.7154 | 1.0252 | 1.0125 | | No log | 2.4444 | 110 | 0.6535 | 0.72 | 0.6535 | 0.8084 | | No log | 2.4889 | 112 | 0.4594 | 0.7083 | 0.4594 | 0.6778 | | No log | 2.5333 | 114 | 0.3551 | 0.7619 | 0.3551 | 0.5959 | | No log | 2.5778 | 116 | 0.3586 | 0.7619 | 0.3586 | 0.5988 | | No log | 2.6222 | 118 | 0.4895 | 0.7840 | 0.4895 | 0.6997 | | No log | 2.6667 | 120 | 0.8257 | 0.7407 | 0.8257 | 0.9087 | | No log | 2.7111 | 122 | 0.9099 | 0.7407 | 0.9099 | 0.9539 | | No log | 2.7556 | 124 | 0.7306 | 0.7941 | 0.7306 | 0.8548 | | No log | 2.8 | 126 | 0.6391 | 0.7755 | 0.6391 | 0.7994 | | No log | 2.8444 | 128 | 0.5891 | 0.7470 | 0.5891 | 0.7675 | | No log | 2.8889 | 130 | 0.5420 | 0.7165 | 0.5420 | 0.7362 | | No log | 2.9333 | 132 | 0.5820 | 0.7470 | 0.5820 | 0.7629 | | No log | 2.9778 | 134 | 0.6555 | 0.7556 | 0.6555 | 0.8096 | | No log | 3.0222 | 136 | 0.6538 | 0.7556 | 0.6538 | 0.8086 | | No log | 3.0667 | 138 | 0.6013 | 0.7701 | 0.6013 | 0.7754 | | No log | 3.1111 | 140 | 0.6256 | 0.7701 | 0.6256 | 0.7910 | | No log | 3.1556 | 142 | 0.7632 | 0.7470 | 0.7632 | 0.8736 | | No log | 3.2 | 144 | 0.7431 | 0.7667 | 0.7431 | 0.8621 | | No log | 3.2444 | 146 | 0.6433 | 0.6527 | 0.6433 | 0.8021 | | No log | 3.2889 | 148 | 0.6175 | 0.6480 | 0.6175 | 0.7858 | | No log | 3.3333 | 150 | 0.6131 | 0.7407 | 0.6131 | 0.7830 | | No log | 3.3778 | 152 | 0.7499 | 0.6776 | 0.7499 | 0.8660 | | No log | 3.4222 | 154 | 0.9000 | 0.6691 | 0.9000 | 0.9487 | | No log | 3.4667 | 156 | 0.8301 | 0.6978 | 0.8301 | 0.9111 | | No log | 3.5111 | 158 | 0.5912 | 0.7103 | 0.5912 | 0.7689 | | No log | 3.5556 | 160 | 0.5163 | 0.6894 | 0.5163 | 0.7185 | | No log | 3.6 | 162 | 0.5182 | 0.7021 | 0.5182 | 0.7198 | | No log | 3.6444 | 164 | 0.6091 | 0.7042 | 0.6091 | 0.7804 | | No log | 3.6889 | 166 | 0.8286 | 0.6691 | 0.8286 | 0.9103 | | No log | 3.7333 | 168 | 0.8981 | 0.6912 | 0.8981 | 0.9477 | | No log | 3.7778 | 170 | 0.8358 | 0.6475 | 0.8358 | 0.9142 | | No log | 3.8222 | 172 | 0.8260 | 0.6851 | 0.8260 | 0.9088 | | No log | 3.8667 | 174 | 0.7376 | 0.7055 | 0.7376 | 0.8588 | | No log | 3.9111 | 176 | 0.6743 | 0.7660 | 0.6743 | 0.8211 | | No log | 3.9556 | 178 | 0.7112 | 0.7660 | 0.7112 | 0.8433 | | No log | 4.0 | 180 | 0.8755 | 0.6851 | 0.8755 | 0.9357 | | No log | 4.0444 | 182 | 0.9757 | 0.7619 | 0.9757 | 0.9878 | | No log | 4.0889 | 184 | 0.9356 | 0.7619 | 0.9356 | 0.9673 | | No log | 4.1333 | 186 | 0.6926 | 0.7756 | 0.6926 | 0.8322 | | No log | 4.1778 | 188 | 0.5585 | 0.7442 | 0.5585 | 0.7473 | | No log | 4.2222 | 190 | 0.5327 | 0.7442 | 0.5327 | 0.7298 | | No log | 4.2667 | 192 | 0.5420 | 0.7442 | 0.5420 | 0.7362 | | No log | 4.3111 | 194 | 0.5895 | 0.7756 | 0.5895 | 0.7678 | | No log | 4.3556 | 196 | 0.7837 | 0.7508 | 0.7837 | 0.8853 | | No log | 4.4 | 198 | 0.8694 | 0.7508 | 0.8694 | 0.9324 | | No log | 4.4444 | 200 | 0.7923 | 0.7508 | 0.7923 | 0.8901 | | No log | 4.4889 | 202 | 0.6802 | 0.7508 | 0.6802 | 0.8248 | | No log | 4.5333 | 204 | 0.5874 | 0.7651 | 0.5874 | 0.7664 | | No log | 4.5778 | 206 | 0.5541 | 0.7651 | 0.5541 | 0.7444 | | No log | 4.6222 | 208 | 0.6135 | 0.7508 | 0.6135 | 0.7833 | | No log | 4.6667 | 210 | 0.6457 | 0.7508 | 0.6457 | 0.8036 | | No log | 4.7111 | 212 | 0.6241 | 0.7423 | 0.6241 | 0.7900 | | No log | 4.7556 | 214 | 0.6275 | 0.7556 | 0.6275 | 0.7922 | | No log | 4.8 | 216 | 0.5903 | 0.8119 | 0.5903 | 0.7683 | | No log | 4.8444 | 218 | 0.6429 | 0.7556 | 0.6429 | 0.8018 | | No log | 4.8889 | 220 | 0.8524 | 0.8182 | 0.8524 | 0.9233 | | No log | 4.9333 | 222 | 0.9941 | 0.8182 | 0.9941 | 0.9970 | | No log | 4.9778 | 224 | 1.1285 | 0.7682 | 1.1285 | 1.0623 | | No log | 5.0222 | 226 | 0.9966 | 0.7835 | 0.9966 | 0.9983 | | No log | 5.0667 | 228 | 0.7130 | 0.7921 | 0.7130 | 0.8444 | | No log | 5.1111 | 230 | 0.4722 | 0.7635 | 0.4722 | 0.6872 | | No log | 5.1556 | 232 | 0.3973 | 0.7298 | 0.3973 | 0.6303 | | No log | 5.2 | 234 | 0.4260 | 0.7298 | 0.4260 | 0.6527 | | No log | 5.2444 | 236 | 0.5369 | 0.7317 | 0.5369 | 0.7328 | | No log | 5.2889 | 238 | 0.7463 | 0.7336 | 0.7463 | 0.8639 | | No log | 5.3333 | 240 | 1.0427 | 0.7835 | 1.0427 | 1.0211 | | No log | 5.3778 | 242 | 1.1381 | 0.7835 | 1.1381 | 1.0668 | | No log | 5.4222 | 244 | 1.0315 | 0.7835 | 1.0315 | 1.0157 | | No log | 5.4667 | 246 | 0.8190 | 0.7336 | 0.8190 | 0.9050 | | No log | 5.5111 | 248 | 0.6333 | 0.7508 | 0.6333 | 0.7958 | | No log | 5.5556 | 250 | 0.5988 | 0.7111 | 0.5988 | 0.7738 | | No log | 5.6 | 252 | 0.6143 | 0.7111 | 0.6143 | 0.7838 | | No log | 5.6444 | 254 | 0.6902 | 0.7423 | 0.6902 | 0.8308 | | No log | 5.6889 | 256 | 0.7687 | 0.7508 | 0.7687 | 0.8767 | | No log | 5.7333 | 258 | 0.8131 | 0.7508 | 0.8131 | 0.9017 | | No log | 5.7778 | 260 | 0.8370 | 0.7123 | 0.8370 | 0.9149 | | No log | 5.8222 | 262 | 0.8030 | 0.7508 | 0.8030 | 0.8961 | | No log | 5.8667 | 264 | 0.7342 | 0.7423 | 0.7342 | 0.8569 | | No log | 5.9111 | 266 | 0.5823 | 0.7701 | 0.5823 | 0.7631 | | No log | 5.9556 | 268 | 0.4959 | 0.7701 | 0.4959 | 0.7042 | | No log | 6.0 | 270 | 0.5064 | 0.7701 | 0.5064 | 0.7116 | | No log | 6.0444 | 272 | 0.5502 | 0.7701 | 0.5502 | 0.7418 | | No log | 6.0889 | 274 | 0.6633 | 0.7470 | 0.6633 | 0.8144 | | No log | 6.1333 | 276 | 0.7941 | 0.7423 | 0.7941 | 0.8911 | | No log | 6.1778 | 278 | 0.8693 | 0.7063 | 0.8693 | 0.9324 | | No log | 6.2222 | 280 | 0.8540 | 0.7063 | 0.8540 | 0.9241 | | No log | 6.2667 | 282 | 0.7325 | 0.7336 | 0.7325 | 0.8558 | | No log | 6.3111 | 284 | 0.6224 | 0.7181 | 0.6224 | 0.7890 | | No log | 6.3556 | 286 | 0.5378 | 0.7111 | 0.5378 | 0.7334 | | No log | 6.4 | 288 | 0.5179 | 0.7358 | 0.5179 | 0.7196 | | No log | 6.4444 | 290 | 0.5736 | 0.7111 | 0.5736 | 0.7573 | | No log | 6.4889 | 292 | 0.7018 | 0.7616 | 0.7018 | 0.8378 | | No log | 6.5333 | 294 | 0.7965 | 0.7712 | 0.7965 | 0.8925 | | No log | 6.5778 | 296 | 0.7783 | 0.7712 | 0.7783 | 0.8822 | | No log | 6.6222 | 298 | 0.7714 | 0.7712 | 0.7714 | 0.8783 | | No log | 6.6667 | 300 | 0.6733 | 0.7712 | 0.6733 | 0.8206 | | No log | 6.7111 | 302 | 0.5652 | 0.7111 | 0.5652 | 0.7518 | | No log | 6.7556 | 304 | 0.5270 | 0.7358 | 0.5270 | 0.7259 | | No log | 6.8 | 306 | 0.5700 | 0.7111 | 0.5700 | 0.7550 | | No log | 6.8444 | 308 | 0.5859 | 0.7308 | 0.5859 | 0.7655 | | No log | 6.8889 | 310 | 0.5868 | 0.7308 | 0.5868 | 0.7660 | | No log | 6.9333 | 312 | 0.6444 | 0.7308 | 0.6444 | 0.8028 | | No log | 6.9778 | 314 | 0.7491 | 0.7812 | 0.7491 | 0.8655 | | No log | 7.0222 | 316 | 0.8528 | 0.7667 | 0.8528 | 0.9235 | | No log | 7.0667 | 318 | 0.8941 | 0.7619 | 0.8941 | 0.9456 | | No log | 7.1111 | 320 | 0.8267 | 0.7667 | 0.8267 | 0.9092 | | No log | 7.1556 | 322 | 0.7637 | 0.7812 | 0.7637 | 0.8739 | | No log | 7.2 | 324 | 0.7661 | 0.7812 | 0.7661 | 0.8753 | | No log | 7.2444 | 326 | 0.7660 | 0.7812 | 0.7660 | 0.8752 | | No log | 7.2889 | 328 | 0.7335 | 0.7812 | 0.7335 | 0.8565 | | No log | 7.3333 | 330 | 0.7283 | 0.7812 | 0.7283 | 0.8534 | | No log | 7.3778 | 332 | 0.7050 | 0.7812 | 0.7050 | 0.8397 | | No log | 7.4222 | 334 | 0.6808 | 0.7123 | 0.6808 | 0.8251 | | No log | 7.4667 | 336 | 0.6910 | 0.72 | 0.6910 | 0.8313 | | No log | 7.5111 | 338 | 0.7104 | 0.7552 | 0.7104 | 0.8428 | | No log | 7.5556 | 340 | 0.7600 | 0.7279 | 0.7600 | 0.8718 | | No log | 7.6 | 342 | 0.8177 | 0.7279 | 0.8177 | 0.9043 | | No log | 7.6444 | 344 | 0.8233 | 0.7279 | 0.8233 | 0.9074 | | No log | 7.6889 | 346 | 0.7455 | 0.7667 | 0.7455 | 0.8634 | | No log | 7.7333 | 348 | 0.6825 | 0.7921 | 0.6825 | 0.8262 | | No log | 7.7778 | 350 | 0.6387 | 0.7111 | 0.6387 | 0.7992 | | No log | 7.8222 | 352 | 0.6213 | 0.7111 | 0.6213 | 0.7882 | | No log | 7.8667 | 354 | 0.6104 | 0.7111 | 0.6104 | 0.7813 | | No log | 7.9111 | 356 | 0.6131 | 0.7111 | 0.6131 | 0.7830 | | No log | 7.9556 | 358 | 0.6103 | 0.7111 | 0.6103 | 0.7812 | | No log | 8.0 | 360 | 0.5798 | 0.7111 | 0.5798 | 0.7615 | | No log | 8.0444 | 362 | 0.5539 | 0.7111 | 0.5539 | 0.7443 | | No log | 8.0889 | 364 | 0.5522 | 0.7111 | 0.5522 | 0.7431 | | No log | 8.1333 | 366 | 0.5833 | 0.7111 | 0.5833 | 0.7638 | | No log | 8.1778 | 368 | 0.5982 | 0.7470 | 0.5982 | 0.7734 | | No log | 8.2222 | 370 | 0.6293 | 0.7470 | 0.6293 | 0.7933 | | No log | 8.2667 | 372 | 0.6434 | 0.7470 | 0.6434 | 0.8021 | | No log | 8.3111 | 374 | 0.6544 | 0.7470 | 0.6544 | 0.8090 | | No log | 8.3556 | 376 | 0.6369 | 0.7470 | 0.6369 | 0.7981 | | No log | 8.4 | 378 | 0.6097 | 0.7470 | 0.6097 | 0.7808 | | No log | 8.4444 | 380 | 0.5876 | 0.7470 | 0.5876 | 0.7666 | | No log | 8.4889 | 382 | 0.5832 | 0.7470 | 0.5832 | 0.7637 | | No log | 8.5333 | 384 | 0.6026 | 0.7470 | 0.6026 | 0.7763 | | No log | 8.5778 | 386 | 0.6088 | 0.7470 | 0.6088 | 0.7803 | | No log | 8.6222 | 388 | 0.6099 | 0.7470 | 0.6099 | 0.7809 | | No log | 8.6667 | 390 | 0.6091 | 0.7470 | 0.6091 | 0.7805 | | No log | 8.7111 | 392 | 0.6285 | 0.7660 | 0.6285 | 0.7928 | | No log | 8.7556 | 394 | 0.6667 | 0.7603 | 0.6667 | 0.8165 | | No log | 8.8 | 396 | 0.6736 | 0.7336 | 0.6736 | 0.8208 | | No log | 8.8444 | 398 | 0.6629 | 0.7336 | 0.6629 | 0.8142 | | No log | 8.8889 | 400 | 0.6538 | 0.7603 | 0.6538 | 0.8086 | | No log | 8.9333 | 402 | 0.6436 | 0.7390 | 0.6436 | 0.8022 | | No log | 8.9778 | 404 | 0.6508 | 0.7390 | 0.6508 | 0.8067 | | No log | 9.0222 | 406 | 0.6510 | 0.7390 | 0.6510 | 0.8068 | | No log | 9.0667 | 408 | 0.6596 | 0.7603 | 0.6596 | 0.8122 | | No log | 9.1111 | 410 | 0.6640 | 0.7603 | 0.6640 | 0.8149 | | No log | 9.1556 | 412 | 0.6660 | 0.7603 | 0.6660 | 0.8161 | | No log | 9.2 | 414 | 0.6581 | 0.7603 | 0.6581 | 0.8112 | | No log | 9.2444 | 416 | 0.6610 | 0.7603 | 0.6610 | 0.8130 | | No log | 9.2889 | 418 | 0.6725 | 0.7603 | 0.6725 | 0.8201 | | No log | 9.3333 | 420 | 0.6817 | 0.7336 | 0.6817 | 0.8257 | | No log | 9.3778 | 422 | 0.6897 | 0.7336 | 0.6897 | 0.8305 | | No log | 9.4222 | 424 | 0.6891 | 0.7336 | 0.6891 | 0.8301 | | No log | 9.4667 | 426 | 0.6885 | 0.7603 | 0.6885 | 0.8298 | | No log | 9.5111 | 428 | 0.6961 | 0.7336 | 0.6961 | 0.8343 | | No log | 9.5556 | 430 | 0.6949 | 0.7603 | 0.6949 | 0.8336 | | No log | 9.6 | 432 | 0.6901 | 0.7603 | 0.6901 | 0.8307 | | No log | 9.6444 | 434 | 0.6842 | 0.7603 | 0.6842 | 0.8272 | | No log | 9.6889 | 436 | 0.6775 | 0.7603 | 0.6775 | 0.8231 | | No log | 9.7333 | 438 | 0.6751 | 0.7603 | 0.6751 | 0.8217 | | No log | 9.7778 | 440 | 0.6689 | 0.7603 | 0.6689 | 0.8179 | | No log | 9.8222 | 442 | 0.6638 | 0.7603 | 0.6638 | 0.8147 | | No log | 9.8667 | 444 | 0.6581 | 0.7390 | 0.6581 | 0.8112 | | No log | 9.9111 | 446 | 0.6520 | 0.7390 | 0.6520 | 0.8075 | | No log | 9.9556 | 448 | 0.6488 | 0.7390 | 0.6488 | 0.8055 | | No log | 10.0 | 450 | 0.6479 | 0.7390 | 0.6479 | 0.8049 | ### Framework versions - Transformers 4.44.2 - Pytorch 2.4.0+cu118 - Datasets 2.21.0 - Tokenizers 0.19.1