File size: 6,657 Bytes
16895da 7affa90 16895da 7affa90 16895da c655b34 16895da 7affa90 16895da 7affa90 16895da 7affa90 16895da |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 |
---
license: cc-by-sa-4.0
base_model: kiddothe2b/longformer-base-4096
tags:
- generated_from_trainer
metrics:
- f1
model-index:
- name: longformer-base-4096-airlines-news-multi-label
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# longformer-base-4096-airlines-news-multi-label
This model is a fine-tuned version of [kiddothe2b/longformer-base-4096](https://huggingface.co./kiddothe2b/longformer-base-4096) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.2421
- F1: 0.9070
- Roc Auc: 0.6668
- Hamming: 0.9137
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 9e-05
- train_batch_size: 32
- eval_batch_size: 32
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 65
### Training results
| Training Loss | Epoch | Step | Validation Loss | F1 | Roc Auc | Hamming |
|:-------------:|:-----:|:----:|:---------------:|:------:|:-------:|:-------:|
| No log | 1.0 | 57 | 0.3454 | 0.8319 | 0.5 | 0.8850 |
| No log | 2.0 | 114 | 0.3372 | 0.8319 | 0.5 | 0.8850 |
| No log | 3.0 | 171 | 0.3353 | 0.8319 | 0.5 | 0.8850 |
| No log | 4.0 | 228 | 0.3310 | 0.8319 | 0.5 | 0.8850 |
| No log | 5.0 | 285 | 0.3278 | 0.8319 | 0.5 | 0.8850 |
| No log | 6.0 | 342 | 0.3242 | 0.8319 | 0.5 | 0.8850 |
| No log | 7.0 | 399 | 0.3206 | 0.8319 | 0.5 | 0.8850 |
| No log | 8.0 | 456 | 0.3168 | 0.8319 | 0.5 | 0.8850 |
| 0.3599 | 9.0 | 513 | 0.3120 | 0.8319 | 0.5 | 0.8850 |
| 0.3599 | 10.0 | 570 | 0.3089 | 0.8319 | 0.5 | 0.8850 |
| 0.3599 | 11.0 | 627 | 0.3039 | 0.8319 | 0.5 | 0.8850 |
| 0.3599 | 12.0 | 684 | 0.3000 | 0.8319 | 0.5 | 0.8850 |
| 0.3599 | 13.0 | 741 | 0.2969 | 0.8319 | 0.5 | 0.8850 |
| 0.3599 | 14.0 | 798 | 0.2932 | 0.8319 | 0.5 | 0.8850 |
| 0.3599 | 15.0 | 855 | 0.2893 | 0.8449 | 0.5064 | 0.8864 |
| 0.3599 | 16.0 | 912 | 0.2859 | 0.8449 | 0.5064 | 0.8864 |
| 0.3599 | 17.0 | 969 | 0.2824 | 0.8449 | 0.5064 | 0.8864 |
| 0.3111 | 18.0 | 1026 | 0.2800 | 0.8613 | 0.5192 | 0.8894 |
| 0.3111 | 19.0 | 1083 | 0.2773 | 0.8606 | 0.5160 | 0.8886 |
| 0.3111 | 20.0 | 1140 | 0.2752 | 0.8586 | 0.5248 | 0.8894 |
| 0.3111 | 21.0 | 1197 | 0.2727 | 0.8586 | 0.5248 | 0.8894 |
| 0.3111 | 22.0 | 1254 | 0.2703 | 0.8597 | 0.5280 | 0.8901 |
| 0.3111 | 23.0 | 1311 | 0.2679 | 0.8761 | 0.5532 | 0.8953 |
| 0.3111 | 24.0 | 1368 | 0.2665 | 0.8783 | 0.5684 | 0.8975 |
| 0.3111 | 25.0 | 1425 | 0.2645 | 0.8791 | 0.5688 | 0.8982 |
| 0.3111 | 26.0 | 1482 | 0.2627 | 0.8789 | 0.5776 | 0.8990 |
| 0.2854 | 27.0 | 1539 | 0.2611 | 0.8780 | 0.5716 | 0.8982 |
| 0.2854 | 28.0 | 1596 | 0.2597 | 0.8791 | 0.5688 | 0.8982 |
| 0.2854 | 29.0 | 1653 | 0.2584 | 0.8818 | 0.5845 | 0.9012 |
| 0.2854 | 30.0 | 1710 | 0.2570 | 0.8825 | 0.5877 | 0.9019 |
| 0.2854 | 31.0 | 1767 | 0.2564 | 0.8930 | 0.6405 | 0.9115 |
| 0.2854 | 32.0 | 1824 | 0.2556 | 0.8913 | 0.6396 | 0.9100 |
| 0.2854 | 33.0 | 1881 | 0.2547 | 0.8870 | 0.6296 | 0.9071 |
| 0.2854 | 34.0 | 1938 | 0.2531 | 0.8843 | 0.6029 | 0.9041 |
| 0.2854 | 35.0 | 1995 | 0.2522 | 0.8912 | 0.6341 | 0.9100 |
| 0.2722 | 36.0 | 2052 | 0.2516 | 0.8914 | 0.6341 | 0.9100 |
| 0.2722 | 37.0 | 2109 | 0.2507 | 0.8913 | 0.6369 | 0.9100 |
| 0.2722 | 38.0 | 2166 | 0.2501 | 0.8899 | 0.6392 | 0.9093 |
| 0.2722 | 39.0 | 2223 | 0.2491 | 0.8865 | 0.6264 | 0.9063 |
| 0.2722 | 40.0 | 2280 | 0.2486 | 0.8939 | 0.6409 | 0.9122 |
| 0.2722 | 41.0 | 2337 | 0.2483 | 0.8921 | 0.6516 | 0.9115 |
| 0.2722 | 42.0 | 2394 | 0.2474 | 0.8913 | 0.6512 | 0.9108 |
| 0.2722 | 43.0 | 2451 | 0.2466 | 0.8911 | 0.6341 | 0.9100 |
| 0.2652 | 44.0 | 2508 | 0.2461 | 0.8950 | 0.6557 | 0.9137 |
| 0.2652 | 45.0 | 2565 | 0.2459 | 0.8913 | 0.6540 | 0.9108 |
| 0.2652 | 46.0 | 2622 | 0.2453 | 0.8934 | 0.6521 | 0.9122 |
| 0.2652 | 47.0 | 2679 | 0.2446 | 0.8950 | 0.6557 | 0.9137 |
| 0.2652 | 48.0 | 2736 | 0.2445 | 0.8922 | 0.6572 | 0.9115 |
| 0.2652 | 49.0 | 2793 | 0.2442 | 0.8931 | 0.6521 | 0.9122 |
| 0.2652 | 50.0 | 2850 | 0.2440 | 0.8938 | 0.6608 | 0.9130 |
| 0.2652 | 51.0 | 2907 | 0.2436 | 0.8930 | 0.6576 | 0.9122 |
| 0.2652 | 52.0 | 2964 | 0.2432 | 0.8940 | 0.6553 | 0.9130 |
| 0.2603 | 53.0 | 3021 | 0.2430 | 0.8940 | 0.6553 | 0.9130 |
| 0.2603 | 54.0 | 3078 | 0.2428 | 0.8930 | 0.6576 | 0.9122 |
| 0.2603 | 55.0 | 3135 | 0.2425 | 0.8938 | 0.6608 | 0.9130 |
| 0.2603 | 56.0 | 3192 | 0.2424 | 0.8904 | 0.6480 | 0.9100 |
| 0.2603 | 57.0 | 3249 | 0.2424 | 0.8938 | 0.6636 | 0.9130 |
| 0.2603 | 58.0 | 3306 | 0.2422 | 0.8938 | 0.6636 | 0.9130 |
| 0.2603 | 59.0 | 3363 | 0.2421 | 0.9070 | 0.6668 | 0.9137 |
| 0.2603 | 60.0 | 3420 | 0.2419 | 0.9070 | 0.6668 | 0.9137 |
| 0.2603 | 61.0 | 3477 | 0.2418 | 0.8938 | 0.6636 | 0.9130 |
| 0.2578 | 62.0 | 3534 | 0.2418 | 0.8938 | 0.6636 | 0.9130 |
| 0.2578 | 63.0 | 3591 | 0.2416 | 0.8930 | 0.6576 | 0.9122 |
| 0.2578 | 64.0 | 3648 | 0.2416 | 0.8938 | 0.6608 | 0.9130 |
| 0.2578 | 65.0 | 3705 | 0.2416 | 0.8930 | 0.6576 | 0.9122 |
### Framework versions
- Transformers 4.41.1
- Pytorch 2.3.0+cu121
- Datasets 2.19.1
- Tokenizers 0.19.1
|