Hugging Face's logo

Theoreticallyhugo
/
longformer-sep_tok_full_labels

Token Classification
Transformers
Safetensors
longformer
Generated from Trainer
Eval Results
Inference Endpoints
Model card Files Community
Theoreticallyhugo commited on
Commit
c2bd144
•
1 Parent(s): 5cca7b7

trainer: training complete at 2024-10-26 21:12:51.522480.

Browse files
Files changed (4) hide show
  1. README.md +29 -43
  2. meta_data/README_s42_e5.md +30 -28
  3. meta_data/meta_s42_e5_cvi0.json +1 -1
  4. model.safetensors +1 -1
README.md CHANGED
@@ -1,10 +1,11 @@
1
  ---
 
2
  license: apache-2.0
3
  base_model: allenai/longformer-base-4096
4
  tags:
5
  - generated_from_trainer
6
  datasets:
7
- - essays_su_g
8
  metrics:
9
  - accuracy
10
  model-index:
@@ -14,15 +15,15 @@ model-index:
14
  name: Token Classification
15
  type: token-classification
16
  dataset:
17
- name: essays_su_g
18
- type: essays_su_g
19
  config: sep_tok_full_labels
20
  split: train[0%:20%]
21
  args: sep_tok_full_labels
22
  metrics:
23
  - name: Accuracy
24
  type: accuracy
25
- value: 0.9077215189873418
26
  ---
27
 
28
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -30,19 +31,19 @@ should probably proofread and complete it, then remove this comment. -->
30
 
31
  # longformer-sep_tok_full_labels
32
 
33
- This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
34
  It achieves the following results on the evaluation set:
35
- - Loss: 0.6997
36
- - B-claim: {'precision': 0.6735395189003437, 'recall': 0.6901408450704225, 'f1-score': 0.6817391304347826, 'support': 284.0}
37
- - B-majorclaim: {'precision': 0.9457364341085271, 'recall': 0.8652482269503546, 'f1-score': 0.9037037037037037, 'support': 141.0}
38
- - B-premise: {'precision': 0.8835904628330996, 'recall': 0.8898305084745762, 'f1-score': 0.8866995073891626, 'support': 708.0}
39
- - I-claim: {'precision': 0.6694601922602909, 'recall': 0.6661761098847192, 'f1-score': 0.6678141135972462, 'support': 4077.0}
40
- - I-majorclaim: {'precision': 0.9496166484118291, 'recall': 0.8567193675889329, 'f1-score': 0.9007792207792208, 'support': 2024.0}
41
- - I-premise: {'precision': 0.896984318455971, 'recall': 0.9118705035971223, 'f1-score': 0.9043661572140916, 'support': 12232.0}
42
- - O: {'precision': 0.9986007078771916, 'recall': 0.9998351738915444, 'f1-score': 0.9992175596096035, 'support': 12134.0}
43
- - Accuracy: 0.9077
44
- - Macro avg: {'precision': 0.8596468975496075, 'recall': 0.8399743907796676, 'f1-score': 0.8491884846754016, 'support': 31600.0}
45
- - Weighted avg: {'precision': 0.9079291956085084, 'recall': 0.9077215189873418, 'f1-score': 0.9076388409441786, 'support': 31600.0}
46
 
47
  ## Model description
48
 
@@ -67,37 +68,22 @@ The following hyperparameters were used during training:
67
  - seed: 42
68
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
69
  - lr_scheduler_type: linear
70
- - num_epochs: 20
71
 
72
  ### Training results
73
 
74
- | Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg |
75
- |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
76
- | No log | 1.0 | 81 | 0.3267 | {'precision': 0.3333333333333333, 'recall': 0.1619718309859155, 'f1-score': 0.21800947867298578, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.71900826446281, 'recall': 0.9830508474576272, 'f1-score': 0.8305489260143198, 'support': 708.0} | {'precision': 0.5643589102724319, 'recall': 0.5538386068187393, 'f1-score': 0.5590492696211935, 'support': 4077.0} | {'precision': 0.7205056179775281, 'recall': 0.7603754940711462, 'f1-score': 0.7399038461538461, 'support': 2024.0} | {'precision': 0.8883965205974068, 'recall': 0.8850555918901243, 'f1-score': 0.8867229093291834, 'support': 12232.0} | {'precision': 0.9961383616794018, 'recall': 0.9991758694577221, 'f1-score': 0.9976548035383666, 'support': 12134.0} | 0.8699 | {'precision': 0.6031058583318446, 'recall': 0.6204954629544678, 'f1-score': 0.6045556047614136, 'support': 31600.0} | {'precision': 0.8644597558999653, 'recall': 0.8699050632911393, 'f1-score': 0.8664142595402331, 'support': 31600.0} |
77
- | No log | 2.0 | 162 | 0.2872 | {'precision': 0.6515151515151515, 'recall': 0.45422535211267606, 'f1-score': 0.5352697095435685, 'support': 284.0} | {'precision': 0.7142857142857143, 'recall': 0.851063829787234, 'f1-score': 0.7766990291262136, 'support': 141.0} | {'precision': 0.8490813648293963, 'recall': 0.9138418079096046, 'f1-score': 0.8802721088435375, 'support': 708.0} | {'precision': 0.6325869180907484, 'recall': 0.5266127054206524, 'f1-score': 0.5747557221255521, 'support': 4077.0} | {'precision': 0.6986831913245546, 'recall': 0.8913043478260869, 'f1-score': 0.7833260963960051, 'support': 2024.0} | {'precision': 0.8971377459749553, 'recall': 0.9019784172661871, 'f1-score': 0.8995515695067264, 'support': 12232.0} | {'precision': 0.9945072962780783, 'recall': 0.9997527608373167, 'f1-score': 0.9971231300345225, 'support': 12134.0} | 0.8864 | {'precision': 0.7768281974712284, 'recall': 0.791254174451394, 'f1-score': 0.7781424807965893, 'support': 31600.0} | {'precision': 0.8835831101628018, 'recall': 0.8864240506329114, 'f1-score': 0.8834146129726252, 'support': 31600.0} |
78
- | No log | 3.0 | 243 | 0.2827 | {'precision': 0.6618357487922706, 'recall': 0.4823943661971831, 'f1-score': 0.5580448065173116, 'support': 284.0} | {'precision': 0.8956521739130435, 'recall': 0.7304964539007093, 'f1-score': 0.8046875, 'support': 141.0} | {'precision': 0.8300492610837439, 'recall': 0.9519774011299436, 'f1-score': 0.8868421052631579, 'support': 708.0} | {'precision': 0.6751432664756447, 'recall': 0.4623497669855286, 'f1-score': 0.5488426262920367, 'support': 4077.0} | {'precision': 0.8620689655172413, 'recall': 0.7658102766798419, 'f1-score': 0.8110936682365254, 'support': 2024.0} | {'precision': 0.854842235662756, 'recall': 0.9590418574231524, 'f1-score': 0.9039491427470623, 'support': 12232.0} | {'precision': 0.9972023368715544, 'recall': 0.9987638041865832, 'f1-score': 0.9979824597521307, 'support': 12134.0} | 0.8924 | {'precision': 0.825256284045179, 'recall': 0.7644048466432775, 'f1-score': 0.7873489012583177, 'support': 31600.0} | {'precision': 0.8846770016417851, 'recall': 0.892373417721519, 'f1-score': 0.88435885840807, 'support': 31600.0} |
79
- | No log | 4.0 | 324 | 0.2764 | {'precision': 0.6165644171779141, 'recall': 0.7077464788732394, 'f1-score': 0.659016393442623, 'support': 284.0} | {'precision': 0.8136645962732919, 'recall': 0.9290780141843972, 'f1-score': 0.8675496688741722, 'support': 141.0} | {'precision': 0.9130434782608695, 'recall': 0.8305084745762712, 'f1-score': 0.8698224852071005, 'support': 708.0} | {'precision': 0.5964839710444674, 'recall': 0.70738287956831, 'f1-score': 0.6472172351885099, 'support': 4077.0} | {'precision': 0.8131966116807846, 'recall': 0.9011857707509882, 'f1-score': 0.8549332083430982, 'support': 2024.0} | {'precision': 0.9271658801531475, 'recall': 0.8512916939175932, 'f1-score': 0.8876102800153433, 'support': 12232.0} | {'precision': 0.9970394736842105, 'recall': 0.9991758694577221, 'f1-score': 0.9981065283609122, 'support': 12134.0} | 0.8913 | {'precision': 0.8110226326106693, 'recall': 0.8466241687612174, 'f1-score': 0.8263222570616798, 'support': 31600.0} | {'precision': 0.9004180663566287, 'recall': 0.8912974683544304, 'f1-score': 0.8943886873545748, 'support': 31600.0} |
80
- | No log | 5.0 | 405 | 0.2893 | {'precision': 0.6763636363636364, 'recall': 0.6549295774647887, 'f1-score': 0.665474060822898, 'support': 284.0} | {'precision': 0.8657718120805369, 'recall': 0.9148936170212766, 'f1-score': 0.8896551724137931, 'support': 141.0} | {'precision': 0.8830985915492958, 'recall': 0.885593220338983, 'f1-score': 0.8843441466854725, 'support': 708.0} | {'precision': 0.6597668332143707, 'recall': 0.6801569781702232, 'f1-score': 0.6698067632850242, 'support': 4077.0} | {'precision': 0.8751828376401756, 'recall': 0.8868577075098815, 'f1-score': 0.8809815950920246, 'support': 2024.0} | {'precision': 0.9094305163539764, 'recall': 0.8956017004578156, 'f1-score': 0.9024631353488756, 'support': 12232.0} | {'precision': 0.9967121486108828, 'recall': 0.9993406955661777, 'f1-score': 0.9980246913580246, 'support': 12134.0} | 0.9048 | {'precision': 0.838046625116125, 'recall': 0.8453390709327352, 'f1-score': 0.8415356521437304, 'support': 31600.0} | {'precision': 0.9056611908459661, 'recall': 0.9047784810126582, 'f1-score': 0.9051715590931133, 'support': 31600.0} |
81
- | No log | 6.0 | 486 | 0.3353 | {'precision': 0.6330935251798561, 'recall': 0.6197183098591549, 'f1-score': 0.6263345195729537, 'support': 284.0} | {'precision': 0.9230769230769231, 'recall': 0.7659574468085106, 'f1-score': 0.8372093023255814, 'support': 141.0} | {'precision': 0.8594594594594595, 'recall': 0.8983050847457628, 'f1-score': 0.8784530386740332, 'support': 708.0} | {'precision': 0.6360499070878683, 'recall': 0.5876870247731175, 'f1-score': 0.6109127995920448, 'support': 4077.0} | {'precision': 0.9126662810873337, 'recall': 0.7796442687747036, 'f1-score': 0.8409272581934453, 'support': 2024.0} | {'precision': 0.8762685402029664, 'recall': 0.9176749509483323, 'f1-score': 0.8964938902643559, 'support': 12232.0} | {'precision': 0.9976971790443293, 'recall': 0.9997527608373167, 'f1-score': 0.9987239122380934, 'support': 12134.0} | 0.8940 | {'precision': 0.8340445450198194, 'recall': 0.7955342638209855, 'f1-score': 0.8127221029800725, 'support': 31600.0} | {'precision': 0.891880888702747, 'recall': 0.8939873417721519, 'f1-score': 0.8922477132246446, 'support': 31600.0} |
82
- | 0.2608 | 7.0 | 567 | 0.4514 | {'precision': 0.6820512820512821, 'recall': 0.46830985915492956, 'f1-score': 0.5553235908141962, 'support': 284.0} | {'precision': 0.954954954954955, 'recall': 0.75177304964539, 'f1-score': 0.8412698412698413, 'support': 141.0} | {'precision': 0.8210399032648126, 'recall': 0.9590395480225988, 'f1-score': 0.8846905537459283, 'support': 708.0} | {'precision': 0.6880088823094005, 'recall': 0.45597252882021094, 'f1-score': 0.5484584747012834, 'support': 4077.0} | {'precision': 0.9536019536019537, 'recall': 0.7717391304347826, 'f1-score': 0.8530857454942655, 'support': 2024.0} | {'precision': 0.8438103411285132, 'recall': 0.9646010464355788, 'f1-score': 0.900171657448026, 'support': 12232.0} | {'precision': 0.9982707509881423, 'recall': 0.9990934564034943, 'f1-score': 0.9986819342614713, 'support': 12134.0} | 0.8943 | {'precision': 0.8488197240427228, 'recall': 0.7672183741309979, 'f1-score': 0.7973831139621446, 'support': 31600.0} | {'precision': 0.8885840321741321, 'recall': 0.8943354430379746, 'f1-score': 0.8858958517067363, 'support': 31600.0} |
83
- | 0.2608 | 8.0 | 648 | 0.4097 | {'precision': 0.6814814814814815, 'recall': 0.647887323943662, 'f1-score': 0.6642599277978338, 'support': 284.0} | {'precision': 0.967479674796748, 'recall': 0.8439716312056738, 'f1-score': 0.9015151515151516, 'support': 141.0} | {'precision': 0.8687415426251691, 'recall': 0.9067796610169492, 'f1-score': 0.8873531444367657, 'support': 708.0} | {'precision': 0.6823907799517556, 'recall': 0.6244787834191807, 'f1-score': 0.6521516393442622, 'support': 4077.0} | {'precision': 0.9711649365628604, 'recall': 0.8320158102766798, 'f1-score': 0.8962213943587014, 'support': 2024.0} | {'precision': 0.8831269952503309, 'recall': 0.9272400261608895, 'f1-score': 0.9046460618145563, 'support': 12232.0} | {'precision': 0.996875, 'recall': 0.9990110433492665, 'f1-score': 0.9979418786531653, 'support': 12134.0} | 0.9063 | {'precision': 0.8644657729526208, 'recall': 0.8259120399103289, 'f1-score': 0.8434413139886338, 'support': 31600.0} | {'precision': 0.9047867115327305, 'recall': 0.9062974683544304, 'f1-score': 0.9047924430886448, 'support': 31600.0} |
84
- | 0.2608 | 9.0 | 729 | 0.4417 | {'precision': 0.686411149825784, 'recall': 0.6936619718309859, 'f1-score': 0.6900175131348512, 'support': 284.0} | {'precision': 0.9197080291970803, 'recall': 0.8936170212765957, 'f1-score': 0.9064748201438848, 'support': 141.0} | {'precision': 0.8885754583921015, 'recall': 0.8898305084745762, 'f1-score': 0.8892025405786873, 'support': 708.0} | {'precision': 0.6771096513390601, 'recall': 0.657346087809664, 'f1-score': 0.6670815183571872, 'support': 4077.0} | {'precision': 0.9159268929503916, 'recall': 0.866600790513834, 'f1-score': 0.8905813658288906, 'support': 2024.0} | {'precision': 0.8978524893428779, 'recall': 0.9126062786134729, 'f1-score': 0.9051692681937971, 'support': 12232.0} | {'precision': 0.9976153276868679, 'recall': 0.9998351738915444, 'f1-score': 0.9987240172875077, 'support': 12134.0} | 0.9077 | {'precision': 0.8547427141048803, 'recall': 0.8447854046300962, 'f1-score': 0.849607291932115, 'support': 31600.0} | {'precision': 0.9068271879381139, 'recall': 0.9076582278481012, 'f1-score': 0.9071553503542205, 'support': 31600.0} |
85
- | 0.2608 | 10.0 | 810 | 0.4593 | {'precision': 0.6834532374100719, 'recall': 0.6690140845070423, 'f1-score': 0.6761565836298933, 'support': 284.0} | {'precision': 0.9104477611940298, 'recall': 0.8652482269503546, 'f1-score': 0.8872727272727273, 'support': 141.0} | {'precision': 0.8821081830790569, 'recall': 0.8983050847457628, 'f1-score': 0.8901329601119664, 'support': 708.0} | {'precision': 0.6705637828007275, 'recall': 0.6330635271032622, 'f1-score': 0.6512742871561948, 'support': 4077.0} | {'precision': 0.9037947621592731, 'recall': 0.8354743083003953, 'f1-score': 0.8682926829268293, 'support': 2024.0} | {'precision': 0.8933428775948461, 'recall': 0.9182472204054938, 'f1-score': 0.905623866156017, 'support': 12232.0} | {'precision': 0.9965500246426812, 'recall': 0.9998351738915444, 'f1-score': 0.9981898963304262, 'support': 12134.0} | 0.9046 | {'precision': 0.8486086612686695, 'recall': 0.8313125179862652, 'f1-score': 0.8395632862262934, 'support': 31600.0} | {'precision': 0.9028380907030437, 'recall': 0.9045569620253164, 'f1-score': 0.9034697801243392, 'support': 31600.0} |
86
- | 0.2608 | 11.0 | 891 | 0.5346 | {'precision': 0.7142857142857143, 'recall': 0.5809859154929577, 'f1-score': 0.6407766990291263, 'support': 284.0} | {'precision': 0.9130434782608695, 'recall': 0.8936170212765957, 'f1-score': 0.9032258064516129, 'support': 141.0} | {'precision': 0.8547120418848168, 'recall': 0.922316384180791, 'f1-score': 0.8872282608695653, 'support': 708.0} | {'precision': 0.7111412123264477, 'recall': 0.515084621044886, 'f1-score': 0.5974395448079659, 'support': 4077.0} | {'precision': 0.8927318295739348, 'recall': 0.8799407114624506, 'f1-score': 0.8862901219208758, 'support': 2024.0} | {'precision': 0.862670258943272, 'recall': 0.9423642903858731, 'f1-score': 0.9007579901539423, 'support': 12232.0} | {'precision': 0.9980258287406433, 'recall': 0.9999175869457723, 'f1-score': 0.9989708122349842, 'support': 12134.0} | 0.9014 | {'precision': 0.8495157662879569, 'recall': 0.8191752186841895, 'f1-score': 0.8306698907811532, 'support': 31600.0} | {'precision': 0.8957333024681017, 'recall': 0.9014240506329114, 'f1-score': 0.8957812921551218, 'support': 31600.0} |
87
- | 0.2608 | 12.0 | 972 | 0.6067 | {'precision': 0.6622950819672131, 'recall': 0.7112676056338029, 'f1-score': 0.6859083191850596, 'support': 284.0} | {'precision': 0.9375, 'recall': 0.851063829787234, 'f1-score': 0.8921933085501859, 'support': 141.0} | {'precision': 0.8914285714285715, 'recall': 0.8813559322033898, 'f1-score': 0.8863636363636365, 'support': 708.0} | {'precision': 0.6508319266939957, 'recall': 0.6620063772381654, 'f1-score': 0.6563715953307394, 'support': 4077.0} | {'precision': 0.9370034052213394, 'recall': 0.8157114624505929, 'f1-score': 0.8721605916534602, 'support': 2024.0} | {'precision': 0.8954670108081949, 'recall': 0.907619359058208, 'f1-score': 0.9015022330491272, 'support': 12232.0} | {'precision': 0.9977796052631579, 'recall': 0.9999175869457723, 'f1-score': 0.9988474520457726, 'support': 12134.0} | 0.9029 | {'precision': 0.8531865144832105, 'recall': 0.8327060219024521, 'f1-score': 0.8419067337397116, 'support': 31600.0} | {'precision': 0.9038530884689406, 'recall': 0.902879746835443, 'f1-score': 0.9030573735174033, 'support': 31600.0} |
88
- | 0.0322 | 13.0 | 1053 | 0.5914 | {'precision': 0.7125984251968503, 'recall': 0.6373239436619719, 'f1-score': 0.6728624535315986, 'support': 284.0} | {'precision': 0.9090909090909091, 'recall': 0.9219858156028369, 'f1-score': 0.9154929577464789, 'support': 141.0} | {'precision': 0.873641304347826, 'recall': 0.9081920903954802, 'f1-score': 0.8905817174515235, 'support': 708.0} | {'precision': 0.6932808546527973, 'recall': 0.6048565121412803, 'f1-score': 0.6460571129159025, 'support': 4077.0} | {'precision': 0.905688622754491, 'recall': 0.8967391304347826, 'f1-score': 0.9011916583912611, 'support': 2024.0} | {'precision': 0.8856001879257693, 'recall': 0.9246239372138653, 'f1-score': 0.9046914370275567, 'support': 12232.0} | {'precision': 0.9990935311083642, 'recall': 0.9991758694577221, 'f1-score': 0.9991346985866744, 'support': 12134.0} | 0.9072 | {'precision': 0.8541419764395725, 'recall': 0.8418424712725627, 'f1-score': 0.8471445765215709, 'support': 31600.0} | {'precision': 0.9039360771033997, 'recall': 0.907246835443038, 'f1-score': 0.9050120935479347, 'support': 31600.0} |
89
- | 0.0322 | 14.0 | 1134 | 0.6610 | {'precision': 0.7283950617283951, 'recall': 0.6232394366197183, 'f1-score': 0.6717267552182163, 'support': 284.0} | {'precision': 0.927536231884058, 'recall': 0.9078014184397163, 'f1-score': 0.9175627240143368, 'support': 141.0} | {'precision': 0.8656914893617021, 'recall': 0.9194915254237288, 'f1-score': 0.8917808219178082, 'support': 708.0} | {'precision': 0.7092882991556092, 'recall': 0.5768947755702722, 'f1-score': 0.6362775598539159, 'support': 4077.0} | {'precision': 0.9176531137416366, 'recall': 0.8809288537549407, 'f1-score': 0.8989160574741618, 'support': 2024.0} | {'precision': 0.8762918165811835, 'recall': 0.9358240680183126, 'f1-score': 0.9050800553469064, 'support': 12232.0} | {'precision': 0.9989296006587073, 'recall': 0.9998351738915444, 'f1-score': 0.9993821821327072, 'support': 12134.0} | 0.9073 | {'precision': 0.8605408018730417, 'recall': 0.8348593216740333, 'f1-score': 0.8458180222797218, 'support': 31600.0} | {'precision': 0.9031477200436355, 'recall': 0.9072784810126582, 'f1-score': 0.9038759465613782, 'support': 31600.0} |
90
- | 0.0322 | 15.0 | 1215 | 0.6557 | {'precision': 0.6868686868686869, 'recall': 0.7183098591549296, 'f1-score': 0.70223752151463, 'support': 284.0} | {'precision': 0.9197080291970803, 'recall': 0.8936170212765957, 'f1-score': 0.9064748201438848, 'support': 141.0} | {'precision': 0.8955650929899857, 'recall': 0.884180790960452, 'f1-score': 0.8898365316275765, 'support': 708.0} | {'precision': 0.6798825256975036, 'recall': 0.681383370125092, 'f1-score': 0.6806321205439176, 'support': 4077.0} | {'precision': 0.9241952232606438, 'recall': 0.8794466403162056, 'f1-score': 0.9012658227848102, 'support': 2024.0} | {'precision': 0.9036790384146837, 'recall': 0.9096631785480707, 'f1-score': 0.906661234467305, 'support': 12232.0} | {'precision': 0.9991764124526438, 'recall': 0.9998351738915444, 'f1-score': 0.9995056846267919, 'support': 12134.0} | 0.9105 | {'precision': 0.8584392869830325, 'recall': 0.8523480048961273, 'f1-score': 0.8552305336727023, 'support': 31600.0} | {'precision': 0.910730075973464, 'recall': 0.9105379746835442, 'f1-score': 0.9105896850690615, 'support': 31600.0} |
91
- | 0.0322 | 16.0 | 1296 | 0.6791 | {'precision': 0.6896551724137931, 'recall': 0.6338028169014085, 'f1-score': 0.6605504587155964, 'support': 284.0} | {'precision': 0.9453125, 'recall': 0.8581560283687943, 'f1-score': 0.8996282527881041, 'support': 141.0} | {'precision': 0.8669354838709677, 'recall': 0.9110169491525424, 'f1-score': 0.8884297520661157, 'support': 708.0} | {'precision': 0.6769836803601575, 'recall': 0.5901398086828551, 'f1-score': 0.6305857685755472, 'support': 4077.0} | {'precision': 0.9421308815575987, 'recall': 0.8606719367588933, 'f1-score': 0.8995610637748515, 'support': 2024.0} | {'precision': 0.8771861940876026, 'recall': 0.9266677567037279, 'f1-score': 0.9012483104078874, 'support': 12232.0} | {'precision': 0.9991764124526438, 'recall': 0.9998351738915444, 'f1-score': 0.9995056846267919, 'support': 12134.0} | 0.9038 | {'precision': 0.8567686178203948, 'recall': 0.8257557814942523, 'f1-score': 0.8399298987078421, 'support': 31600.0} | {'precision': 0.9007476246179443, 'recall': 0.9038291139240506, 'f1-score': 0.9014915588643904, 'support': 31600.0} |
92
- | 0.0322 | 17.0 | 1377 | 0.6997 | {'precision': 0.6735395189003437, 'recall': 0.6901408450704225, 'f1-score': 0.6817391304347826, 'support': 284.0} | {'precision': 0.952755905511811, 'recall': 0.8581560283687943, 'f1-score': 0.9029850746268657, 'support': 141.0} | {'precision': 0.8839160839160839, 'recall': 0.8926553672316384, 'f1-score': 0.8882642304989459, 'support': 708.0} | {'precision': 0.6603307825228338, 'recall': 0.6561196958547952, 'f1-score': 0.6582185039370079, 'support': 4077.0} | {'precision': 0.946725860155383, 'recall': 0.8428853754940712, 'f1-score': 0.8917929952953477, 'support': 2024.0} | {'precision': 0.8941922027915932, 'recall': 0.9112982341399608, 'f1-score': 0.902664183334683, 'support': 12232.0} | {'precision': 0.9986829107672045, 'recall': 0.9998351738915444, 'f1-score': 0.9992587101556709, 'support': 12134.0} | 0.9053 | {'precision': 0.8585918949378932, 'recall': 0.8358701028644611, 'f1-score': 0.8464175468976148, 'support': 31600.0} | {'precision': 0.905555556916252, 'recall': 0.9053481012658228, 'f1-score': 0.9052140894419886, 'support': 31600.0} |
93
- | 0.0322 | 18.0 | 1458 | 0.7011 | {'precision': 0.6818181818181818, 'recall': 0.6866197183098591, 'f1-score': 0.6842105263157894, 'support': 284.0} | {'precision': 0.946969696969697, 'recall': 0.8865248226950354, 'f1-score': 0.9157509157509157, 'support': 141.0} | {'precision': 0.8825174825174825, 'recall': 0.8912429378531074, 'f1-score': 0.8868587491215741, 'support': 708.0} | {'precision': 0.6766766766766766, 'recall': 0.6632327691930341, 'f1-score': 0.6698872785829307, 'support': 4077.0} | {'precision': 0.9471698113207547, 'recall': 0.8680830039525692, 'f1-score': 0.9059035833977829, 'support': 2024.0} | {'precision': 0.8964162591196986, 'recall': 0.9140778286461739, 'f1-score': 0.9051608986035214, 'support': 12232.0} | {'precision': 0.9990941283043729, 'recall': 0.9998351738915444, 'f1-score': 0.9994645137372822, 'support': 12134.0} | 0.9090 | {'precision': 0.8615231766752662, 'recall': 0.8442308935059034, 'f1-score': 0.8524623522156853, 'support': 31600.0} | {'precision': 0.90872898138775, 'recall': 0.9090189873417721, 'f1-score': 0.9087165339227473, 'support': 31600.0} |
94
- | 0.0055 | 19.0 | 1539 | 0.7171 | {'precision': 0.6834532374100719, 'recall': 0.6690140845070423, 'f1-score': 0.6761565836298933, 'support': 284.0} | {'precision': 0.9586776859504132, 'recall': 0.8226950354609929, 'f1-score': 0.8854961832061068, 'support': 141.0} | {'precision': 0.8760217983651226, 'recall': 0.9081920903954802, 'f1-score': 0.8918169209431346, 'support': 708.0} | {'precision': 0.6699947451392538, 'recall': 0.6254598969830758, 'f1-score': 0.6469618165672967, 'support': 4077.0} | {'precision': 0.9521889400921659, 'recall': 0.816699604743083, 'f1-score': 0.8792553191489363, 'support': 2024.0} | {'precision': 0.8853498200031303, 'recall': 0.9248691955526488, 'f1-score': 0.9046781287485005, 'support': 12232.0} | {'precision': 0.9987651271918992, 'recall': 0.9998351738915444, 'f1-score': 0.9992998640912648, 'support': 12134.0} | 0.9050 | {'precision': 0.8606359077360082, 'recall': 0.8238235830762669, 'f1-score': 0.8405235451907334, 'support': 31600.0} | {'precision': 0.9036997388825967, 'recall': 0.9049683544303797, 'f1-score': 0.9037054849825339, 'support': 31600.0} |
95
- | 0.0055 | 20.0 | 1620 | 0.6997 | {'precision': 0.6735395189003437, 'recall': 0.6901408450704225, 'f1-score': 0.6817391304347826, 'support': 284.0} | {'precision': 0.9457364341085271, 'recall': 0.8652482269503546, 'f1-score': 0.9037037037037037, 'support': 141.0} | {'precision': 0.8835904628330996, 'recall': 0.8898305084745762, 'f1-score': 0.8866995073891626, 'support': 708.0} | {'precision': 0.6694601922602909, 'recall': 0.6661761098847192, 'f1-score': 0.6678141135972462, 'support': 4077.0} | {'precision': 0.9496166484118291, 'recall': 0.8567193675889329, 'f1-score': 0.9007792207792208, 'support': 2024.0} | {'precision': 0.896984318455971, 'recall': 0.9118705035971223, 'f1-score': 0.9043661572140916, 'support': 12232.0} | {'precision': 0.9986007078771916, 'recall': 0.9998351738915444, 'f1-score': 0.9992175596096035, 'support': 12134.0} | 0.9077 | {'precision': 0.8596468975496075, 'recall': 0.8399743907796676, 'f1-score': 0.8491884846754016, 'support': 31600.0} | {'precision': 0.9079291956085084, 'recall': 0.9077215189873418, 'f1-score': 0.9076388409441786, 'support': 31600.0} |
96
 
97
 
98
  ### Framework versions
99
 
100
- - Transformers 4.38.2
101
- - Pytorch 2.2.1+cu121
102
- - Datasets 2.18.0
103
- - Tokenizers 0.15.2
 
1
  ---
2
+ library_name: transformers
3
  license: apache-2.0
4
  base_model: allenai/longformer-base-4096
5
  tags:
6
  - generated_from_trainer
7
  datasets:
8
+ - stab-gurevych-essays
9
  metrics:
10
  - accuracy
11
  model-index:
 
15
  name: Token Classification
16
  type: token-classification
17
  dataset:
18
+ name: stab-gurevych-essays
19
+ type: stab-gurevych-essays
20
  config: sep_tok_full_labels
21
  split: train[0%:20%]
22
  args: sep_tok_full_labels
23
  metrics:
24
  - name: Accuracy
25
  type: accuracy
26
+ value: 0.8874031749771744
27
  ---
28
 
29
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
31
 
32
  # longformer-sep_tok_full_labels
33
 
34
+ This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the stab-gurevych-essays dataset.
35
  It achieves the following results on the evaluation set:
36
+ - Loss: 0.2775
37
+ - B-claim: {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0}
38
+ - B-majorclaim: {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0}
39
+ - B-premise: {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0}
40
+ - I-claim: {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0}
41
+ - I-majorclaim: {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0}
42
+ - I-premise: {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0}
43
+ - O: {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0}
44
+ - Accuracy: 0.8874
45
+ - Macro avg: {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0}
46
+ - Weighted avg: {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0}
47
 
48
  ## Model description
49
 
 
68
  - seed: 42
69
  - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
70
  - lr_scheduler_type: linear
71
+ - num_epochs: 5
72
 
73
  ### Training results
74
 
75
+ | Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg |
76
+ |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
77
+ | No log | 1.0 | 41 | 0.4487 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.7102510460251046, 'recall': 0.9590395480225988, 'f1-score': 0.8161057692307693, 'support': 708.0} | {'precision': 0.5242566510172144, 'recall': 0.0722917565817868, 'f1-score': 0.1270623933244832, 'support': 4634.0} | {'precision': 0.635728952772074, 'recall': 0.6736292428198434, 'f1-score': 0.6541305725755335, 'support': 2298.0} | {'precision': 0.7685153090699018, 'recall': 0.9773712438468886, 'f1-score': 0.8604508262992788, 'support': 13611.0} | {'precision': 0.9707444699912788, 'recall': 0.9973120469169993, 'f1-score': 0.9838489353153878, 'support': 12277.0} | 0.8279 | {'precision': 0.5156423469822248, 'recall': 0.5256634054554452, 'f1-score': 0.49165692810649325, 'support': 33953.0} | {'precision': 0.7884799553707518, 'recall': 0.827879716078108, 'f1-score': 0.7793158674251499, 'support': 33953.0} |
78
+ | No log | 2.0 | 82 | 0.3651 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6355475763016158, 'recall': 1.0, 'f1-score': 0.7771679473106476, 'support': 708.0} | {'precision': 0.5581831831831832, 'recall': 0.3208890807078118, 'f1-score': 0.40750890654973965, 'support': 4634.0} | {'precision': 0.888728323699422, 'recall': 0.5352480417754569, 'f1-score': 0.66811515480717, 'support': 2298.0} | {'precision': 0.8056312443233424, 'recall': 0.9775181838219088, 'f1-score': 0.8832901812387971, 'support': 13611.0} | {'precision': 0.9999184139675288, 'recall': 0.9982894844017268, 'f1-score': 0.9991032852368142, 'support': 12277.0} | 0.8537 | {'precision': 0.5554298202107274, 'recall': 0.5474206843867007, 'f1-score': 0.5335979250204527, 'support': 33953.0} | {'precision': 0.8341039518604557, 'recall': 0.8537095396577622, 'f1-score': 0.832398123732452, 'support': 33953.0} |
79
+ | No log | 3.0 | 123 | 0.2896 | {'precision': 0.47393364928909953, 'recall': 0.352112676056338, 'f1-score': 0.40404040404040403, 'support': 284.0} | {'precision': 0.9333333333333333, 'recall': 0.2978723404255319, 'f1-score': 0.45161290322580644, 'support': 141.0} | {'precision': 0.7856328392246295, 'recall': 0.9731638418079096, 'f1-score': 0.8694006309148264, 'support': 708.0} | {'precision': 0.6642079381805409, 'recall': 0.4080707811825637, 'f1-score': 0.5055473867130063, 'support': 4634.0} | {'precision': 0.7170077628793226, 'recall': 0.8842471714534378, 'f1-score': 0.7918939984411536, 'support': 2298.0} | {'precision': 0.8606260075228371, 'recall': 0.9413709499669385, 'f1-score': 0.8991894452436927, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9978822187830904, 'f1-score': 0.9989399869536856, 'support': 12277.0} | 0.8782 | {'precision': 0.7763916472042519, 'recall': 0.6935314256679729, 'f1-score': 0.7029463936475108, 'support': 33953.0} | {'precision': 0.8699976208166521, 'recall': 0.8782140017082437, 'f1-score': 0.867649200314498, 'support': 33953.0} |
80
+ | No log | 4.0 | 164 | 0.2798 | {'precision': 0.5757575757575758, 'recall': 0.5352112676056338, 'f1-score': 0.5547445255474452, 'support': 284.0} | {'precision': 0.9054054054054054, 'recall': 0.475177304964539, 'f1-score': 0.6232558139534884, 'support': 141.0} | {'precision': 0.8377358490566038, 'recall': 0.940677966101695, 'f1-score': 0.8862275449101796, 'support': 708.0} | {'precision': 0.6079838528818121, 'recall': 0.5850237375917134, 'f1-score': 0.596282854943363, 'support': 4634.0} | {'precision': 0.8411037107516651, 'recall': 0.7693646649260226, 'f1-score': 0.8036363636363636, 'support': 2298.0} | {'precision': 0.8822353864820498, 'recall': 0.9081625156123724, 'f1-score': 0.8950112229382376, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9976378594119084, 'f1-score': 0.9988175331294598, 'support': 12277.0} | 0.8828 | {'precision': 0.8071745400478731, 'recall': 0.7444650451734122, 'f1-score': 0.7654251227226482, 'support': 33953.0} | {'precision': 0.881207953399647, 'recall': 0.882779135864283, 'f1-score': 0.8814327847290655, 'support': 33953.0} |
81
+ | No log | 5.0 | 205 | 0.2775 | {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0} | {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0} | {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0} | {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0} | {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0} | {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0} | 0.8874 | {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0} | {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0} |
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
82
 
83
 
84
  ### Framework versions
85
 
86
+ - Transformers 4.45.2
87
+ - Pytorch 2.5.0+cu124
88
+ - Datasets 2.19.1
89
+ - Tokenizers 0.20.1
meta_data/README_s42_e5.md CHANGED
@@ -1,9 +1,11 @@
1
  ---
 
 
2
  base_model: allenai/longformer-base-4096
3
  tags:
4
  - generated_from_trainer
5
  datasets:
6
- - essays_su_g
7
  metrics:
8
  - accuracy
9
  model-index:
@@ -13,15 +15,15 @@ model-index:
13
  name: Token Classification
14
  type: token-classification
15
  dataset:
16
- name: essays_su_g
17
- type: essays_su_g
18
  config: sep_tok_full_labels
19
- split: train[80%:100%]
20
  args: sep_tok_full_labels
21
  metrics:
22
  - name: Accuracy
23
  type: accuracy
24
- value: 0.8887729338495203
25
  ---
26
 
27
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -29,19 +31,19 @@ should probably proofread and complete it, then remove this comment. -->
29
 
30
  # longformer-sep_tok_full_labels
31
 
32
- This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
33
  It achieves the following results on the evaluation set:
34
- - Loss: 0.2742
35
- - B-claim: {'precision': 0.5965665236051502, 'recall': 0.5129151291512916, 'f1-score': 0.5515873015873016, 'support': 271.0}
36
- - B-majorclaim: {'precision': 0.8135593220338984, 'recall': 0.6906474820143885, 'f1-score': 0.7470817120622568, 'support': 139.0}
37
- - B-premise: {'precision': 0.8395953757225434, 'recall': 0.9178515007898894, 'f1-score': 0.8769811320754718, 'support': 633.0}
38
- - I-claim: {'precision': 0.6239658393381372, 'recall': 0.5843539115221195, 'f1-score': 0.6035105833763552, 'support': 4001.0}
39
- - I-majorclaim: {'precision': 0.8184480234260615, 'recall': 0.8330849478390462, 'f1-score': 0.8257016248153619, 'support': 2013.0}
40
- - I-premise: {'precision': 0.8878504672897196, 'recall': 0.9050811573747354, 'f1-score': 0.8963830159007513, 'support': 11336.0}
41
- - O: {'precision': 1.0, 'recall': 0.9998231966053748, 'f1-score': 0.9999115904871364, 'support': 11312.0}
42
- - Accuracy: 0.8888
43
- - Macro avg: {'precision': 0.7971407930593586, 'recall': 0.7776796178995493, 'f1-score': 0.7858795657578049, 'support': 29705.0}
44
- - Weighted avg: {'precision': 0.8862788836235276, 'recall': 0.8887729338495203, 'f1-score': 0.8873130640630906, 'support': 29705.0}
45
 
46
  ## Model description
47
 
@@ -70,18 +72,18 @@ The following hyperparameters were used during training:
70
 
71
  ### Training results
72
 
73
- | Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg |
74
- |:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
75
- | No log | 1.0 | 41 | 0.3939 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 271.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 139.0} | {'precision': 0.6519871106337272, 'recall': 0.9589257503949447, 'f1-score': 0.7762148337595907, 'support': 633.0} | {'precision': 0.4607276436586195, 'recall': 0.33866533366658336, 'f1-score': 0.390377412849323, 'support': 4001.0} | {'precision': 0.7161716171617162, 'recall': 0.53899652260308, 'f1-score': 0.6150793650793651, 'support': 2013.0} | {'precision': 0.8191702652683529, 'recall': 0.9371030345800988, 'f1-score': 0.8741770901909152, 'support': 11336.0} | {'precision': 0.9918942731277534, 'recall': 0.9952263083451203, 'f1-score': 0.9935574971317624, 'support': 11312.0} | 0.8392 | {'precision': 0.5199929871214527, 'recall': 0.538416707084261, 'f1-score': 0.5213437427158508, 'support': 29705.0} | {'precision': 0.8148175308318133, 'recall': 0.839185322336307, 'f1-score': 0.8227635981297234, 'support': 29705.0} |
76
- | No log | 2.0 | 82 | 0.3194 | {'precision': 0.3546099290780142, 'recall': 0.18450184501845018, 'f1-score': 0.24271844660194175, 'support': 271.0} | {'precision': 0.7, 'recall': 0.050359712230215826, 'f1-score': 0.09395973154362416, 'support': 139.0} | {'precision': 0.7255813953488373, 'recall': 0.985781990521327, 'f1-score': 0.8359008707300737, 'support': 633.0} | {'precision': 0.5617191404297851, 'recall': 0.2809297675581105, 'f1-score': 0.37454181939353554, 'support': 4001.0} | {'precision': 0.7387470997679815, 'recall': 0.7908594138102335, 'f1-score': 0.7639155470249521, 'support': 2013.0} | {'precision': 0.8175242974459429, 'recall': 0.9572159491884262, 'f1-score': 0.8818724856759722, 'support': 11336.0} | {'precision': 0.9999112294718153, 'recall': 0.9957567185289957, 'f1-score': 0.9978296496434425, 'support': 11312.0} | 0.8588 | {'precision': 0.6997275845060538, 'recall': 0.6064864852651084, 'f1-score': 0.5986769358019346, 'support': 29705.0} | {'precision': 0.8404537879266404, 'recall': 0.8588453122369971, 'f1-score': 0.8392062502215126, 'support': 29705.0} |
77
- | No log | 3.0 | 123 | 0.2681 | {'precision': 0.5185185185185185, 'recall': 0.46494464944649444, 'f1-score': 0.490272373540856, 'support': 271.0} | {'precision': 0.7804878048780488, 'recall': 0.460431654676259, 'f1-score': 0.579185520361991, 'support': 139.0} | {'precision': 0.8223776223776224, 'recall': 0.9289099526066351, 'f1-score': 0.8724035608308606, 'support': 633.0} | {'precision': 0.6154261057173679, 'recall': 0.5703574106473381, 'f1-score': 0.5920352834349463, 'support': 4001.0} | {'precision': 0.772628843655262, 'recall': 0.8862394436164928, 'f1-score': 0.8255437297547431, 'support': 2013.0} | {'precision': 0.8900924702774108, 'recall': 0.8915843330980946, 'f1-score': 0.8908377770922393, 'support': 11336.0} | {'precision': 1.0, 'recall': 0.9983203677510608, 'f1-score': 0.9991594779915948, 'support': 11312.0} | 0.8835 | {'precision': 0.7713616236320329, 'recall': 0.7429696874060535, 'f1-score': 0.7499196747153186, 'support': 29705.0} | {'precision': 0.8816455584137204, 'recall': 0.8834876283453964, 'f1-score': 0.8819115909019992, 'support': 29705.0} |
78
- | No log | 4.0 | 164 | 0.2651 | {'precision': 0.5869565217391305, 'recall': 0.4981549815498155, 'f1-score': 0.5389221556886228, 'support': 271.0} | {'precision': 0.8034188034188035, 'recall': 0.6762589928057554, 'f1-score': 0.7343750000000001, 'support': 139.0} | {'precision': 0.8359712230215828, 'recall': 0.9178515007898894, 'f1-score': 0.8750000000000001, 'support': 633.0} | {'precision': 0.6168500134156157, 'recall': 0.5746063484128968, 'f1-score': 0.5949792960662525, 'support': 4001.0} | {'precision': 0.8074074074074075, 'recall': 0.812220566318927, 'f1-score': 0.8098068350668647, 'support': 2013.0} | {'precision': 0.8854211569962928, 'recall': 0.9059633027522935, 'f1-score': 0.8955744495312841, 'support': 11336.0} | {'precision': 0.9999115983026874, 'recall': 0.9999115983026874, 'f1-score': 0.9999115983026874, 'support': 11312.0} | 0.8862 | {'precision': 0.7908481034716457, 'recall': 0.769281041561752, 'f1-score': 0.7783670478079587, 'support': 29705.0} | {'precision': 0.8833991740695555, 'recall': 0.8862144420131292, 'f1-score': 0.884561060819018, 'support': 29705.0} |
79
- | No log | 5.0 | 205 | 0.2742 | {'precision': 0.5965665236051502, 'recall': 0.5129151291512916, 'f1-score': 0.5515873015873016, 'support': 271.0} | {'precision': 0.8135593220338984, 'recall': 0.6906474820143885, 'f1-score': 0.7470817120622568, 'support': 139.0} | {'precision': 0.8395953757225434, 'recall': 0.9178515007898894, 'f1-score': 0.8769811320754718, 'support': 633.0} | {'precision': 0.6239658393381372, 'recall': 0.5843539115221195, 'f1-score': 0.6035105833763552, 'support': 4001.0} | {'precision': 0.8184480234260615, 'recall': 0.8330849478390462, 'f1-score': 0.8257016248153619, 'support': 2013.0} | {'precision': 0.8878504672897196, 'recall': 0.9050811573747354, 'f1-score': 0.8963830159007513, 'support': 11336.0} | {'precision': 1.0, 'recall': 0.9998231966053748, 'f1-score': 0.9999115904871364, 'support': 11312.0} | 0.8888 | {'precision': 0.7971407930593586, 'recall': 0.7776796178995493, 'f1-score': 0.7858795657578049, 'support': 29705.0} | {'precision': 0.8862788836235276, 'recall': 0.8887729338495203, 'f1-score': 0.8873130640630906, 'support': 29705.0} |
80
 
81
 
82
  ### Framework versions
83
 
84
- - Transformers 4.37.2
85
- - Pytorch 2.2.0+cu121
86
- - Datasets 2.17.0
87
- - Tokenizers 0.15.2
 
1
  ---
2
+ library_name: transformers
3
+ license: apache-2.0
4
  base_model: allenai/longformer-base-4096
5
  tags:
6
  - generated_from_trainer
7
  datasets:
8
+ - stab-gurevych-essays
9
  metrics:
10
  - accuracy
11
  model-index:
 
15
  name: Token Classification
16
  type: token-classification
17
  dataset:
18
+ name: stab-gurevych-essays
19
+ type: stab-gurevych-essays
20
  config: sep_tok_full_labels
21
+ split: train[0%:20%]
22
  args: sep_tok_full_labels
23
  metrics:
24
  - name: Accuracy
25
  type: accuracy
26
+ value: 0.8874031749771744
27
  ---
28
 
29
  <!-- This model card has been generated automatically according to the information the Trainer had access to. You
 
31
 
32
  # longformer-sep_tok_full_labels
33
 
34
+ This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the stab-gurevych-essays dataset.
35
  It achieves the following results on the evaluation set:
36
+ - Loss: 0.2775
37
+ - B-claim: {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0}
38
+ - B-majorclaim: {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0}
39
+ - B-premise: {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0}
40
+ - I-claim: {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0}
41
+ - I-majorclaim: {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0}
42
+ - I-premise: {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0}
43
+ - O: {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0}
44
+ - Accuracy: 0.8874
45
+ - Macro avg: {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0}
46
+ - Weighted avg: {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0}
47
 
48
  ## Model description
49
 
 
72
 
73
  ### Training results
74
 
75
+ | Training Loss | Epoch | Step | Validation Loss | B-claim | B-majorclaim | B-premise | I-claim | I-majorclaim | I-premise | O | Accuracy | Macro avg | Weighted avg |
76
+ |:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:--------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
77
+ | No log | 1.0 | 41 | 0.4487 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.7102510460251046, 'recall': 0.9590395480225988, 'f1-score': 0.8161057692307693, 'support': 708.0} | {'precision': 0.5242566510172144, 'recall': 0.0722917565817868, 'f1-score': 0.1270623933244832, 'support': 4634.0} | {'precision': 0.635728952772074, 'recall': 0.6736292428198434, 'f1-score': 0.6541305725755335, 'support': 2298.0} | {'precision': 0.7685153090699018, 'recall': 0.9773712438468886, 'f1-score': 0.8604508262992788, 'support': 13611.0} | {'precision': 0.9707444699912788, 'recall': 0.9973120469169993, 'f1-score': 0.9838489353153878, 'support': 12277.0} | 0.8279 | {'precision': 0.5156423469822248, 'recall': 0.5256634054554452, 'f1-score': 0.49165692810649325, 'support': 33953.0} | {'precision': 0.7884799553707518, 'recall': 0.827879716078108, 'f1-score': 0.7793158674251499, 'support': 33953.0} |
78
+ | No log | 2.0 | 82 | 0.3651 | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0} | {'precision': 0.6355475763016158, 'recall': 1.0, 'f1-score': 0.7771679473106476, 'support': 708.0} | {'precision': 0.5581831831831832, 'recall': 0.3208890807078118, 'f1-score': 0.40750890654973965, 'support': 4634.0} | {'precision': 0.888728323699422, 'recall': 0.5352480417754569, 'f1-score': 0.66811515480717, 'support': 2298.0} | {'precision': 0.8056312443233424, 'recall': 0.9775181838219088, 'f1-score': 0.8832901812387971, 'support': 13611.0} | {'precision': 0.9999184139675288, 'recall': 0.9982894844017268, 'f1-score': 0.9991032852368142, 'support': 12277.0} | 0.8537 | {'precision': 0.5554298202107274, 'recall': 0.5474206843867007, 'f1-score': 0.5335979250204527, 'support': 33953.0} | {'precision': 0.8341039518604557, 'recall': 0.8537095396577622, 'f1-score': 0.832398123732452, 'support': 33953.0} |
79
+ | No log | 3.0 | 123 | 0.2896 | {'precision': 0.47393364928909953, 'recall': 0.352112676056338, 'f1-score': 0.40404040404040403, 'support': 284.0} | {'precision': 0.9333333333333333, 'recall': 0.2978723404255319, 'f1-score': 0.45161290322580644, 'support': 141.0} | {'precision': 0.7856328392246295, 'recall': 0.9731638418079096, 'f1-score': 0.8694006309148264, 'support': 708.0} | {'precision': 0.6642079381805409, 'recall': 0.4080707811825637, 'f1-score': 0.5055473867130063, 'support': 4634.0} | {'precision': 0.7170077628793226, 'recall': 0.8842471714534378, 'f1-score': 0.7918939984411536, 'support': 2298.0} | {'precision': 0.8606260075228371, 'recall': 0.9413709499669385, 'f1-score': 0.8991894452436927, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9978822187830904, 'f1-score': 0.9989399869536856, 'support': 12277.0} | 0.8782 | {'precision': 0.7763916472042519, 'recall': 0.6935314256679729, 'f1-score': 0.7029463936475108, 'support': 33953.0} | {'precision': 0.8699976208166521, 'recall': 0.8782140017082437, 'f1-score': 0.867649200314498, 'support': 33953.0} |
80
+ | No log | 4.0 | 164 | 0.2798 | {'precision': 0.5757575757575758, 'recall': 0.5352112676056338, 'f1-score': 0.5547445255474452, 'support': 284.0} | {'precision': 0.9054054054054054, 'recall': 0.475177304964539, 'f1-score': 0.6232558139534884, 'support': 141.0} | {'precision': 0.8377358490566038, 'recall': 0.940677966101695, 'f1-score': 0.8862275449101796, 'support': 708.0} | {'precision': 0.6079838528818121, 'recall': 0.5850237375917134, 'f1-score': 0.596282854943363, 'support': 4634.0} | {'precision': 0.8411037107516651, 'recall': 0.7693646649260226, 'f1-score': 0.8036363636363636, 'support': 2298.0} | {'precision': 0.8822353864820498, 'recall': 0.9081625156123724, 'f1-score': 0.8950112229382376, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9976378594119084, 'f1-score': 0.9988175331294598, 'support': 12277.0} | 0.8828 | {'precision': 0.8071745400478731, 'recall': 0.7444650451734122, 'f1-score': 0.7654251227226482, 'support': 33953.0} | {'precision': 0.881207953399647, 'recall': 0.882779135864283, 'f1-score': 0.8814327847290655, 'support': 33953.0} |
81
+ | No log | 5.0 | 205 | 0.2775 | {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0} | {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0} | {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0} | {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0} | {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0} | {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0} | {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0} | 0.8874 | {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0} | {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0} |
82
 
83
 
84
  ### Framework versions
85
 
86
+ - Transformers 4.45.2
87
+ - Pytorch 2.5.0+cu124
88
+ - Datasets 2.19.1
89
+ - Tokenizers 0.20.1
meta_data/meta_s42_e5_cvi0.json CHANGED
@@ -1 +1 @@
1
- {"B-Claim": {"precision": 0.47393364928909953, "recall": 0.352112676056338, "f1-score": 0.40404040404040403, "support": 284.0}, "B-MajorClaim": {"precision": 0.9333333333333333, "recall": 0.2978723404255319, "f1-score": 0.45161290322580644, "support": 141.0}, "B-Premise": {"precision": 0.7856328392246295, "recall": 0.9731638418079096, "f1-score": 0.8694006309148264, "support": 708.0}, "I-Claim": {"precision": 0.6642079381805409, "recall": 0.4080707811825637, "f1-score": 0.5055473867130063, "support": 4634.0}, "I-MajorClaim": {"precision": 0.7170077628793226, "recall": 0.8842471714534378, "f1-score": 0.7918939984411536, "support": 2298.0}, "I-Premise": {"precision": 0.8606260075228371, "recall": 0.9413709499669385, "f1-score": 0.8991894452436927, "support": 13611.0}, "O": {"precision": 1.0, "recall": 0.9978822187830904, "f1-score": 0.9989399869536856, "support": 12277.0}, "accuracy": 0.8782140017082437, "macro avg": {"precision": 0.7763916472042519, "recall": 0.6935314256679729, "f1-score": 0.7029463936475108, "support": 33953.0}, "weighted avg": {"precision": 0.8699976208166521, "recall": 0.8782140017082437, "f1-score": 0.867649200314498, "support": 33953.0}}
 
1
+ {"B-Claim": {"precision": 0.6083333333333333, "recall": 0.5140845070422535, "f1-score": 0.5572519083969466, "support": 284.0}, "B-MajorClaim": {"precision": 0.88, "recall": 0.624113475177305, "f1-score": 0.7302904564315352, "support": 141.0}, "B-Premise": {"precision": 0.8373266078184111, "recall": 0.9378531073446328, "f1-score": 0.8847435043304464, "support": 708.0}, "I-Claim": {"precision": 0.6361367606688295, "recall": 0.5500647388864911, "f1-score": 0.5899780118041893, "support": 4634.0}, "I-MajorClaim": {"precision": 0.8413284132841329, "recall": 0.793733681462141, "f1-score": 0.8168383340797134, "support": 2298.0}, "I-Premise": {"precision": 0.8758342602892102, "recall": 0.9255749026522665, "f1-score": 0.9000178603322022, "support": 13611.0}, "O": {"precision": 1.0, "recall": 0.9986967500203633, "f1-score": 0.999347950118184, "support": 12277.0}, "accuracy": 0.8874031749771744, "macro avg": {"precision": 0.8112799107705595, "recall": 0.7634458803693505, "f1-score": 0.782638289356174, "support": 33953.0}, "weighted avg": {"precision": 0.8826579231427218, "recall": 0.8874031749771744, "f1-score": 0.8840991775809467, "support": 33953.0}}
model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:f14b7db205675e39b7bd2b7766040d5767ef618a5de3d85f933fd230453ae8bd
3
  size 592330980
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:26b7d85a9aceefccf48dcff74fe9ffbdc3b0b6a05d9be18b1bc4de9ca5d2e95e
3
  size 592330980