trainer: training complete at 2024-02-19 21:15:06.553139.

README.md

@@ -22,7 +22,7 @@ model-index:
     metrics:
     - name: Accuracy
       type: accuracy
-      value: 0.8900989109801178
+      value: 0.8942951343790588
 ---
 
 <!-- This model card has been generated automatically according to the information the Trainer had access to. You
@@ -32,17 +32,17 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.2764
-- B-claim: {'precision': 0.6491228070175439, 'recall': 0.5342960288808665, 'f1-score': 0.5861386138613861, 'support': 277.0}
-- B-majorclaim: {'precision': 0.849624060150376, 'recall': 0.8014184397163121, 'f1-score': 0.8248175182481753, 'support': 141.0}
-- B-premise: {'precision': 0.8424068767908309, 'recall': 0.9173166926677067, 'f1-score': 0.878267363704257, 'support': 641.0}
-- I-claim: {'precision': 0.635204773528614, 'recall': 0.5741603334150527, 'f1-score': 0.6031419005923255, 'support': 4079.0}
-- I-majorclaim: {'precision': 0.8378640776699029, 'recall': 0.8456638902498775, 'f1-score': 0.8417459156303341, 'support': 2041.0}
-- I-premise: {'precision': 0.8807168822385663, 'recall': 0.9094718463553033, 'f1-score': 0.8948634255282598, 'support': 11455.0}
-- O: {'precision': 1.0, 'recall': 0.9999122268059335, 'f1-score': 0.9999561114768488, 'support': 11393.0}
-- Accuracy: 0.8901
-- Macro avg: {'precision': 0.8135627824851192, 'recall': 0.7974627797272931, 'f1-score': 0.8041329784345124, 'support': 30027.0}
-- Weighted avg: {'precision': 0.8866113683630815, 'recall': 0.8900989109801178, 'f1-score': 0.887967788658427, 'support': 30027.0}
+- Loss: 0.2899
+- B-claim: {'precision': 0.657258064516129, 'recall': 0.5884476534296029, 'f1-score': 0.620952380952381, 'support': 277.0}
+- B-majorclaim: {'precision': 0.8380281690140845, 'recall': 0.8439716312056738, 'f1-score': 0.8409893992932863, 'support': 141.0}
+- B-premise: {'precision': 0.8622754491017964, 'recall': 0.8985959438377535, 'f1-score': 0.880061115355233, 'support': 641.0}
+- I-claim: {'precision': 0.6403553299492386, 'recall': 0.6185339544005883, 'f1-score': 0.6292555181444071, 'support': 4079.0}
+- I-majorclaim: {'precision': 0.8763931104356636, 'recall': 0.8476237138657521, 'f1-score': 0.8617683686176836, 'support': 2041.0}
+- I-premise: {'precision': 0.8874882085584427, 'recall': 0.903448275862069, 'f1-score': 0.8953971275307147, 'support': 11455.0}
+- O: {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}
+- Accuracy: 0.8943
+- Macro avg: {'precision': 0.8231015094406778, 'recall': 0.8143744532287771, 'f1-score': 0.8183400036032316, 'support': 30027.0}
+- Weighted avg: {'precision': 0.8929245767023277, 'recall': 0.8942951343790588, 'f1-score': 0.8935151399218458, 'support': 30027.0}
 
 ## Model description
 
@@ -67,18 +67,19 @@ The following hyperparameters were used during training:
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 6
+- num_epochs: 7
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                                             | B-majorclaim                                                                                                      | B-premise                                                                                                         | I-claim                                                                                                              | I-majorclaim                                                                                                       | I-premise                                                                                                           | O                                                                                                                   | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
-|:-------------:|:-----:|:----:|:---------------:|:-------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
-| No log        | 1.0   | 41   | 0.4582          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                                | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                              | {'precision': 0.7420178799489144, 'recall': 0.906396255850234, 'f1-score': 0.8160112359550561, 'support': 641.0}  | {'precision': 0.5056320400500626, 'recall': 0.39617553321892623, 'f1-score': 0.4442611683848798, 'support': 4079.0}  | {'precision': 0.5302879841112215, 'recall': 0.7849093581577657, 'f1-score': 0.6329514026076649, 'support': 2041.0} | {'precision': 0.8788833858622243, 'recall': 0.8520296813618508, 'f1-score': 0.8652482269503546, 'support': 11455.0} | {'precision': 0.9525247441704412, 'recall': 0.9967523918195383, 'f1-score': 0.9741368217885481, 'support': 11393.0} | 0.8298   | {'precision': 0.5156208620204091, 'recall': 0.5623233172011879, 'f1-score': 0.5332298365266434, 'support': 30027.0} | {'precision': 0.8172693883294395, 'recall': 0.8297532221001099, 'f1-score': 0.8204888124409565, 'support': 30027.0} |
-| No log        | 2.0   | 82   | 0.3126          | {'precision': 0.33516483516483514, 'recall': 0.22021660649819494, 'f1-score': 0.2657952069716775, 'support': 277.0} | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                              | {'precision': 0.728110599078341, 'recall': 0.9859594383775351, 'f1-score': 0.8376408217362491, 'support': 641.0}  | {'precision': 0.5190856635596982, 'recall': 0.573424859034077, 'f1-score': 0.5449039021549213, 'support': 4079.0}    | {'precision': 0.8797524314765695, 'recall': 0.4875061244487996, 'f1-score': 0.6273644388398487, 'support': 2041.0} | {'precision': 0.864876171352075, 'recall': 0.9024006983849847, 'f1-score': 0.8832400563933867, 'support': 11455.0}  | {'precision': 0.9998243611135506, 'recall': 0.9992978144474678, 'f1-score': 0.9995610184372257, 'support': 11393.0} | 0.8575   | {'precision': 0.6181162945350099, 'recall': 0.5955436487415798, 'f1-score': 0.5940722063619013, 'support': 30027.0} | {'precision': 0.8582489151982637, 'recall': 0.8575282245978619, 'f1-score': 0.8532047958393347, 'support': 30027.0} |
-| No log        | 3.0   | 123  | 0.3020          | {'precision': 0.4793814432989691, 'recall': 0.33574007220216606, 'f1-score': 0.39490445859872614, 'support': 277.0} | {'precision': 0.8611111111111112, 'recall': 0.4397163120567376, 'f1-score': 0.5821596244131456, 'support': 141.0} | {'precision': 0.7793190416141236, 'recall': 0.9641185647425897, 'f1-score': 0.8619246861924686, 'support': 641.0} | {'precision': 0.6011146496815286, 'recall': 0.37018877175778375, 'f1-score': 0.45820057654377183, 'support': 4079.0} | {'precision': 0.8804286520022561, 'recall': 0.7648211660950515, 'f1-score': 0.8185631882538018, 'support': 2041.0} | {'precision': 0.8224763051000451, 'recall': 0.9545176778699258, 'f1-score': 0.8835912562123723, 'support': 11455.0} | {'precision': 1.0, 'recall': 0.9996489072237339, 'f1-score': 0.9998244227899218, 'support': 11393.0}                | 0.8714   | {'precision': 0.7748330289725762, 'recall': 0.6898216388497127, 'f1-score': 0.7141668875720297, 'support': 30027.0} | {'precision': 0.8597967310035538, 'recall': 0.871449029207047, 'f1-score': 0.8590998725690157, 'support': 30027.0}  |
-| No log        | 4.0   | 164  | 0.2667          | {'precision': 0.5948275862068966, 'recall': 0.4981949458483754, 'f1-score': 0.5422396856581532, 'support': 277.0}   | {'precision': 0.8547008547008547, 'recall': 0.7092198581560284, 'f1-score': 0.7751937984496124, 'support': 141.0} | {'precision': 0.8267605633802817, 'recall': 0.9157566302652106, 'f1-score': 0.8689859363434492, 'support': 641.0} | {'precision': 0.6182104434531619, 'recall': 0.5775925471929394, 'f1-score': 0.5972116603295311, 'support': 4079.0}   | {'precision': 0.8661375661375661, 'recall': 0.8020578147966683, 'f1-score': 0.8328669549732893, 'support': 2041.0} | {'precision': 0.8760316658244904, 'recall': 0.9080750763858577, 'f1-score': 0.8917656136139568, 'support': 11455.0} | {'precision': 1.0, 'recall': 1.0, 'f1-score': 1.0, 'support': 11393.0}                                              | 0.8863   | {'precision': 0.8052383828147504, 'recall': 0.7729852675207257, 'f1-score': 0.7868948070525704, 'support': 30027.0} | {'precision': 0.8836261859783078, 'recall': 0.8863023279048856, 'f1-score': 0.8845574191639981, 'support': 30027.0} |
-| No log        | 5.0   | 205  | 0.2851          | {'precision': 0.645021645021645, 'recall': 0.5379061371841155, 'f1-score': 0.5866141732283465, 'support': 277.0}    | {'precision': 0.8538461538461538, 'recall': 0.7872340425531915, 'f1-score': 0.8191881918819187, 'support': 141.0} | {'precision': 0.8412017167381974, 'recall': 0.9173166926677067, 'f1-score': 0.8776119402985075, 'support': 641.0} | {'precision': 0.6292367399741268, 'recall': 0.5962245648443246, 'f1-score': 0.6122860020140987, 'support': 4079.0}   | {'precision': 0.8551020408163266, 'recall': 0.8211660950514453, 'f1-score': 0.8377905523619096, 'support': 2041.0} | {'precision': 0.8839247723597992, 'recall': 0.9067656045395024, 'f1-score': 0.8951995173661983, 'support': 11455.0} | {'precision': 1.0, 'recall': 0.9998244536118669, 'f1-score': 0.9999122191011235, 'support': 11393.0}                | 0.8903   | {'precision': 0.815476152679464, 'recall': 0.7952053700645932, 'f1-score': 0.8040860851788718, 'support': 30027.0}  | {'precision': 0.8881523894772896, 'recall': 0.8903320345022813, 'f1-score': 0.8890166322977991, 'support': 30027.0} |
-| No log        | 6.0   | 246  | 0.2764          | {'precision': 0.6491228070175439, 'recall': 0.5342960288808665, 'f1-score': 0.5861386138613861, 'support': 277.0}   | {'precision': 0.849624060150376, 'recall': 0.8014184397163121, 'f1-score': 0.8248175182481753, 'support': 141.0}  | {'precision': 0.8424068767908309, 'recall': 0.9173166926677067, 'f1-score': 0.878267363704257, 'support': 641.0}  | {'precision': 0.635204773528614, 'recall': 0.5741603334150527, 'f1-score': 0.6031419005923255, 'support': 4079.0}    | {'precision': 0.8378640776699029, 'recall': 0.8456638902498775, 'f1-score': 0.8417459156303341, 'support': 2041.0} | {'precision': 0.8807168822385663, 'recall': 0.9094718463553033, 'f1-score': 0.8948634255282598, 'support': 11455.0} | {'precision': 1.0, 'recall': 0.9999122268059335, 'f1-score': 0.9999561114768488, 'support': 11393.0}                | 0.8901   | {'precision': 0.8135627824851192, 'recall': 0.7974627797272931, 'f1-score': 0.8041329784345124, 'support': 30027.0} | {'precision': 0.8866113683630815, 'recall': 0.8900989109801178, 'f1-score': 0.887967788658427, 'support': 30027.0}  |
+| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                                            | B-majorclaim                                                                                                       | B-premise                                                                                                         | I-claim                                                                                                              | I-majorclaim                                                                                                       | I-premise                                                                                                           | O                                                                                                                   | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.4221          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                               | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                               | {'precision': 0.7233782129742962, 'recall': 0.921996879875195, 'f1-score': 0.8106995884773661, 'support': 641.0}  | {'precision': 0.5306122448979592, 'recall': 0.26771267467516546, 'f1-score': 0.35587420563793387, 'support': 4079.0} | {'precision': 0.5553590378493102, 'recall': 0.7692307692307693, 'f1-score': 0.6450287592440428, 'support': 2041.0} | {'precision': 0.8375491849353569, 'recall': 0.9105194238323876, 'f1-score': 0.8725112932909486, 'support': 11455.0} | {'precision': 0.9565363881401617, 'recall': 0.9967523918195383, 'f1-score': 0.9762303889963465, 'support': 11393.0} | 0.8339   | {'precision': 0.5147764383995834, 'recall': 0.5523160199190079, 'f1-score': 0.5229063193780912, 'support': 30027.0} | {'precision': 0.8077225683958144, 'recall': 0.8338828387784327, 'f1-score': 0.8127546110204618, 'support': 30027.0} |
+| No log        | 2.0   | 82   | 0.2939          | {'precision': 0.3915094339622642, 'recall': 0.2996389891696751, 'f1-score': 0.3394683026584867, 'support': 277.0}  | {'precision': 0.9166666666666666, 'recall': 0.07801418439716312, 'f1-score': 0.1437908496732026, 'support': 141.0} | {'precision': 0.7590361445783133, 'recall': 0.982839313572543, 'f1-score': 0.8565601631543168, 'support': 641.0}  | {'precision': 0.5572158867479355, 'recall': 0.6947781318950723, 'f1-score': 0.6184397163120567, 'support': 4079.0}   | {'precision': 0.8706896551724138, 'recall': 0.6927976482116609, 'f1-score': 0.7716234652114597, 'support': 2041.0} | {'precision': 0.9044070291655166, 'recall': 0.8581405499781755, 'f1-score': 0.8806665472137609, 'support': 11455.0} | {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}                | 0.8724   | {'precision': 0.7713481501146429, 'recall': 0.6580298310320414, 'f1-score': 0.6586435942217427, 'support': 30027.0} | {'precision': 0.8834110659403355, 'recall': 0.8723815232957005, 'f1-score': 0.8739266896403793, 'support': 30027.0} |
+| No log        | 3.0   | 123  | 0.2914          | {'precision': 0.5445026178010471, 'recall': 0.37545126353790614, 'f1-score': 0.4444444444444445, 'support': 277.0} | {'precision': 0.8777777777777778, 'recall': 0.5602836879432624, 'f1-score': 0.683982683982684, 'support': 141.0}   | {'precision': 0.7904269081500647, 'recall': 0.953198127925117, 'f1-score': 0.8642149929278642, 'support': 641.0}  | {'precision': 0.6203389830508474, 'recall': 0.3589114979161559, 'f1-score': 0.45472899518558785, 'support': 4079.0}  | {'precision': 0.8564718162839249, 'recall': 0.8040176384125429, 'f1-score': 0.8294162244124337, 'support': 2041.0} | {'precision': 0.8237107201924523, 'recall': 0.9565255347010039, 'f1-score': 0.8851637920588118, 'support': 11455.0} | {'precision': 0.9997367266344888, 'recall': 0.9999122268059335, 'f1-score': 0.9998244690187819, 'support': 11393.0} | 0.8741   | {'precision': 0.7875665071272291, 'recall': 0.7154714253202744, 'f1-score': 0.737396514575801, 'support': 30027.0}  | {'precision': 0.8620670081983853, 'recall': 0.8741466013920804, 'f1-score': 0.8609499443491488, 'support': 30027.0} |
+| No log        | 4.0   | 164  | 0.2585          | {'precision': 0.6327272727272727, 'recall': 0.628158844765343, 'f1-score': 0.6304347826086956, 'support': 277.0}   | {'precision': 0.8308823529411765, 'recall': 0.8014184397163121, 'f1-score': 0.8158844765342961, 'support': 141.0}  | {'precision': 0.874806800618238, 'recall': 0.8829953198127926, 'f1-score': 0.8788819875776397, 'support': 641.0}  | {'precision': 0.6304878048780488, 'recall': 0.6337337582740867, 'f1-score': 0.6321066145005502, 'support': 4079.0}   | {'precision': 0.8236389018147976, 'recall': 0.8672219500244978, 'f1-score': 0.8448687350835322, 'support': 2041.0} | {'precision': 0.9004061451527459, 'recall': 0.8902662592754256, 'f1-score': 0.8953074930863438, 'support': 11455.0} | {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}                | 0.8925   | {'precision': 0.8132659303773815, 'recall': 0.8148277959812082, 'f1-score': 0.8139200292457104, 'support': 30027.0} | {'precision': 0.892934034725354, 'recall': 0.8924967529223699, 'f1-score': 0.8926639632367819, 'support': 30027.0}  |
+| No log        | 5.0   | 205  | 0.2866          | {'precision': 0.6352941176470588, 'recall': 0.5848375451263538, 'f1-score': 0.6090225563909775, 'support': 277.0}  | {'precision': 0.8382352941176471, 'recall': 0.8085106382978723, 'f1-score': 0.8231046931407943, 'support': 141.0}  | {'precision': 0.8605697151424287, 'recall': 0.8954758190327613, 'f1-score': 0.8776758409785932, 'support': 641.0} | {'precision': 0.6380439868975198, 'recall': 0.668546212306938, 'f1-score': 0.6529390638094098, 'support': 4079.0}    | {'precision': 0.8673106253177427, 'recall': 0.8358647721705047, 'f1-score': 0.8512974051896207, 'support': 2041.0} | {'precision': 0.9037239675255913, 'recall': 0.8940200785683108, 'f1-score': 0.8988458331504806, 'support': 11455.0} | {'precision': 0.9997367497367498, 'recall': 1.0, 'f1-score': 0.9998683575409189, 'support': 11393.0}                | 0.8964   | {'precision': 0.8204163509121054, 'recall': 0.8124650093575345, 'f1-score': 0.8161076786001136, 'support': 30027.0} | {'precision': 0.8978823752306275, 'recall': 0.8964265494388384, 'f1-score': 0.8970580772435964, 'support': 30027.0} |
+| No log        | 6.0   | 246  | 0.2945          | {'precision': 0.6460176991150443, 'recall': 0.5270758122743683, 'f1-score': 0.58051689860835, 'support': 277.0}    | {'precision': 0.8646616541353384, 'recall': 0.8156028368794326, 'f1-score': 0.8394160583941607, 'support': 141.0}  | {'precision': 0.8371428571428572, 'recall': 0.9141965678627145, 'f1-score': 0.8739746457867263, 'support': 641.0} | {'precision': 0.6458272753707474, 'recall': 0.5444962000490317, 'f1-score': 0.5908486299547753, 'support': 4079.0}   | {'precision': 0.8827444956477215, 'recall': 0.8446839784419402, 'f1-score': 0.8632949424136204, 'support': 2041.0} | {'precision': 0.8662890913568086, 'recall': 0.9213443910955914, 'f1-score': 0.8929689483035789, 'support': 11455.0} | {'precision': 0.9999122268059335, 'recall': 0.9999122268059335, 'f1-score': 0.9999122268059335, 'support': 11393.0} | 0.8905   | {'precision': 0.8203707570820643, 'recall': 0.7953302876298588, 'f1-score': 0.805847478609592, 'support': 30027.0}  | {'precision': 0.8854972285164366, 'recall': 0.8904652479435174, 'f1-score': 0.886948227760569, 'support': 30027.0}  |
+| No log        | 7.0   | 287  | 0.2899          | {'precision': 0.657258064516129, 'recall': 0.5884476534296029, 'f1-score': 0.620952380952381, 'support': 277.0}    | {'precision': 0.8380281690140845, 'recall': 0.8439716312056738, 'f1-score': 0.8409893992932863, 'support': 141.0}  | {'precision': 0.8622754491017964, 'recall': 0.8985959438377535, 'f1-score': 0.880061115355233, 'support': 641.0}  | {'precision': 0.6403553299492386, 'recall': 0.6185339544005883, 'f1-score': 0.6292555181444071, 'support': 4079.0}   | {'precision': 0.8763931104356636, 'recall': 0.8476237138657521, 'f1-score': 0.8617683686176836, 'support': 2041.0} | {'precision': 0.8874882085584427, 'recall': 0.903448275862069, 'f1-score': 0.8953971275307147, 'support': 11455.0}  | {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}                | 0.8943   | {'precision': 0.8231015094406778, 'recall': 0.8143744532287771, 'f1-score': 0.8183400036032316, 'support': 30027.0} | {'precision': 0.8929245767023277, 'recall': 0.8942951343790588, 'f1-score': 0.8935151399218458, 'support': 30027.0} |
 
 
 ### Framework versions

meta_data/README_s42_e7.md

@@ -0,0 +1,89 @@
+---
+base_model: allenai/longformer-base-4096
+tags:
+- generated_from_trainer
+datasets:
+- essays_su_g
+metrics:
+- accuracy
+model-index:
+- name: longformer-sep_tok_full_labels
+  results:
+  - task:
+      name: Token Classification
+      type: token-classification
+    dataset:
+      name: essays_su_g
+      type: essays_su_g
+      config: sep_tok_full_labels
+      split: test
+      args: sep_tok_full_labels
+    metrics:
+    - name: Accuracy
+      type: accuracy
+      value: 0.8942951343790588
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# longformer-sep_tok_full_labels
+
+This model is a fine-tuned version of [allenai/longformer-base-4096](https://huggingface.co/allenai/longformer-base-4096) on the essays_su_g dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.2899
+- B-claim: {'precision': 0.657258064516129, 'recall': 0.5884476534296029, 'f1-score': 0.620952380952381, 'support': 277.0}
+- B-majorclaim: {'precision': 0.8380281690140845, 'recall': 0.8439716312056738, 'f1-score': 0.8409893992932863, 'support': 141.0}
+- B-premise: {'precision': 0.8622754491017964, 'recall': 0.8985959438377535, 'f1-score': 0.880061115355233, 'support': 641.0}
+- I-claim: {'precision': 0.6403553299492386, 'recall': 0.6185339544005883, 'f1-score': 0.6292555181444071, 'support': 4079.0}
+- I-majorclaim: {'precision': 0.8763931104356636, 'recall': 0.8476237138657521, 'f1-score': 0.8617683686176836, 'support': 2041.0}
+- I-premise: {'precision': 0.8874882085584427, 'recall': 0.903448275862069, 'f1-score': 0.8953971275307147, 'support': 11455.0}
+- O: {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}
+- Accuracy: 0.8943
+- Macro avg: {'precision': 0.8231015094406778, 'recall': 0.8143744532287771, 'f1-score': 0.8183400036032316, 'support': 30027.0}
+- Weighted avg: {'precision': 0.8929245767023277, 'recall': 0.8942951343790588, 'f1-score': 0.8935151399218458, 'support': 30027.0}
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 2e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- num_epochs: 7
+
+### Training results
+
+| Training Loss | Epoch | Step | Validation Loss | B-claim                                                                                                            | B-majorclaim                                                                                                       | B-premise                                                                                                         | I-claim                                                                                                              | I-majorclaim                                                                                                       | I-premise                                                                                                           | O                                                                                                                   | Accuracy | Macro avg                                                                                                           | Weighted avg                                                                                                        |
+|:-------------:|:-----:|:----:|:---------------:|:------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-----------------------------------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------:|:------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|:--------:|:-------------------------------------------------------------------------------------------------------------------:|:-------------------------------------------------------------------------------------------------------------------:|
+| No log        | 1.0   | 41   | 0.4221          | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 277.0}                                               | {'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}                                               | {'precision': 0.7233782129742962, 'recall': 0.921996879875195, 'f1-score': 0.8106995884773661, 'support': 641.0}  | {'precision': 0.5306122448979592, 'recall': 0.26771267467516546, 'f1-score': 0.35587420563793387, 'support': 4079.0} | {'precision': 0.5553590378493102, 'recall': 0.7692307692307693, 'f1-score': 0.6450287592440428, 'support': 2041.0} | {'precision': 0.8375491849353569, 'recall': 0.9105194238323876, 'f1-score': 0.8725112932909486, 'support': 11455.0} | {'precision': 0.9565363881401617, 'recall': 0.9967523918195383, 'f1-score': 0.9762303889963465, 'support': 11393.0} | 0.8339   | {'precision': 0.5147764383995834, 'recall': 0.5523160199190079, 'f1-score': 0.5229063193780912, 'support': 30027.0} | {'precision': 0.8077225683958144, 'recall': 0.8338828387784327, 'f1-score': 0.8127546110204618, 'support': 30027.0} |
+| No log        | 2.0   | 82   | 0.2939          | {'precision': 0.3915094339622642, 'recall': 0.2996389891696751, 'f1-score': 0.3394683026584867, 'support': 277.0}  | {'precision': 0.9166666666666666, 'recall': 0.07801418439716312, 'f1-score': 0.1437908496732026, 'support': 141.0} | {'precision': 0.7590361445783133, 'recall': 0.982839313572543, 'f1-score': 0.8565601631543168, 'support': 641.0}  | {'precision': 0.5572158867479355, 'recall': 0.6947781318950723, 'f1-score': 0.6184397163120567, 'support': 4079.0}   | {'precision': 0.8706896551724138, 'recall': 0.6927976482116609, 'f1-score': 0.7716234652114597, 'support': 2041.0} | {'precision': 0.9044070291655166, 'recall': 0.8581405499781755, 'f1-score': 0.8806665472137609, 'support': 11455.0} | {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}                | 0.8724   | {'precision': 0.7713481501146429, 'recall': 0.6580298310320414, 'f1-score': 0.6586435942217427, 'support': 30027.0} | {'precision': 0.8834110659403355, 'recall': 0.8723815232957005, 'f1-score': 0.8739266896403793, 'support': 30027.0} |
+| No log        | 3.0   | 123  | 0.2914          | {'precision': 0.5445026178010471, 'recall': 0.37545126353790614, 'f1-score': 0.4444444444444445, 'support': 277.0} | {'precision': 0.8777777777777778, 'recall': 0.5602836879432624, 'f1-score': 0.683982683982684, 'support': 141.0}   | {'precision': 0.7904269081500647, 'recall': 0.953198127925117, 'f1-score': 0.8642149929278642, 'support': 641.0}  | {'precision': 0.6203389830508474, 'recall': 0.3589114979161559, 'f1-score': 0.45472899518558785, 'support': 4079.0}  | {'precision': 0.8564718162839249, 'recall': 0.8040176384125429, 'f1-score': 0.8294162244124337, 'support': 2041.0} | {'precision': 0.8237107201924523, 'recall': 0.9565255347010039, 'f1-score': 0.8851637920588118, 'support': 11455.0} | {'precision': 0.9997367266344888, 'recall': 0.9999122268059335, 'f1-score': 0.9998244690187819, 'support': 11393.0} | 0.8741   | {'precision': 0.7875665071272291, 'recall': 0.7154714253202744, 'f1-score': 0.737396514575801, 'support': 30027.0}  | {'precision': 0.8620670081983853, 'recall': 0.8741466013920804, 'f1-score': 0.8609499443491488, 'support': 30027.0} |
+| No log        | 4.0   | 164  | 0.2585          | {'precision': 0.6327272727272727, 'recall': 0.628158844765343, 'f1-score': 0.6304347826086956, 'support': 277.0}   | {'precision': 0.8308823529411765, 'recall': 0.8014184397163121, 'f1-score': 0.8158844765342961, 'support': 141.0}  | {'precision': 0.874806800618238, 'recall': 0.8829953198127926, 'f1-score': 0.8788819875776397, 'support': 641.0}  | {'precision': 0.6304878048780488, 'recall': 0.6337337582740867, 'f1-score': 0.6321066145005502, 'support': 4079.0}   | {'precision': 0.8236389018147976, 'recall': 0.8672219500244978, 'f1-score': 0.8448687350835322, 'support': 2041.0} | {'precision': 0.9004061451527459, 'recall': 0.8902662592754256, 'f1-score': 0.8953074930863438, 'support': 11455.0} | {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}                | 0.8925   | {'precision': 0.8132659303773815, 'recall': 0.8148277959812082, 'f1-score': 0.8139200292457104, 'support': 30027.0} | {'precision': 0.892934034725354, 'recall': 0.8924967529223699, 'f1-score': 0.8926639632367819, 'support': 30027.0}  |
+| No log        | 5.0   | 205  | 0.2866          | {'precision': 0.6352941176470588, 'recall': 0.5848375451263538, 'f1-score': 0.6090225563909775, 'support': 277.0}  | {'precision': 0.8382352941176471, 'recall': 0.8085106382978723, 'f1-score': 0.8231046931407943, 'support': 141.0}  | {'precision': 0.8605697151424287, 'recall': 0.8954758190327613, 'f1-score': 0.8776758409785932, 'support': 641.0} | {'precision': 0.6380439868975198, 'recall': 0.668546212306938, 'f1-score': 0.6529390638094098, 'support': 4079.0}    | {'precision': 0.8673106253177427, 'recall': 0.8358647721705047, 'f1-score': 0.8512974051896207, 'support': 2041.0} | {'precision': 0.9037239675255913, 'recall': 0.8940200785683108, 'f1-score': 0.8988458331504806, 'support': 11455.0} | {'precision': 0.9997367497367498, 'recall': 1.0, 'f1-score': 0.9998683575409189, 'support': 11393.0}                | 0.8964   | {'precision': 0.8204163509121054, 'recall': 0.8124650093575345, 'f1-score': 0.8161076786001136, 'support': 30027.0} | {'precision': 0.8978823752306275, 'recall': 0.8964265494388384, 'f1-score': 0.8970580772435964, 'support': 30027.0} |
+| No log        | 6.0   | 246  | 0.2945          | {'precision': 0.6460176991150443, 'recall': 0.5270758122743683, 'f1-score': 0.58051689860835, 'support': 277.0}    | {'precision': 0.8646616541353384, 'recall': 0.8156028368794326, 'f1-score': 0.8394160583941607, 'support': 141.0}  | {'precision': 0.8371428571428572, 'recall': 0.9141965678627145, 'f1-score': 0.8739746457867263, 'support': 641.0} | {'precision': 0.6458272753707474, 'recall': 0.5444962000490317, 'f1-score': 0.5908486299547753, 'support': 4079.0}   | {'precision': 0.8827444956477215, 'recall': 0.8446839784419402, 'f1-score': 0.8632949424136204, 'support': 2041.0} | {'precision': 0.8662890913568086, 'recall': 0.9213443910955914, 'f1-score': 0.8929689483035789, 'support': 11455.0} | {'precision': 0.9999122268059335, 'recall': 0.9999122268059335, 'f1-score': 0.9999122268059335, 'support': 11393.0} | 0.8905   | {'precision': 0.8203707570820643, 'recall': 0.7953302876298588, 'f1-score': 0.805847478609592, 'support': 30027.0}  | {'precision': 0.8854972285164366, 'recall': 0.8904652479435174, 'f1-score': 0.886948227760569, 'support': 30027.0}  |
+| No log        | 7.0   | 287  | 0.2899          | {'precision': 0.657258064516129, 'recall': 0.5884476534296029, 'f1-score': 0.620952380952381, 'support': 277.0}    | {'precision': 0.8380281690140845, 'recall': 0.8439716312056738, 'f1-score': 0.8409893992932863, 'support': 141.0}  | {'precision': 0.8622754491017964, 'recall': 0.8985959438377535, 'f1-score': 0.880061115355233, 'support': 641.0}  | {'precision': 0.6403553299492386, 'recall': 0.6185339544005883, 'f1-score': 0.6292555181444071, 'support': 4079.0}   | {'precision': 0.8763931104356636, 'recall': 0.8476237138657521, 'f1-score': 0.8617683686176836, 'support': 2041.0} | {'precision': 0.8874882085584427, 'recall': 0.903448275862069, 'f1-score': 0.8953971275307147, 'support': 11455.0}  | {'precision': 0.9999122345093909, 'recall': 1.0, 'f1-score': 0.9999561153289156, 'support': 11393.0}                | 0.8943   | {'precision': 0.8231015094406778, 'recall': 0.8143744532287771, 'f1-score': 0.8183400036032316, 'support': 30027.0} | {'precision': 0.8929245767023277, 'recall': 0.8942951343790588, 'f1-score': 0.8935151399218458, 'support': 30027.0} |
+
+
+### Framework versions
+
+- Transformers 4.37.2
+- Pytorch 2.2.0+cu121
+- Datasets 2.17.0
+- Tokenizers 0.15.2

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:28df5a2bc8dadfc69bc92dfbda4f08b5c77ec0af99269e675f99e3b94f5a3aab
+oid sha256:35f233b1623da55ad7b04639e7d205739d8eecf5e918b69b3247079838ae8fb9
 size 592330980