longformer-sep_tok_full_labels

This model is a fine-tuned version of allenai/longformer-base-4096 on the stab-gurevych-essays dataset. It achieves the following results on the evaluation set:

• Loss: 0.2775
• B-claim: {'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0}
• B-majorclaim: {'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0}
• B-premise: {'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0}
• I-claim: {'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0}
• I-majorclaim: {'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0}
• I-premise: {'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0}
• O: {'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0}
• Accuracy: 0.8874
• Macro avg: {'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0}
• Weighted avg: {'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0}

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 5

Training results

Training Loss	Epoch	Step	Validation Loss	B-claim	B-majorclaim	B-premise	I-claim	I-majorclaim	I-premise	O	Accuracy	Macro avg	Weighted avg
No log	1.0	41	0.4487	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}	{'precision': 0.7102510460251046, 'recall': 0.9590395480225988, 'f1-score': 0.8161057692307693, 'support': 708.0}	{'precision': 0.5242566510172144, 'recall': 0.0722917565817868, 'f1-score': 0.1270623933244832, 'support': 4634.0}	{'precision': 0.635728952772074, 'recall': 0.6736292428198434, 'f1-score': 0.6541305725755335, 'support': 2298.0}	{'precision': 0.7685153090699018, 'recall': 0.9773712438468886, 'f1-score': 0.8604508262992788, 'support': 13611.0}	{'precision': 0.9707444699912788, 'recall': 0.9973120469169993, 'f1-score': 0.9838489353153878, 'support': 12277.0}	0.8279	{'precision': 0.5156423469822248, 'recall': 0.5256634054554452, 'f1-score': 0.49165692810649325, 'support': 33953.0}	{'precision': 0.7884799553707518, 'recall': 0.827879716078108, 'f1-score': 0.7793158674251499, 'support': 33953.0}
No log	2.0	82	0.3651	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 284.0}	{'precision': 0.0, 'recall': 0.0, 'f1-score': 0.0, 'support': 141.0}	{'precision': 0.6355475763016158, 'recall': 1.0, 'f1-score': 0.7771679473106476, 'support': 708.0}	{'precision': 0.5581831831831832, 'recall': 0.3208890807078118, 'f1-score': 0.40750890654973965, 'support': 4634.0}	{'precision': 0.888728323699422, 'recall': 0.5352480417754569, 'f1-score': 0.66811515480717, 'support': 2298.0}	{'precision': 0.8056312443233424, 'recall': 0.9775181838219088, 'f1-score': 0.8832901812387971, 'support': 13611.0}	{'precision': 0.9999184139675288, 'recall': 0.9982894844017268, 'f1-score': 0.9991032852368142, 'support': 12277.0}	0.8537	{'precision': 0.5554298202107274, 'recall': 0.5474206843867007, 'f1-score': 0.5335979250204527, 'support': 33953.0}	{'precision': 0.8341039518604557, 'recall': 0.8537095396577622, 'f1-score': 0.832398123732452, 'support': 33953.0}
No log	3.0	123	0.2896	{'precision': 0.47393364928909953, 'recall': 0.352112676056338, 'f1-score': 0.40404040404040403, 'support': 284.0}	{'precision': 0.9333333333333333, 'recall': 0.2978723404255319, 'f1-score': 0.45161290322580644, 'support': 141.0}	{'precision': 0.7856328392246295, 'recall': 0.9731638418079096, 'f1-score': 0.8694006309148264, 'support': 708.0}	{'precision': 0.6642079381805409, 'recall': 0.4080707811825637, 'f1-score': 0.5055473867130063, 'support': 4634.0}	{'precision': 0.7170077628793226, 'recall': 0.8842471714534378, 'f1-score': 0.7918939984411536, 'support': 2298.0}	{'precision': 0.8606260075228371, 'recall': 0.9413709499669385, 'f1-score': 0.8991894452436927, 'support': 13611.0}	{'precision': 1.0, 'recall': 0.9978822187830904, 'f1-score': 0.9989399869536856, 'support': 12277.0}	0.8782	{'precision': 0.7763916472042519, 'recall': 0.6935314256679729, 'f1-score': 0.7029463936475108, 'support': 33953.0}	{'precision': 0.8699976208166521, 'recall': 0.8782140017082437, 'f1-score': 0.867649200314498, 'support': 33953.0}
No log	4.0	164	0.2798	{'precision': 0.5757575757575758, 'recall': 0.5352112676056338, 'f1-score': 0.5547445255474452, 'support': 284.0}	{'precision': 0.9054054054054054, 'recall': 0.475177304964539, 'f1-score': 0.6232558139534884, 'support': 141.0}	{'precision': 0.8377358490566038, 'recall': 0.940677966101695, 'f1-score': 0.8862275449101796, 'support': 708.0}	{'precision': 0.6079838528818121, 'recall': 0.5850237375917134, 'f1-score': 0.596282854943363, 'support': 4634.0}	{'precision': 0.8411037107516651, 'recall': 0.7693646649260226, 'f1-score': 0.8036363636363636, 'support': 2298.0}	{'precision': 0.8822353864820498, 'recall': 0.9081625156123724, 'f1-score': 0.8950112229382376, 'support': 13611.0}	{'precision': 1.0, 'recall': 0.9976378594119084, 'f1-score': 0.9988175331294598, 'support': 12277.0}	0.8828	{'precision': 0.8071745400478731, 'recall': 0.7444650451734122, 'f1-score': 0.7654251227226482, 'support': 33953.0}	{'precision': 0.881207953399647, 'recall': 0.882779135864283, 'f1-score': 0.8814327847290655, 'support': 33953.0}
No log	5.0	205	0.2775	{'precision': 0.6083333333333333, 'recall': 0.5140845070422535, 'f1-score': 0.5572519083969466, 'support': 284.0}	{'precision': 0.88, 'recall': 0.624113475177305, 'f1-score': 0.7302904564315352, 'support': 141.0}	{'precision': 0.8373266078184111, 'recall': 0.9378531073446328, 'f1-score': 0.8847435043304464, 'support': 708.0}	{'precision': 0.6361367606688295, 'recall': 0.5500647388864911, 'f1-score': 0.5899780118041893, 'support': 4634.0}	{'precision': 0.8413284132841329, 'recall': 0.793733681462141, 'f1-score': 0.8168383340797134, 'support': 2298.0}	{'precision': 0.8758342602892102, 'recall': 0.9255749026522665, 'f1-score': 0.9000178603322022, 'support': 13611.0}	{'precision': 1.0, 'recall': 0.9986967500203633, 'f1-score': 0.999347950118184, 'support': 12277.0}	0.8874	{'precision': 0.8112799107705595, 'recall': 0.7634458803693505, 'f1-score': 0.782638289356174, 'support': 33953.0}	{'precision': 0.8826579231427218, 'recall': 0.8874031749771744, 'f1-score': 0.8840991775809467, 'support': 33953.0}

Framework versions

• Transformers 4.45.2
• Pytorch 2.5.0+cu124
• Datasets 2.19.1
• Tokenizers 0.20.1