t5-summarization-one-shot-better-prompt

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.4802
• Rouge: {'rouge1': 37.3827, 'rouge2': 17.5806, 'rougeL': 20.1333, 'rougeLsum': 20.1333}
• Bert Score: 0.881
• Bleurt 20: -0.8056
• Gen Len: 13.305

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 7
• eval_batch_size: 7
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge	Bert Score	Bleurt 20	Gen Len
2.8911	1.0	172	2.6479	{'rouge1': 44.1146, 'rouge2': 17.635, 'rougeL': 19.7288, 'rougeLsum': 19.7288}	0.8758	-0.8151	14.79
2.7068	2.0	344	2.5580	{'rouge1': 42.4931, 'rouge2': 18.5074, 'rougeL': 19.757, 'rougeLsum': 19.757}	0.8774	-0.8319	14.025
2.4884	3.0	516	2.5123	{'rouge1': 43.5811, 'rouge2': 19.0798, 'rougeL': 20.1143, 'rougeLsum': 20.1143}	0.8794	-0.7737	14.455
2.3827	4.0	688	2.4875	{'rouge1': 42.0721, 'rouge2': 18.4704, 'rougeL': 20.0131, 'rougeLsum': 20.0131}	0.8801	-0.7754	14.175
2.345	5.0	860	2.4596	{'rouge1': 43.7021, 'rouge2': 20.0234, 'rougeL': 20.0962, 'rougeLsum': 20.0962}	0.8809	-0.7379	14.325
2.2438	6.0	1032	2.4466	{'rouge1': 41.0624, 'rouge2': 18.8098, 'rougeL': 19.8672, 'rougeLsum': 19.8672}	0.8803	-0.7893	13.565
2.1878	7.0	1204	2.4505	{'rouge1': 40.3802, 'rouge2': 18.9902, 'rougeL': 20.1633, 'rougeLsum': 20.1633}	0.88	-0.7735	13.26
2.084	8.0	1376	2.4384	{'rouge1': 38.4615, 'rouge2': 18.3148, 'rougeL': 19.61, 'rougeLsum': 19.61}	0.8802	-0.7813	13.235
2.096	9.0	1548	2.4380	{'rouge1': 38.9264, 'rouge2': 18.2137, 'rougeL': 19.7464, 'rougeLsum': 19.7464}	0.8794	-0.8013	13.18
2.0251	10.0	1720	2.4445	{'rouge1': 36.7486, 'rouge2': 17.2998, 'rougeL': 20.0546, 'rougeLsum': 20.0546}	0.8807	-0.8057	12.97
2.0139	11.0	1892	2.4449	{'rouge1': 38.374, 'rouge2': 18.4603, 'rougeL': 20.3794, 'rougeLsum': 20.3794}	0.88	-0.7885	13.415
1.957	12.0	2064	2.4502	{'rouge1': 38.5085, 'rouge2': 18.9339, 'rougeL': 20.2576, 'rougeLsum': 20.2576}	0.8812	-0.7956	13.24
1.8508	13.0	2236	2.4510	{'rouge1': 37.1308, 'rouge2': 17.4541, 'rougeL': 19.7904, 'rougeLsum': 19.7904}	0.8802	-0.7981	13.28
1.9315	14.0	2408	2.4590	{'rouge1': 37.1896, 'rouge2': 17.8766, 'rougeL': 20.0, 'rougeLsum': 20.0}	0.8813	-0.7869	13.205
1.8447	15.0	2580	2.4635	{'rouge1': 38.7071, 'rouge2': 18.5335, 'rougeL': 20.5302, 'rougeLsum': 20.5302}	0.8813	-0.7887	13.505
1.8488	16.0	2752	2.4643	{'rouge1': 38.0242, 'rouge2': 17.8906, 'rougeL': 20.3013, 'rougeLsum': 20.3013}	0.8818	-0.7789	13.23
1.7909	17.0	2924	2.4739	{'rouge1': 37.1561, 'rouge2': 17.7929, 'rougeL': 20.0763, 'rougeLsum': 20.0763}	0.8806	-0.7882	13.22
1.8615	18.0	3096	2.4770	{'rouge1': 37.2891, 'rouge2': 17.6695, 'rougeL': 19.951, 'rougeLsum': 19.951}	0.8803	-0.8131	13.305
1.7938	19.0	3268	2.4796	{'rouge1': 37.1339, 'rouge2': 17.6046, 'rougeL': 20.1063, 'rougeLsum': 20.1063}	0.881	-0.8031	13.26
1.7814	20.0	3440	2.4802	{'rouge1': 37.3827, 'rouge2': 17.5806, 'rougeL': 20.1333, 'rougeLsum': 20.1333}	0.881	-0.8056	13.305

Framework versions

• Transformers 4.37.2
• Pytorch 2.1.0+cu121
• Datasets 2.17.1
• Tokenizers 0.15.2