t5-summarization-headers-50-epochs

This model is a fine-tuned version of google/flan-t5-small on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 2.2125
• Rouge: {'rouge1': 0.4117, 'rouge2': 0.2163, 'rougeL': 0.2158, 'rougeLsum': 0.2158}
• Bert Score: 0.8818
• Bleurt 20: -0.8026
• Gen Len: 14.46

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 7
• eval_batch_size: 7
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 20

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge	Bert Score	Bleurt 20	Gen Len
3.0256	1.0	186	2.6300	{'rouge1': 0.4643, 'rouge2': 0.1902, 'rougeL': 0.1973, 'rougeLsum': 0.1973}	0.8664	-0.8801	15.55
2.734	2.0	372	2.4218	{'rouge1': 0.4489, 'rouge2': 0.2037, 'rougeL': 0.209, 'rougeLsum': 0.209}	0.8737	-0.8686	14.995
2.5147	3.0	558	2.3219	{'rouge1': 0.4363, 'rouge2': 0.1984, 'rougeL': 0.2067, 'rougeLsum': 0.2067}	0.8742	-0.8762	14.69
2.3007	4.0	744	2.2752	{'rouge1': 0.4465, 'rouge2': 0.2043, 'rougeL': 0.2022, 'rougeLsum': 0.2022}	0.8761	-0.8603	14.625
2.1922	5.0	930	2.2331	{'rouge1': 0.425, 'rouge2': 0.2033, 'rougeL': 0.2042, 'rougeLsum': 0.2042}	0.8779	-0.829	14.87
2.1185	6.0	1116	2.2092	{'rouge1': 0.4231, 'rouge2': 0.2096, 'rougeL': 0.2073, 'rougeLsum': 0.2073}	0.8783	-0.8359	14.68
2.0584	7.0	1302	2.1993	{'rouge1': 0.4302, 'rouge2': 0.2114, 'rougeL': 0.2126, 'rougeLsum': 0.2126}	0.8793	-0.8202	15.015
2.0189	8.0	1488	2.1872	{'rouge1': 0.4255, 'rouge2': 0.2086, 'rougeL': 0.2106, 'rougeLsum': 0.2106}	0.879	-0.8359	14.485
1.8933	9.0	1674	2.1967	{'rouge1': 0.4307, 'rouge2': 0.2175, 'rougeL': 0.2165, 'rougeLsum': 0.2165}	0.8821	-0.7803	14.865
1.8859	10.0	1860	2.1905	{'rouge1': 0.4342, 'rouge2': 0.2139, 'rougeL': 0.2193, 'rougeLsum': 0.2193}	0.8828	-0.7683	14.93
1.8395	11.0	2046	2.2006	{'rouge1': 0.42, 'rouge2': 0.2135, 'rougeL': 0.2175, 'rougeLsum': 0.2175}	0.8815	-0.7958	14.485
1.7848	12.0	2232	2.1970	{'rouge1': 0.4309, 'rouge2': 0.2096, 'rougeL': 0.2171, 'rougeLsum': 0.2171}	0.8826	-0.8131	14.51
1.7855	13.0	2418	2.2026	{'rouge1': 0.4218, 'rouge2': 0.2099, 'rougeL': 0.2182, 'rougeLsum': 0.2182}	0.8812	-0.8068	14.555
1.6971	14.0	2604	2.2006	{'rouge1': 0.4035, 'rouge2': 0.2056, 'rougeL': 0.2109, 'rougeLsum': 0.2109}	0.8816	-0.817	14.145
1.7226	15.0	2790	2.2000	{'rouge1': 0.413, 'rouge2': 0.2072, 'rougeL': 0.2145, 'rougeLsum': 0.2145}	0.8818	-0.8106	14.415
1.7164	16.0	2976	2.2067	{'rouge1': 0.4117, 'rouge2': 0.212, 'rougeL': 0.215, 'rougeLsum': 0.215}	0.8815	-0.8198	14.235
1.6908	17.0	3162	2.2061	{'rouge1': 0.4125, 'rouge2': 0.2193, 'rougeL': 0.2154, 'rougeLsum': 0.2154}	0.8814	-0.8089	14.37
1.6865	18.0	3348	2.2088	{'rouge1': 0.4125, 'rouge2': 0.2173, 'rougeL': 0.217, 'rougeLsum': 0.217}	0.8819	-0.807	14.46
1.6225	19.0	3534	2.2127	{'rouge1': 0.4111, 'rouge2': 0.2161, 'rougeL': 0.2123, 'rougeLsum': 0.2123}	0.8815	-0.8039	14.425
1.6304	20.0	3720	2.2125	{'rouge1': 0.4117, 'rouge2': 0.2163, 'rougeL': 0.2158, 'rougeLsum': 0.2158}	0.8818	-0.8026	14.46

Framework versions

• Transformers 4.35.2
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.0