bart_CNN_NLP

This model is a fine-tuned version of facebook/bart-large-cnn on the None dataset. It achieves the following results on the evaluation set:

• Loss: 2.9587
• Rouge1: 44.3648
• Rouge2: 27.4695
• Rougel: 40.784
• Rougelsum: 40.8715
• Gen Len: 6433791.8444

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 4
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum	Gen Len
3.4531	0.4	40	3.1797	51.2851	31.4488	47.5636	47.6331	6433791.8778
2.92	0.8	80	2.9022	44.4735	27.5057	40.8813	40.9664	6433791.8444
2.7525	1.2	120	2.7883	50.1082	32.1501	46.2518	46.3502	6433791.8667
2.5508	1.6	160	2.7864	46.4768	29.626	42.7412	42.8483	6433791.8667
2.6446	2.0	200	2.7885	47.0824	29.892	43.3584	43.4973	6433791.8556
2.2677	2.4	240	2.8746	45.4243	28.5383	41.5278	41.6177	6433791.8778
2.3926	2.8	280	2.8450	45.2873	28.1695	41.6031	41.7244	6433791.9667
2.0455	3.2	320	2.9505	48.4876	30.0367	44.5252	44.662	6433791.8778
2.0197	3.6	360	2.9869	44.687	27.4827	41.0148	41.1126	6433791.8556
2.0922	4.0	400	2.9587	44.3648	27.4695	40.784	40.8715	6433791.8444

Framework versions

• Transformers 4.39.3
• Pytorch 2.1.2
• Datasets 2.18.0
• Tokenizers 0.15.2