pegasus-x-large-finetuned-samsum1000

This model is a fine-tuned version of google/pegasus-x-large on the samsum dataset. It achieves the following results on the evaluation set:

• Loss: 1.4802
• Rouge1: 46.6996
• Rouge2: 21.5586
• Rougel: 38.1002
• Rougelsum: 41.42

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5.6e-05
• train_batch_size: 2
• eval_batch_size: 1
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Rouge1	Rouge2	Rougel	Rougelsum
1.7681	1.0	500	1.4689	47.1766	21.8869	38.8854	42.9534
1.4626	2.0	1000	1.4781	46.6978	20.786	37.764	41.2028
1.3591	3.0	1500	1.4804	47.1756	21.8821	38.2072	41.6812
1.3466	4.0	2000	1.4804	46.9411	21.5169	38.18	41.471
1.3464	5.0	2500	1.4803	46.8083	21.5333	38.1539	41.4872
1.3353	6.0	3000	1.4804	46.6675	21.1336	37.7059	41.0869
1.3483	7.0	3500	1.4803	46.6768	21.1916	37.7642	41.1696
1.3536	8.0	4000	1.4804	46.7311	21.5169	38.057	41.42
1.3533	9.0	4500	1.4802	46.6403	21.529	37.9922	41.3437
1.3469	10.0	5000	1.4802	46.6996	21.5586	38.1002	41.42

Framework versions

• Transformers 4.37.1
• Pytorch 2.1.0+cu121
• Datasets 2.16.1
• Tokenizers 0.15.1