mbart-en-id-smaller-indo-amr-generation-translated-nafkhan

This model was trained from scratch on the data dataset. It achieves the following results on the evaluation set:

• Loss: 1.9816
• Bleu: 50.4231
• Gen Len: 8.2667

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-06
• train_batch_size: 5
• eval_batch_size: 5
• seed: 42
• gradient_accumulation_steps: 5
• total_train_batch_size: 25
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: polynomial
• lr_scheduler_warmup_steps: 200
• num_epochs: 640.0
• label_smoothing_factor: 0.1

Training results

Training Loss	Epoch	Step	Validation Loss	Bleu	Gen Len
1.5364	39.7351	3600	2.0474	41.1359	9.5667
1.4794	79.4702	7200	2.0256	44.4125	9.0667
1.3621	119.2053	10800	2.0037	49.121	8.8
1.3278	158.9404	14400	1.9857	52.1498	8.5
1.2906	198.6755	18000	2.0048	48.3044	8.5333
1.2272	238.4106	21600	2.0017	47.3367	8.6
1.2339	278.1457	25200	1.9705	46.5655	8.5667
1.194	317.8808	28800	1.9818	51.4066	8.5
1.1416	357.6159	32400	1.9699	45.7022	8.4333
1.1437	397.3510	36000	1.9692	46.8726	8.2
1.156	437.0861	39600	1.9549	48.7386	8.4333
1.1355	476.8212	43200	1.9726	48.3929	8.5667
1.1246	516.5563	46800	1.9701	47.8897	8.4667
1.1132	556.2914	50400	1.9736	48.9071	8.3667
1.0843	596.0265	54000	1.9765	49.9897	8.4333
1.1211	635.7616	57600	1.9816	50.4231	8.2667

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.20.0
• Tokenizers 0.19.1