yesj1234's picture
Upload folder using huggingface_hub
4f0b01d
metadata
language:
  - ja
  - ko
base_model: facebook/mbart-large-50-many-to-many-mmt
tags:
  - generated_from_trainer
metrics:
  - bleu
model-index:
  - name: jako_mbartLarge_6p
    results: []

jako_mbartLarge_6p

This model is a fine-tuned version of facebook/mbart-large-50-many-to-many-mmt on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.1470
  • Bleu: 21.2511
  • Gen Len: 19.566

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 5e-05
  • train_batch_size: 4
  • eval_batch_size: 4
  • seed: 42
  • distributed_type: multi-GPU
  • num_devices: 4
  • total_train_batch_size: 16
  • total_eval_batch_size: 16
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 300
  • num_epochs: 10

Training results

Training Loss Epoch Step Validation Loss Bleu Gen Len
1.4498 0.48 1000 1.3303 16.7947 20.0778
1.263 0.96 2000 1.1884 18.6214 20.6673
0.8999 1.44 3000 1.1702 18.9014 19.4064
0.827 1.92 4000 1.1470 21.2511 19.566
0.5948 2.4 5000 1.2115 20.5498 19.3941
0.5404 2.88 6000 1.2167 21.2187 19.4357
0.3842 3.36 7000 1.2711 20.7732 19.3493
0.3419 3.84 8000 1.2910 20.7001 19.3106

Framework versions

  • Transformers 4.34.1
  • Pytorch 2.1.0+cu121
  • Datasets 2.14.6
  • Tokenizers 0.14.1