Hugging Face's logo

yasmineee
/
NLLB_DoRA

PEFT
Safetensors
Generated from Trainer
Model card Files Community
NLLB_DoRA / README.md
yasmineee's picture
yasmineee
finetune-NLLB-600M-on-opus100-Ar2En-with-Dora
880b4df verified
preview code
|
raw
history blame
1.93 kB
metadata
base_model: facebook/nllb-200-distilled-600M
library_name: peft
license: cc-by-nc-4.0
metrics:
  - bleu
  - rouge
tags:
  - generated_from_trainer
model-index:
  - name: NLLB_DoRA
    results: []

Visualize in Weights & Biases

NLLB_DoRA

This model is a fine-tuned version of facebook/nllb-200-distilled-600M on an unknown dataset. It achieves the following results on the evaluation set:

  • Loss: 1.3271
  • Bleu: 32.6656
  • Rouge: 0.593
  • Gen Len: 17.403

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 2e-05
  • train_batch_size: 2
  • eval_batch_size: 2
  • seed: 42
  • gradient_accumulation_steps: 4
  • total_train_batch_size: 8
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • num_epochs: 3

Training results

Training Loss Epoch Step Validation Loss Bleu Rouge Gen Len
2.722 1.0 875 1.3916 31.7382 0.5849 17.493
1.4579 2.0 1750 1.3379 32.34 0.5931 17.3715
1.4263 3.0 2625 1.3271 32.6656 0.593 17.403

Framework versions

  • PEFT 0.12.0
  • Transformers 4.42.3
  • Pytorch 2.1.2
  • Datasets 2.20.0
  • Tokenizers 0.19.1