metadata
license: mit
base_model: facebook/bart-large-cnn
tags:
- generated_from_trainer
model-index:
- name: bart_sum_samsum
results: []
bart_sum_samsum
This model is a fine-tuned version of facebook/bart-large-cnn on an unknown dataset. It achieves the following results on the evaluation set:
- Loss: 2.5320
- Gen Len: 59.9242
- Rouge Score: {'rouge1': 0.3935658688306535, 'rouge2': 0.18713851540657486, 'rougeL': 0.29574644161280017, 'rougeLsum': 0.3606436542704101}
- Bleu Score: {'bleu': 0.10800411600387674, 'precisions': [0.2944046763926386, 0.13710024017191252, 0.07618039600382064, 0.044252221841293286], 'brevity_penalty': 1.0, 'length_ratio': 2.163959907809401, 'translation_length': 40373, 'reference_length': 18657}
- Bleurt Score: -0.4998
- Bert Score: [0.8805868625640869, 0.9189654588699341, 0.899208664894104]
Model description
More information needed
Intended uses & limitations
More information needed
Training and evaluation data
More information needed
Training procedure
Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 0.0002
- train_batch_size: 16
- eval_batch_size: 16
- seed: 42
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- num_epochs: 5
Training results
| Training Loss | Epoch | Step | Validation Loss | Gen Len | Rouge Score | Bleu Score | Bleurt Score | Bert Score |
|---|---|---|---|---|---|---|---|---|
| 1.9517 | 1.0 | 921 | 1.8653 | 59.8374 | {'rouge1': 0.38519198024299967, 'rouge2': 0.18637611248242514, 'rougeL': 0.29114807190727665, 'rougeLsum': 0.35950287045215523} | {'bleu': 0.10947202918144075, 'precisions': [0.2891732184886574, 0.1408997955010225, 0.07921257375593964, 0.04449898623412656], 'brevity_penalty': 1.0, 'length_ratio': 2.1406442622072146, 'translation_length': 39938, 'reference_length': 18657} | -0.5574 | [0.881794273853302, 0.914982795715332, 0.897921621799469] |
| 1.4162 | 2.0 | 1842 | 2.1673 | 60.6736 | {'rouge1': 0.3824027985681461, 'rouge2': 0.17720440481192257, 'rougeL': 0.27951993033831063, 'rougeLsum': 0.3523751309023303} | {'bleu': 0.10292900287115767, 'precisions': [0.29144708090182264, 0.13358367689924108, 0.07251160668759896, 0.03975854026615448], 'brevity_penalty': 1.0, 'length_ratio': 2.084954708688428, 'translation_length': 38899, 'reference_length': 18657} | -0.7567 | [0.873441755771637, 0.9113098978996277, 0.8918185234069824] |
| 0.9763 | 3.0 | 2763 | 1.8854 | 59.8851 | {'rouge1': 0.3925367542901428, 'rouge2': 0.19030742072418566, 'rougeL': 0.29557020575264703, 'rougeLsum': 0.36302164503856826} | {'bleu': 0.11050318220968344, 'precisions': [0.29364664926022627, 0.14059446150722135, 0.0786956634438425, 0.04589391170784672], 'brevity_penalty': 1.0, 'length_ratio': 2.1554912365332046, 'translation_length': 40215, 'reference_length': 18657} | -0.5280 | [0.880211353302002, 0.9188302755355835, 0.8989349007606506] |
| 0.5749 | 4.0 | 3684 | 2.1209 | 59.8313 | {'rouge1': 0.39413787163188574, 'rouge2': 0.18797763014604468, 'rougeL': 0.29824353058090336, 'rougeLsum': 0.36387927887558746} | {'bleu': 0.10944201950995913, 'precisions': [0.2954957640803955, 0.1391474146019831, 0.07730156674867279, 0.045135857343175385], 'brevity_penalty': 1.0, 'length_ratio': 2.1574208072037306, 'translation_length': 40251, 'reference_length': 18657} | -0.5075 | [0.8815322518348694, 0.9193716049194336, 0.89988774061203] |
| 0.2765 | 5.0 | 4605 | 2.5320 | 59.9242 | {'rouge1': 0.3935658688306535, 'rouge2': 0.18713851540657486, 'rougeL': 0.29574644161280017, 'rougeLsum': 0.3606436542704101} | {'bleu': 0.10800411600387674, 'precisions': [0.2944046763926386, 0.13710024017191252, 0.07618039600382064, 0.044252221841293286], 'brevity_penalty': 1.0, 'length_ratio': 2.163959907809401, 'translation_length': 40373, 'reference_length': 18657} | -0.4998 | [0.8805868625640869, 0.9189654588699341, 0.899208664894104] |
Framework versions
- Transformers 4.31.0
- Pytorch 2.0.1+cu118
- Datasets 2.10.0
- Tokenizers 0.13.3