|
--- |
|
license: mit |
|
base_model: facebook/bart-large-cnn |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: bart_sum_samsum |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# bart_sum_samsum |
|
|
|
This model is a fine-tuned version of [facebook/bart-large-cnn](https://huggingface.co./facebook/bart-large-cnn) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 2.5320 |
|
- Gen Len: 59.9242 |
|
- Rouge Score: {'rouge1': 0.3935658688306535, 'rouge2': 0.18713851540657486, 'rougeL': 0.29574644161280017, 'rougeLsum': 0.3606436542704101} |
|
- Bleu Score: {'bleu': 0.10800411600387674, 'precisions': [0.2944046763926386, 0.13710024017191252, 0.07618039600382064, 0.044252221841293286], 'brevity_penalty': 1.0, 'length_ratio': 2.163959907809401, 'translation_length': 40373, 'reference_length': 18657} |
|
- Bleurt Score: -0.4998 |
|
- Bert Score: [0.8805868625640869, 0.9189654588699341, 0.899208664894104] |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 0.0002 |
|
- train_batch_size: 16 |
|
- eval_batch_size: 16 |
|
- seed: 42 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 5 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | Gen Len | Rouge Score | Bleu Score | Bleurt Score | Bert Score | |
|
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------------------------------------------------------------------------------------------------------------------------------:|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|:------------:|:-----------------------------------------------------------:| |
|
| 1.9517 | 1.0 | 921 | 1.8653 | 59.8374 | {'rouge1': 0.38519198024299967, 'rouge2': 0.18637611248242514, 'rougeL': 0.29114807190727665, 'rougeLsum': 0.35950287045215523} | {'bleu': 0.10947202918144075, 'precisions': [0.2891732184886574, 0.1408997955010225, 0.07921257375593964, 0.04449898623412656], 'brevity_penalty': 1.0, 'length_ratio': 2.1406442622072146, 'translation_length': 39938, 'reference_length': 18657} | -0.5574 | [0.881794273853302, 0.914982795715332, 0.897921621799469] | |
|
| 1.4162 | 2.0 | 1842 | 2.1673 | 60.6736 | {'rouge1': 0.3824027985681461, 'rouge2': 0.17720440481192257, 'rougeL': 0.27951993033831063, 'rougeLsum': 0.3523751309023303} | {'bleu': 0.10292900287115767, 'precisions': [0.29144708090182264, 0.13358367689924108, 0.07251160668759896, 0.03975854026615448], 'brevity_penalty': 1.0, 'length_ratio': 2.084954708688428, 'translation_length': 38899, 'reference_length': 18657} | -0.7567 | [0.873441755771637, 0.9113098978996277, 0.8918185234069824] | |
|
| 0.9763 | 3.0 | 2763 | 1.8854 | 59.8851 | {'rouge1': 0.3925367542901428, 'rouge2': 0.19030742072418566, 'rougeL': 0.29557020575264703, 'rougeLsum': 0.36302164503856826} | {'bleu': 0.11050318220968344, 'precisions': [0.29364664926022627, 0.14059446150722135, 0.0786956634438425, 0.04589391170784672], 'brevity_penalty': 1.0, 'length_ratio': 2.1554912365332046, 'translation_length': 40215, 'reference_length': 18657} | -0.5280 | [0.880211353302002, 0.9188302755355835, 0.8989349007606506] | |
|
| 0.5749 | 4.0 | 3684 | 2.1209 | 59.8313 | {'rouge1': 0.39413787163188574, 'rouge2': 0.18797763014604468, 'rougeL': 0.29824353058090336, 'rougeLsum': 0.36387927887558746} | {'bleu': 0.10944201950995913, 'precisions': [0.2954957640803955, 0.1391474146019831, 0.07730156674867279, 0.045135857343175385], 'brevity_penalty': 1.0, 'length_ratio': 2.1574208072037306, 'translation_length': 40251, 'reference_length': 18657} | -0.5075 | [0.8815322518348694, 0.9193716049194336, 0.89988774061203] | |
|
| 0.2765 | 5.0 | 4605 | 2.5320 | 59.9242 | {'rouge1': 0.3935658688306535, 'rouge2': 0.18713851540657486, 'rougeL': 0.29574644161280017, 'rougeLsum': 0.3606436542704101} | {'bleu': 0.10800411600387674, 'precisions': [0.2944046763926386, 0.13710024017191252, 0.07618039600382064, 0.044252221841293286], 'brevity_penalty': 1.0, 'length_ratio': 2.163959907809401, 'translation_length': 40373, 'reference_length': 18657} | -0.4998 | [0.8805868625640869, 0.9189654588699341, 0.899208664894104] | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.31.0 |
|
- Pytorch 2.0.1+cu118 |
|
- Datasets 2.10.0 |
|
- Tokenizers 0.13.3 |
|
|
