Hugging Face's logo

satyanshu404
/
bart-large-cnn-finetuned-Kaggle-Science-LLM

Text2Text Generation
Transformers
PyTorch
bart
Generated from Trainer
Inference Endpoints
Model card Files Community
satyanshu404 commited on
Commit
ebea5d8
•
1 Parent(s): 2f0f70f

End of training

Browse files
Files changed (2) hide show
  1. README.md +113 -0
  2. generation_config.json +15 -0
README.md ADDED
@@ -0,0 +1,113 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: mit
3
+ base_model: facebook/bart-large-cnn
4
+ tags:
5
+ - generated_from_trainer
6
+ metrics:
7
+ - rouge
8
+ model-index:
9
+ - name: bart-large-cnn-finetuned-Kaggle-Science-LLM
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # bart-large-cnn-finetuned-Kaggle-Science-LLM
17
+
18
+ This model is a fine-tuned version of [facebook/bart-large-cnn](https://huggingface.co/facebook/bart-large-cnn) on the None dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 6.4896
21
+ - Rouge1: 29.4886
22
+ - Rouge2: 10.2696
23
+ - Rougel: 22.611
24
+ - Rougelsum: 23.6936
25
+ - Gen Len: 70.1
26
+
27
+ ## Model description
28
+
29
+ More information needed
30
+
31
+ ## Intended uses & limitations
32
+
33
+ More information needed
34
+
35
+ ## Training and evaluation data
36
+
37
+ More information needed
38
+
39
+ ## Training procedure
40
+
41
+ ### Training hyperparameters
42
+
43
+ The following hyperparameters were used during training:
44
+ - learning_rate: 2e-05
45
+ - train_batch_size: 2
46
+ - eval_batch_size: 2
47
+ - seed: 42
48
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
49
+ - lr_scheduler_type: linear
50
+ - num_epochs: 50
51
+
52
+ ### Training results
53
+
54
+ | Training Loss | Epoch | Step | Validation Loss | Rouge1 | Rouge2 | Rougel | Rougelsum | Gen Len |
55
+ |:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|:-------:|:---------:|:-------:|
56
+ | No log | 1.0 | 90 | 2.9814 | 32.5407 | 12.8638 | 25.9593 | 28.0874 | 66.05 |
57
+ | No log | 2.0 | 180 | 3.1081 | 33.6875 | 13.0896 | 25.2244 | 26.9945 | 68.25 |
58
+ | No log | 3.0 | 270 | 3.4845 | 33.889 | 12.8396 | 26.2138 | 28.2817 | 70.55 |
59
+ | No log | 4.0 | 360 | 3.8911 | 31.8492 | 12.0458 | 23.4026 | 25.8547 | 66.25 |
60
+ | No log | 5.0 | 450 | 4.3530 | 31.2083 | 11.0996 | 23.9196 | 26.1564 | 72.25 |
61
+ | 1.4121 | 6.0 | 540 | 4.4582 | 29.7758 | 11.1798 | 22.9812 | 24.9141 | 72.2 |
62
+ | 1.4121 | 7.0 | 630 | 4.5299 | 30.3925 | 11.41 | 23.9357 | 25.4386 | 74.15 |
63
+ | 1.4121 | 8.0 | 720 | 5.0756 | 30.1282 | 10.1879 | 22.5263 | 24.3294 | 71.05 |
64
+ | 1.4121 | 9.0 | 810 | 5.2213 | 29.1958 | 11.9758 | 22.9344 | 25.3243 | 70.95 |
65
+ | 1.4121 | 10.0 | 900 | 5.0236 | 32.2902 | 12.9557 | 24.9154 | 26.9866 | 71.85 |
66
+ | 1.4121 | 11.0 | 990 | 5.2231 | 29.9105 | 11.4629 | 22.5421 | 24.7261 | 73.15 |
67
+ | 0.1808 | 12.0 | 1080 | 5.4899 | 30.6426 | 10.8586 | 23.0649 | 25.4052 | 69.35 |
68
+ | 0.1808 | 13.0 | 1170 | 5.5205 | 31.4239 | 12.4297 | 24.2742 | 25.8058 | 64.9 |
69
+ | 0.1808 | 14.0 | 1260 | 5.4710 | 31.3377 | 11.5225 | 23.4415 | 25.9487 | 68.3 |
70
+ | 0.1808 | 15.0 | 1350 | 5.3894 | 30.5681 | 11.3301 | 22.5992 | 25.0445 | 67.1 |
71
+ | 0.1808 | 16.0 | 1440 | 5.7293 | 30.7485 | 10.2947 | 23.2461 | 25.1156 | 67.8 |
72
+ | 0.0634 | 17.0 | 1530 | 5.8342 | 27.8846 | 9.4002 | 20.5223 | 22.8928 | 73.7 |
73
+ | 0.0634 | 18.0 | 1620 | 5.7280 | 31.3703 | 12.7091 | 24.947 | 27.6756 | 68.7 |
74
+ | 0.0634 | 19.0 | 1710 | 6.0204 | 29.311 | 10.8717 | 22.2206 | 23.6151 | 66.05 |
75
+ | 0.0634 | 20.0 | 1800 | 5.8662 | 30.3449 | 10.9645 | 22.7105 | 25.3131 | 75.6 |
76
+ | 0.0634 | 21.0 | 1890 | 6.0514 | 29.4108 | 10.9479 | 22.1319 | 23.8446 | 70.6 |
77
+ | 0.0634 | 22.0 | 1980 | 5.9087 | 30.1637 | 10.7748 | 21.7979 | 23.8345 | 71.6 |
78
+ | 0.0281 | 23.0 | 2070 | 6.1406 | 30.3179 | 11.0906 | 23.2057 | 24.9556 | 69.65 |
79
+ | 0.0281 | 24.0 | 2160 | 6.0541 | 29.7931 | 11.492 | 22.7251 | 24.4958 | 68.9 |
80
+ | 0.0281 | 25.0 | 2250 | 6.4349 | 29.6705 | 11.3079 | 22.1845 | 24.0782 | 68.2 |
81
+ | 0.0281 | 26.0 | 2340 | 6.2949 | 30.3573 | 9.7319 | 22.8766 | 25.5102 | 68.65 |
82
+ | 0.0281 | 27.0 | 2430 | 6.3606 | 30.2358 | 10.7457 | 22.9097 | 24.7486 | 69.8 |
83
+ | 0.0167 | 28.0 | 2520 | 6.2235 | 29.131 | 11.0196 | 23.0364 | 24.7254 | 69.0 |
84
+ | 0.0167 | 29.0 | 2610 | 6.2203 | 30.0767 | 10.4042 | 23.0845 | 24.5571 | 71.15 |
85
+ | 0.0167 | 30.0 | 2700 | 6.3899 | 29.524 | 11.0226 | 22.7426 | 24.7137 | 71.45 |
86
+ | 0.0167 | 31.0 | 2790 | 6.4216 | 29.9921 | 11.1592 | 22.7774 | 25.4653 | 70.35 |
87
+ | 0.0167 | 32.0 | 2880 | 6.4758 | 29.4138 | 10.1446 | 22.5501 | 24.4203 | 68.0 |
88
+ | 0.0167 | 33.0 | 2970 | 6.4529 | 30.7129 | 9.9512 | 23.3078 | 25.1444 | 70.1 |
89
+ | 0.0086 | 34.0 | 3060 | 6.3910 | 32.0673 | 11.8157 | 24.4371 | 26.4378 | 67.4 |
90
+ | 0.0086 | 35.0 | 3150 | 6.4725 | 31.0417 | 11.8642 | 23.9718 | 25.9358 | 65.5 |
91
+ | 0.0086 | 36.0 | 3240 | 6.5413 | 31.2471 | 11.9972 | 24.537 | 25.6679 | 66.6 |
92
+ | 0.0086 | 37.0 | 3330 | 6.6040 | 30.6614 | 11.4845 | 23.6335 | 26.3165 | 72.15 |
93
+ | 0.0086 | 38.0 | 3420 | 6.4808 | 30.1209 | 10.4855 | 22.7931 | 24.9675 | 74.75 |
94
+ | 0.0053 | 39.0 | 3510 | 6.4196 | 29.9709 | 11.1147 | 23.3882 | 25.1429 | 73.3 |
95
+ | 0.0053 | 40.0 | 3600 | 6.4798 | 32.6666 | 11.6476 | 24.0167 | 25.8167 | 67.7 |
96
+ | 0.0053 | 41.0 | 3690 | 6.4364 | 31.7081 | 11.4081 | 23.8924 | 25.3477 | 67.35 |
97
+ | 0.0053 | 42.0 | 3780 | 6.4463 | 31.371 | 11.3334 | 23.8642 | 25.5894 | 67.85 |
98
+ | 0.0053 | 43.0 | 3870 | 6.4507 | 29.6148 | 11.0601 | 22.5613 | 24.2758 | 70.95 |
99
+ | 0.0053 | 44.0 | 3960 | 6.5410 | 30.9704 | 10.054 | 22.8276 | 25.1106 | 66.25 |
100
+ | 0.0036 | 45.0 | 4050 | 6.4484 | 30.6993 | 10.2855 | 22.8241 | 25.1591 | 69.3 |
101
+ | 0.0036 | 46.0 | 4140 | 6.4579 | 29.6269 | 10.353 | 21.9677 | 23.4709 | 71.15 |
102
+ | 0.0036 | 47.0 | 4230 | 6.4931 | 29.8756 | 10.4957 | 23.039 | 24.2656 | 69.0 |
103
+ | 0.0036 | 48.0 | 4320 | 6.4831 | 29.6629 | 10.0869 | 22.8167 | 24.0125 | 70.35 |
104
+ | 0.0036 | 49.0 | 4410 | 6.4871 | 29.908 | 10.3116 | 22.9103 | 24.0365 | 71.9 |
105
+ | 0.0023 | 50.0 | 4500 | 6.4896 | 29.4886 | 10.2696 | 22.611 | 23.6936 | 70.1 |
106
+
107
+
108
+ ### Framework versions
109
+
110
+ - Transformers 4.33.3
111
+ - Pytorch 2.0.1+cu118
112
+ - Datasets 2.14.5
113
+ - Tokenizers 0.13.3
generation_config.json ADDED
@@ -0,0 +1,15 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ {
2
+ "bos_token_id": 0,
3
+ "decoder_start_token_id": 2,
4
+ "early_stopping": true,
5
+ "eos_token_id": 2,
6
+ "forced_bos_token_id": 0,
7
+ "forced_eos_token_id": 2,
8
+ "length_penalty": 2.0,
9
+ "max_length": 142,
10
+ "min_length": 56,
11
+ "no_repeat_ngram_size": 3,
12
+ "num_beams": 4,
13
+ "pad_token_id": 1,
14
+ "transformers_version": "4.33.3"
15
+ }