End of training

README.md

@@ -3,6 +3,8 @@ license: apache-2.0
 base_model: google/flan-t5-large
 tags:
 - generated_from_trainer
+metrics:
+- bleu
 model-index:
 - name: flan-t5-large-P-tuning-cpgQA
   results: []
@@ -15,7 +17,9 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [google/flan-t5-large](https://huggingface.co/google/flan-t5-large) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.1216
+- Loss: 0.1423
+- Squad: {'exact_match': 59.63302752293578, 'f1': 82.39001389589451}
+- Bleu: {'bleu': 0.6044954004940006, 'precisions': [0.6696606786427146, 0.6282194848824189, 0.5842839036755386, 0.5432276657060519], 'brevity_penalty': 1.0, 'length_ratio': 1.3012987012987014, 'translation_length': 1002, 'reference_length': 770}
 
 ## Model description
 
@@ -35,24 +39,30 @@ More information needed
 
 The following hyperparameters were used during training:
 - learning_rate: 3e-05
-- train_batch_size: 4
-- eval_batch_size: 4
+- train_batch_size: 8
+- eval_batch_size: 8
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: linear
-- num_epochs: 2
+- num_epochs: 8
 
 ### Training results
 
-| Training Loss | Epoch | Step | Validation Loss |
-|:-------------:|:-----:|:----:|:---------------:|
-| 0.1888        | 1.0   | 215  | 0.1245          |
-| 0.1657        | 2.0   | 430  | 0.1216          |
+| Training Loss | Epoch | Step | Validation Loss | Squad                                                       | Bleu                                                                                                                                                                                                                                          |
+|:-------------:|:-----:|:----:|:---------------:|:-----------------------------------------------------------:|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------:|
+| 0.3136        | 1.0   | 494  | 0.1411          | {'exact_match': 57.79816513761468, 'f1': 81.81283979578464} | {'bleu': 0.5819448175214944, 'precisions': [0.6526946107784432, 0.6080627099664053, 0.5602027883396705, 0.515850144092219], 'brevity_penalty': 1.0, 'length_ratio': 1.363265306122449, 'translation_length': 1002, 'reference_length': 735}   |
+| 0.28          | 2.0   | 988  | 0.1437          | {'exact_match': 57.79816513761468, 'f1': 81.00462660225033} | {'bleu': 0.5717467837073172, 'precisions': [0.6417165668662674, 0.5968645016797313, 0.550063371356147, 0.5072046109510087], 'brevity_penalty': 1.0, 'length_ratio': 1.3707250341997264, 'translation_length': 1002, 'reference_length': 731}  |
+| 0.2351        | 3.0   | 1482 | 0.1436          | {'exact_match': 57.79816513761468, 'f1': 80.92424215489342} | {'bleu': 0.5786949268545348, 'precisions': [0.6477045908183633, 0.6035834266517357, 0.5576679340937896, 0.5144092219020173], 'brevity_penalty': 1.0, 'length_ratio': 1.3431635388739946, 'translation_length': 1002, 'reference_length': 746} |
+| 0.2736        | 4.0   | 1976 | 0.1434          | {'exact_match': 57.79816513761468, 'f1': 80.98182982715996} | {'bleu': 0.5763280403149159, 'precisions': [0.6467065868263473, 0.6024636058230683, 0.5551330798479087, 0.5100864553314121], 'brevity_penalty': 1.0, 'length_ratio': 1.3449664429530201, 'translation_length': 1002, 'reference_length': 745} |
+| 0.2408        | 5.0   | 2470 | 0.1415          | {'exact_match': 57.79816513761468, 'f1': 81.54539470265146} | {'bleu': 0.5849077021683632, 'precisions': [0.653692614770459, 0.6103023516237402, 0.5640050697084917, 0.5201729106628242], 'brevity_penalty': 1.0, 'length_ratio': 1.3306772908366533, 'translation_length': 1002, 'reference_length': 753}  |
+| 0.2513        | 6.0   | 2964 | 0.1426          | {'exact_match': 59.63302752293578, 'f1': 81.88361818381664} | {'bleu': 0.5934811323519371, 'precisions': [0.6606786427145709, 0.6181410974244121, 0.5728770595690748, 0.5302593659942363], 'brevity_penalty': 1.0, 'length_ratio': 1.323645970937913, 'translation_length': 1002, 'reference_length': 757}  |
+| 0.2231        | 7.0   | 3458 | 0.1419          | {'exact_match': 59.63302752293578, 'f1': 82.39001389589451} | {'bleu': 0.6044954004940006, 'precisions': [0.6696606786427146, 0.6282194848824189, 0.5842839036755386, 0.5432276657060519], 'brevity_penalty': 1.0, 'length_ratio': 1.3012987012987014, 'translation_length': 1002, 'reference_length': 770} |
+| 0.2575        | 8.0   | 3952 | 0.1423          | {'exact_match': 59.63302752293578, 'f1': 82.39001389589451} | {'bleu': 0.6044954004940006, 'precisions': [0.6696606786427146, 0.6282194848824189, 0.5842839036755386, 0.5432276657060519], 'brevity_penalty': 1.0, 'length_ratio': 1.3012987012987014, 'translation_length': 1002, 'reference_length': 770} |
 
 
 ### Framework versions
 
-- Transformers 4.32.1
-- Pytorch 2.0.1+cu118
-- Datasets 2.14.4
+- Transformers 4.33.0
+- Pytorch 2.0.0
+- Datasets 2.1.0
 - Tokenizers 0.13.3

adapter_config.json

@@ -2,14 +2,14 @@
   "auto_mapping": null,
   "base_model_name_or_path": "google/flan-t5-large",
   "encoder_dropout": 0.0,
-  "encoder_hidden_size": 1024,
+  "encoder_hidden_size": 512,
   "encoder_num_layers": 2,
   "encoder_reparameterization_type": "MLP",
   "inference_mode": true,
   "num_attention_heads": 16,
   "num_layers": 24,
   "num_transformer_submodules": 2,
-  "num_virtual_tokens": 32,
+  "num_virtual_tokens": 20,
   "peft_type": "P_TUNING",
   "revision": null,
   "task_type": "SEQ_2_SEQ_LM",

adapter_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d6ad6cf6e1c5691e06f395dd7feec208d03062829a2c431db5bf476731cf5449
-size 262973
+oid sha256:5e4ef7462f429e7dc25d2c171b93bbe77dc3dc0afde66274ab11ca6d537247c7
+size 164605

spiece.model

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d60acb128cf7b7f2536e8f38a5b18a05535c9e14c7a355904270e15b0945ea86
+size 791656

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:80bd973e4276b0e39c5425190497a1508f752cf6282ab411c0fc7d8f30b19019
+oid sha256:e7d7726f0105a218d7812459344dce1cd3b1ba20cf586115769f5ccdc324c947
 size 4219