|
--- |
|
tags: |
|
- generated_from_trainer |
|
model-index: |
|
- name: deberta-v3-large-ddlm |
|
results: [] |
|
--- |
|
|
|
<!-- This model card has been generated automatically according to the information the Trainer had access to. You |
|
should probably proofread and complete it, then remove this comment. --> |
|
|
|
# deberta-v3-large-ddlm |
|
|
|
This model is a fine-tuned version of [microsoft/deberta-v3-large](https://huggingface.co./models/microsoft/deberta-v3-large) on an unknown dataset. |
|
It achieves the following results on the evaluation set: |
|
- Loss: 0.5241 |
|
|
|
## Model description |
|
|
|
More information needed |
|
|
|
## Intended uses & limitations |
|
|
|
More information needed |
|
|
|
## Training and evaluation data |
|
|
|
More information needed |
|
|
|
## Training procedure |
|
|
|
### Training hyperparameters |
|
|
|
The following hyperparameters were used during training: |
|
- learning_rate: 5e-05 |
|
- train_batch_size: 2 |
|
- eval_batch_size: 2 |
|
- seed: 42 |
|
- gradient_accumulation_steps: 64 |
|
- total_train_batch_size: 128 |
|
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08 |
|
- lr_scheduler_type: linear |
|
- num_epochs: 3.0 |
|
|
|
### Training results |
|
|
|
| Training Loss | Epoch | Step | Validation Loss | |
|
|:-------------:|:-----:|:------:|:---------------:| |
|
| 0.9823 | 0.01 | 1000 | 0.9163 | |
|
| 0.8817 | 0.02 | 2000 | 0.9022 | |
|
| 0.9647 | 0.03 | 3000 | 0.8879 | |
|
| 0.8646 | 0.04 | 4000 | 0.8577 | |
|
| 0.9159 | 0.06 | 5000 | 0.8677 | |
|
| 0.8449 | 0.07 | 6000 | 0.8221 | |
|
| 0.8681 | 0.08 | 7000 | 0.8332 | |
|
| 0.8738 | 0.09 | 8000 | 0.8334 | |
|
| 0.8638 | 0.1 | 9000 | 0.8236 | |
|
| 0.9066 | 0.11 | 10000 | 0.8200 | |
|
| 0.8686 | 0.12 | 11000 | 0.8092 | |
|
| 0.7736 | 0.13 | 12000 | 0.8199 | |
|
| 0.8054 | 0.14 | 13000 | 0.7972 | |
|
| 0.8934 | 0.16 | 14000 | 0.7998 | |
|
| 0.7884 | 0.17 | 15000 | 0.7895 | |
|
| 0.8278 | 0.18 | 16000 | 0.7586 | |
|
| 0.8482 | 0.19 | 17000 | 0.7562 | |
|
| 0.8716 | 0.2 | 18000 | 0.7819 | |
|
| 0.8881 | 0.21 | 19000 | 0.7878 | |
|
| 0.8397 | 0.22 | 20000 | 0.7989 | |
|
| 0.811 | 0.23 | 21000 | 0.7846 | |
|
| 0.7762 | 0.24 | 22000 | 0.7753 | |
|
| 0.7778 | 0.25 | 23000 | 0.7878 | |
|
| 0.737 | 0.27 | 24000 | 0.7473 | |
|
| 0.8451 | 0.28 | 25000 | 0.7460 | |
|
| 0.823 | 0.29 | 26000 | 0.7300 | |
|
| 0.7472 | 0.3 | 27000 | 0.7292 | |
|
| 0.8048 | 0.31 | 28000 | 0.7697 | |
|
| 0.7962 | 0.32 | 29000 | 0.7359 | |
|
| 0.8048 | 0.33 | 30000 | 0.7409 | |
|
| 0.8095 | 0.34 | 31000 | 0.7434 | |
|
| 0.7451 | 0.35 | 32000 | 0.7534 | |
|
| 0.6997 | 0.37 | 33000 | 0.7602 | |
|
| 0.8116 | 0.38 | 34000 | 0.7566 | |
|
| 0.7963 | 0.39 | 35000 | 0.7245 | |
|
| 0.786 | 0.4 | 36000 | 0.7311 | |
|
| 0.7991 | 0.41 | 37000 | 0.7230 | |
|
| 0.723 | 0.42 | 38000 | 0.7209 | |
|
| 0.789 | 0.43 | 39000 | 0.7418 | |
|
| 0.7296 | 0.44 | 40000 | 0.7325 | |
|
| 0.7363 | 0.45 | 41000 | 0.7134 | |
|
| 0.758 | 0.47 | 42000 | 0.6948 | |
|
| 0.711 | 0.48 | 43000 | 0.6992 | |
|
| 0.7984 | 0.49 | 44000 | 0.7055 | |
|
| 0.8402 | 0.5 | 45000 | 0.7108 | |
|
| 0.8553 | 0.51 | 46000 | 0.7005 | |
|
| 0.7538 | 0.52 | 47000 | 0.7208 | |
|
| 0.7169 | 0.53 | 48000 | 0.7291 | |
|
| 0.7345 | 0.54 | 49000 | 0.7195 | |
|
| 0.758 | 0.55 | 50000 | 0.6694 | |
|
| 0.7868 | 0.56 | 51000 | 0.6938 | |
|
| 0.6966 | 0.58 | 52000 | 0.6867 | |
|
| 0.7389 | 0.59 | 53000 | 0.6862 | |
|
| 0.7529 | 0.6 | 54000 | 0.7175 | |
|
| 0.7345 | 0.61 | 55000 | 0.6970 | |
|
| 0.766 | 0.62 | 56000 | 0.7017 | |
|
| 0.7043 | 0.63 | 57000 | 0.6916 | |
|
| 0.6474 | 0.64 | 58000 | 0.7129 | |
|
| 0.7456 | 0.65 | 59000 | 0.6802 | |
|
| 0.7512 | 0.66 | 60000 | 0.6951 | |
|
| 0.6816 | 0.68 | 61000 | 0.7072 | |
|
| 0.7206 | 0.69 | 62000 | 0.6967 | |
|
| 0.6439 | 0.7 | 63000 | 0.6798 | |
|
| 0.7309 | 0.71 | 64000 | 0.7163 | |
|
| 0.6925 | 0.72 | 65000 | 0.6794 | |
|
| 0.6833 | 0.73 | 66000 | 0.6637 | |
|
| 0.6643 | 0.74 | 67000 | 0.6855 | |
|
| 0.6433 | 0.75 | 68000 | 0.7035 | |
|
| 0.7595 | 0.76 | 69000 | 0.7008 | |
|
| 0.7214 | 0.78 | 70000 | 0.6618 | |
|
| 0.7111 | 0.79 | 71000 | 0.6850 | |
|
| 0.7375 | 0.8 | 72000 | 0.6909 | |
|
| 0.6779 | 0.81 | 73000 | 0.7042 | |
|
| 0.6646 | 0.82 | 74000 | 0.6634 | |
|
| 0.6616 | 0.83 | 75000 | 0.7020 | |
|
| 0.6762 | 0.84 | 76000 | 0.6638 | |
|
| 0.7509 | 0.85 | 77000 | 0.6541 | |
|
| 0.6963 | 0.86 | 78000 | 0.6781 | |
|
| 0.6949 | 0.87 | 79000 | 0.6576 | |
|
| 0.6781 | 0.89 | 80000 | 0.6900 | |
|
| 0.65 | 0.9 | 81000 | 0.6835 | |
|
| 0.7205 | 0.91 | 82000 | 0.6712 | |
|
| 0.6901 | 0.92 | 83000 | 0.6699 | |
|
| 0.6972 | 0.93 | 84000 | 0.6456 | |
|
| 0.7041 | 0.94 | 85000 | 0.6497 | |
|
| 0.6864 | 0.95 | 86000 | 0.6432 | |
|
| 0.7308 | 0.96 | 87000 | 0.6497 | |
|
| 0.6886 | 0.97 | 88000 | 0.6674 | |
|
| 0.6947 | 0.99 | 89000 | 0.6638 | |
|
| 0.6567 | 1.0 | 90000 | 0.6242 | |
|
| 0.7185 | 1.01 | 91000 | 0.6704 | |
|
| 0.7435 | 1.02 | 92000 | 0.6681 | |
|
| 0.7108 | 1.03 | 93000 | 0.6619 | |
|
| 0.6942 | 1.04 | 94000 | 0.6306 | |
|
| 0.6998 | 1.05 | 95000 | 0.6409 | |
|
| 0.6481 | 1.06 | 96000 | 0.6476 | |
|
| 0.727 | 1.07 | 97000 | 0.6354 | |
|
| 0.647 | 1.09 | 98000 | 0.6222 | |
|
| 0.6622 | 1.1 | 99000 | 0.6119 | |
|
| 0.6346 | 1.11 | 100000 | 0.6471 | |
|
| 0.6203 | 1.12 | 101000 | 0.6655 | |
|
| 0.6765 | 1.13 | 102000 | 0.6473 | |
|
| 0.6703 | 1.14 | 103000 | 0.6308 | |
|
| 0.6793 | 1.15 | 104000 | 0.6531 | |
|
| 0.683 | 1.16 | 105000 | 0.6693 | |
|
| 0.6654 | 1.17 | 106000 | 0.6241 | |
|
| 0.6626 | 1.18 | 107000 | 0.6215 | |
|
| 0.6976 | 1.2 | 108000 | 0.6479 | |
|
| 0.7494 | 1.21 | 109000 | 0.6345 | |
|
| 0.691 | 1.22 | 110000 | 0.6322 | |
|
| 0.6568 | 1.23 | 111000 | 0.6265 | |
|
| 0.705 | 1.24 | 112000 | 0.6281 | |
|
| 0.6307 | 1.25 | 113000 | 0.6202 | |
|
| 0.6828 | 1.26 | 114000 | 0.6158 | |
|
| 0.6403 | 1.27 | 115000 | 0.6495 | |
|
| 0.6615 | 1.28 | 116000 | 0.6298 | |
|
| 0.6237 | 1.3 | 117000 | 0.6234 | |
|
| 0.6707 | 1.31 | 118000 | 0.6267 | |
|
| 0.6823 | 1.32 | 119000 | 0.6299 | |
|
| 0.6333 | 1.33 | 120000 | 0.6169 | |
|
| 0.685 | 1.34 | 121000 | 0.6371 | |
|
| 0.6941 | 1.35 | 122000 | 0.6245 | |
|
| 0.6358 | 1.36 | 123000 | 0.6291 | |
|
| 0.6754 | 1.37 | 124000 | 0.6400 | |
|
| 0.6286 | 1.38 | 125000 | 0.6148 | |
|
| 0.7036 | 1.4 | 126000 | 0.6033 | |
|
| 0.645 | 1.41 | 127000 | 0.6295 | |
|
| 0.6578 | 1.42 | 128000 | 0.6348 | |
|
| 0.651 | 1.43 | 129000 | 0.6222 | |
|
| 0.5558 | 1.44 | 130000 | 0.6231 | |
|
| 0.6601 | 1.45 | 131000 | 0.6302 | |
|
| 0.6304 | 1.46 | 132000 | 0.6127 | |
|
| 0.6177 | 1.47 | 133000 | 0.6047 | |
|
| 0.5933 | 1.48 | 134000 | 0.6169 | |
|
| 0.6307 | 1.49 | 135000 | 0.6012 | |
|
| 0.6018 | 1.51 | 136000 | 0.5900 | |
|
| 0.6724 | 1.52 | 137000 | 0.6086 | |
|
| 0.6367 | 1.53 | 138000 | 0.6414 | |
|
| 0.6515 | 1.54 | 139000 | 0.6267 | |
|
| 0.5902 | 1.55 | 140000 | 0.5913 | |
|
| 0.6523 | 1.56 | 141000 | 0.5992 | |
|
| 0.6005 | 1.57 | 142000 | 0.6128 | |
|
| 0.6179 | 1.58 | 143000 | 0.6089 | |
|
| 0.6154 | 1.59 | 144000 | 0.6353 | |
|
| 0.6298 | 1.61 | 145000 | 0.5997 | |
|
| 0.5623 | 1.62 | 146000 | 0.5974 | |
|
| 0.5787 | 1.63 | 147000 | 0.6165 | |
|
| 0.6099 | 1.64 | 148000 | 0.6246 | |
|
| 0.658 | 1.65 | 149000 | 0.6116 | |
|
| 0.6567 | 1.66 | 150000 | 0.5938 | |
|
| 0.6227 | 1.67 | 151000 | 0.5948 | |
|
| 0.5858 | 1.68 | 152000 | 0.5822 | |
|
| 0.6227 | 1.69 | 153000 | 0.5802 | |
|
| 0.6699 | 1.71 | 154000 | 0.6067 | |
|
| 0.5989 | 1.72 | 155000 | 0.6073 | |
|
| 0.6184 | 1.73 | 156000 | 0.6124 | |
|
| 0.6404 | 1.74 | 157000 | 0.6169 | |
|
| 0.639 | 1.75 | 158000 | 0.5997 | |
|
| 0.6433 | 1.76 | 159000 | 0.5989 | |
|
| 0.5574 | 1.77 | 160000 | 0.5796 | |
|
| 0.5983 | 1.78 | 161000 | 0.6036 | |
|
| 0.6532 | 1.79 | 162000 | 0.5888 | |
|
| 0.6679 | 1.8 | 163000 | 0.6038 | |
|
| 0.62 | 1.82 | 164000 | 0.5984 | |
|
| 0.5541 | 1.83 | 165000 | 0.6003 | |
|
| 0.6192 | 1.84 | 166000 | 0.5786 | |
|
| 0.6613 | 1.85 | 167000 | 0.6064 | |
|
| 0.5923 | 1.86 | 168000 | 0.6018 | |
|
| 0.5894 | 1.87 | 169000 | 0.5912 | |
|
| 0.6462 | 1.88 | 170000 | 0.5902 | |
|
| 0.5811 | 1.89 | 171000 | 0.6030 | |
|
| 0.6358 | 1.9 | 172000 | 0.5915 | |
|
| 0.614 | 1.92 | 173000 | 0.5886 | |
|
| 0.5969 | 1.93 | 174000 | 0.6084 | |
|
| 0.6146 | 1.94 | 175000 | 0.6003 | |
|
| 0.6051 | 1.95 | 176000 | 0.5835 | |
|
| 0.6268 | 1.96 | 177000 | 0.5999 | |
|
| 0.6436 | 1.97 | 178000 | 0.5965 | |
|
| 0.6167 | 1.98 | 179000 | 0.5789 | |
|
| 0.5647 | 1.99 | 180000 | 0.5669 | |
|
| 0.6038 | 2.0 | 181000 | 0.6009 | |
|
| 0.6082 | 2.02 | 182000 | 0.5799 | |
|
| 0.6483 | 2.03 | 183000 | 0.5716 | |
|
| 0.5503 | 2.04 | 184000 | 0.5806 | |
|
| 0.6231 | 2.05 | 185000 | 0.5699 | |
|
| 0.5892 | 2.06 | 186000 | 0.5979 | |
|
| 0.5933 | 2.07 | 187000 | 0.5709 | |
|
| 0.594 | 2.08 | 188000 | 0.5719 | |
|
| 0.5838 | 2.09 | 189000 | 0.5879 | |
|
| 0.6039 | 2.1 | 190000 | 0.5984 | |
|
| 0.5911 | 2.11 | 191000 | 0.5953 | |
|
| 0.563 | 2.13 | 192000 | 0.5772 | |
|
| 0.5671 | 2.14 | 193000 | 0.5771 | |
|
| 0.6051 | 2.15 | 194000 | 0.5972 | |
|
| 0.5852 | 2.16 | 195000 | 0.5917 | |
|
| 0.5757 | 2.17 | 196000 | 0.5819 | |
|
| 0.6557 | 2.18 | 197000 | 0.5655 | |
|
| 0.6055 | 2.19 | 198000 | 0.5820 | |
|
| 0.6067 | 2.2 | 199000 | 0.5801 | |
|
| 0.6422 | 2.21 | 200000 | 0.5590 | |
|
| 0.624 | 2.23 | 201000 | 0.5573 | |
|
| 0.6222 | 2.24 | 202000 | 0.5661 | |
|
| 0.5597 | 2.25 | 203000 | 0.5786 | |
|
| 0.5746 | 2.26 | 204000 | 0.5622 | |
|
| 0.6269 | 2.27 | 205000 | 0.5804 | |
|
| 0.6241 | 2.28 | 206000 | 0.5696 | |
|
| 0.6519 | 2.29 | 207000 | 0.5367 | |
|
| 0.6161 | 2.3 | 208000 | 0.5666 | |
|
| 0.5415 | 2.31 | 209000 | 0.5633 | |
|
| 0.633 | 2.33 | 210000 | 0.5499 | |
|
| 0.5566 | 2.34 | 211000 | 0.5822 | |
|
| 0.6158 | 2.35 | 212000 | 0.5826 | |
|
| 0.5574 | 2.36 | 213000 | 0.5429 | |
|
| 0.5748 | 2.37 | 214000 | 0.5736 | |
|
| 0.5818 | 2.38 | 215000 | 0.5599 | |
|
| 0.6226 | 2.39 | 216000 | 0.5407 | |
|
| 0.5733 | 2.4 | 217000 | 0.5759 | |
|
| 0.6268 | 2.41 | 218000 | 0.5725 | |
|
| 0.5885 | 2.42 | 219000 | 0.5771 | |
|
| 0.5708 | 2.44 | 220000 | 0.5654 | |
|
| 0.5783 | 2.45 | 221000 | 0.5756 | |
|
| 0.61 | 2.46 | 222000 | 0.5647 | |
|
| 0.5848 | 2.47 | 223000 | 0.5532 | |
|
| 0.5869 | 2.48 | 224000 | 0.5519 | |
|
| 0.5717 | 2.49 | 225000 | 0.5621 | |
|
| 0.5675 | 2.5 | 226000 | 0.5446 | |
|
| 0.6321 | 2.51 | 227000 | 0.5812 | |
|
| 0.568 | 2.52 | 228000 | 0.5673 | |
|
| 0.5577 | 2.54 | 229000 | 0.5590 | |
|
| 0.5888 | 2.55 | 230000 | 0.5628 | |
|
| 0.6389 | 2.56 | 231000 | 0.5828 | |
|
| 0.5782 | 2.57 | 232000 | 0.5543 | |
|
| 0.5871 | 2.58 | 233000 | 0.5575 | |
|
| 0.5593 | 2.59 | 234000 | 0.5625 | |
|
| 0.6167 | 2.6 | 235000 | 0.5450 | |
|
| 0.5828 | 2.61 | 236000 | 0.5627 | |
|
| 0.5411 | 2.62 | 237000 | 0.5498 | |
|
| 0.6168 | 2.64 | 238000 | 0.5891 | |
|
| 0.6508 | 2.65 | 239000 | 0.5811 | |
|
| 0.6322 | 2.66 | 240000 | 0.5649 | |
|
| 0.6131 | 2.67 | 241000 | 0.5473 | |
|
| 0.5419 | 2.68 | 242000 | 0.5583 | |
|
| 0.5685 | 2.69 | 243000 | 0.5635 | |
|
| 0.5267 | 2.7 | 244000 | 0.5481 | |
|
| 0.5357 | 2.71 | 245000 | 0.5474 | |
|
| 0.585 | 2.72 | 246000 | 0.5281 | |
|
| 0.5894 | 2.73 | 247000 | 0.5457 | |
|
| 0.5665 | 2.75 | 248000 | 0.5579 | |
|
| 0.5409 | 2.76 | 249000 | 0.5412 | |
|
| 0.6051 | 2.77 | 250000 | 0.5447 | |
|
| 0.5866 | 2.78 | 251000 | 0.5535 | |
|
| 0.5348 | 2.79 | 252000 | 0.5377 | |
|
| 0.5606 | 2.8 | 253000 | 0.5524 | |
|
| 0.5142 | 2.81 | 254000 | 0.5441 | |
|
| 0.543 | 2.82 | 255000 | 0.5499 | |
|
| 0.5763 | 2.83 | 256000 | 0.5241 | |
|
|
|
|
|
### Framework versions |
|
|
|
- Transformers 4.20.0.dev0 |
|
- Pytorch 1.10.0+cu102 |
|
- Datasets 1.15.1 |
|
- Tokenizers 0.11.0 |
|
|
