File size: 5,738 Bytes
17d7c13 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 |
---
license: apache-2.0
base_model: mosaicml/mpt-7b-instruct
tags:
- trl
- dpo
- generated_from_trainer
model-index:
- name: MPT_1000_STEPS_1e7_rate_03_beta_DPO
results: []
---
<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->
# MPT_1000_STEPS_1e7_rate_03_beta_DPO
This model is a fine-tuned version of [mosaicml/mpt-7b-instruct](https://huggingface.co./mosaicml/mpt-7b-instruct) on an unknown dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6924
- Rewards/chosen: -0.0146
- Rewards/rejected: -0.0175
- Rewards/accuracies: 0.5275
- Rewards/margins: 0.0029
- Logps/rejected: -21.6159
- Logps/chosen: -20.8410
- Logits/rejected: 14.2241
- Logits/chosen: 14.2267
## Model description
More information needed
## Intended uses & limitations
More information needed
## Training and evaluation data
More information needed
## Training procedure
### Training hyperparameters
The following hyperparameters were used during training:
- learning_rate: 1e-07
- train_batch_size: 2
- eval_batch_size: 1
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 4
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: cosine
- lr_scheduler_warmup_steps: 100
- training_steps: 1000
### Training results
| Training Loss | Epoch | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
|:-------------:|:-----:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
| 0.6908 | 0.05 | 50 | 0.6958 | -0.0024 | 0.0016 | 0.4835 | -0.0040 | -21.5521 | -20.8002 | 14.2618 | 14.2644 |
| 0.7007 | 0.1 | 100 | 0.6940 | -0.0004 | -0.0001 | 0.5033 | -0.0003 | -21.5577 | -20.7936 | 14.2508 | 14.2534 |
| 0.6945 | 0.15 | 150 | 0.6935 | -0.0010 | -0.0016 | 0.4923 | 0.0006 | -21.5629 | -20.7956 | 14.2501 | 14.2527 |
| 0.6911 | 0.2 | 200 | 0.6947 | 0.0111 | 0.0130 | 0.5055 | -0.0019 | -21.5142 | -20.7552 | 14.2536 | 14.2561 |
| 0.6944 | 0.24 | 250 | 0.6926 | -0.0007 | -0.0032 | 0.5297 | 0.0025 | -21.5681 | -20.7945 | 14.2489 | 14.2515 |
| 0.6893 | 0.29 | 300 | 0.6925 | -0.0029 | -0.0056 | 0.5143 | 0.0027 | -21.5761 | -20.8017 | 14.2454 | 14.2480 |
| 0.6964 | 0.34 | 350 | 0.6933 | -0.0031 | -0.0043 | 0.4901 | 0.0012 | -21.5718 | -20.8026 | 14.2500 | 14.2526 |
| 0.6846 | 0.39 | 400 | 0.6899 | -0.0142 | -0.0220 | 0.5516 | 0.0078 | -21.6306 | -20.8394 | 14.2259 | 14.2284 |
| 0.6823 | 0.44 | 450 | 0.6910 | -0.0143 | -0.0200 | 0.5143 | 0.0056 | -21.6240 | -20.8400 | 14.2294 | 14.2320 |
| 0.6838 | 0.49 | 500 | 0.6908 | -0.0099 | -0.0159 | 0.5297 | 0.0059 | -21.6103 | -20.8253 | 14.2237 | 14.2263 |
| 0.678 | 0.54 | 550 | 0.6897 | -0.0151 | -0.0234 | 0.5407 | 0.0082 | -21.6354 | -20.8427 | 14.2251 | 14.2277 |
| 0.6872 | 0.59 | 600 | 0.6915 | -0.0176 | -0.0223 | 0.5385 | 0.0047 | -21.6318 | -20.8508 | 14.2284 | 14.2311 |
| 0.6881 | 0.64 | 650 | 0.6906 | -0.0132 | -0.0196 | 0.5319 | 0.0064 | -21.6228 | -20.8362 | 14.2236 | 14.2262 |
| 0.6841 | 0.68 | 700 | 0.6910 | -0.0146 | -0.0202 | 0.5143 | 0.0057 | -21.6249 | -20.8408 | 14.2152 | 14.2178 |
| 0.6883 | 0.73 | 750 | 0.6901 | -0.0148 | -0.0223 | 0.5626 | 0.0075 | -21.6317 | -20.8414 | 14.2218 | 14.2244 |
| 0.6813 | 0.78 | 800 | 0.6917 | -0.0150 | -0.0192 | 0.5341 | 0.0041 | -21.6213 | -20.8422 | 14.2255 | 14.2281 |
| 0.6987 | 0.83 | 850 | 0.6902 | -0.0129 | -0.0204 | 0.5297 | 0.0075 | -21.6253 | -20.8350 | 14.2198 | 14.2223 |
| 0.687 | 0.88 | 900 | 0.6928 | -0.0126 | -0.0148 | 0.5121 | 0.0021 | -21.6067 | -20.8343 | 14.2248 | 14.2275 |
| 0.6885 | 0.93 | 950 | 0.6924 | -0.0146 | -0.0175 | 0.5275 | 0.0029 | -21.6159 | -20.8410 | 14.2241 | 14.2267 |
| 0.6904 | 0.98 | 1000 | 0.6924 | -0.0146 | -0.0175 | 0.5275 | 0.0029 | -21.6159 | -20.8410 | 14.2241 | 14.2267 |
### Framework versions
- Transformers 4.39.1
- Pytorch 2.0.0+cu117
- Datasets 2.18.0
- Tokenizers 0.15.2
|