End of training

DPO/adapter_config.json

@@ -21,11 +21,11 @@
   "revision": null,
   "target_modules": [
     "down_proj",
-    "up_proj",
     "v_proj",
     "o_proj",
-    "k_proj",
     "gate_proj",
+    "up_proj",
+    "k_proj",
     "q_proj"
   ],
   "task_type": "CAUSAL_LM",

DPO/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:2a7eb1c70ec3f189be41d13b604d9f501f3d2ca8443b33aabb787b6477ba051a
+oid sha256:51b6441543012f61d5fac618e8936ebdf87fb01c69f120a14d955b0eb5082709
 size 167832240

README.md

@@ -18,15 +18,15 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1) on the None dataset.
 It achieves the following results on the evaluation set:
-- Loss: 0.6901
-- Rewards/chosen: 0.0478
-- Rewards/rejected: 0.0414
-- Rewards/accuracies: 0.5986
-- Rewards/margins: 0.0065
-- Logps/rejected: -55.2578
-- Logps/chosen: -58.2493
-- Logits/rejected: -0.0691
-- Logits/chosen: -0.1144
+- Loss: 0.6891
+- Rewards/chosen: 0.0621
+- Rewards/rejected: 0.0538
+- Rewards/accuracies: 0.6213
+- Rewards/margins: 0.0083
+- Logps/rejected: -52.0979
+- Logps/chosen: -55.7624
+- Logits/rejected: -0.2991
+- Logits/chosen: -0.3269
 
 ## Model description
 
@@ -60,11 +60,11 @@ The following hyperparameters were used during training:
 
 | Training Loss | Epoch  | Step | Validation Loss | Rewards/chosen | Rewards/rejected | Rewards/accuracies | Rewards/margins | Logps/rejected | Logps/chosen | Logits/rejected | Logits/chosen |
 |:-------------:|:------:|:----:|:---------------:|:--------------:|:----------------:|:------------------:|:---------------:|:--------------:|:------------:|:---------------:|:-------------:|
-| 0.693         | 0.0882 | 20   | 0.6928          | 0.0070         | 0.0063           | 0.5487             | 0.0007          | -55.6083       | -58.6576     | -0.0693         | -0.1147       |
-| 0.6919        | 0.1763 | 40   | 0.6915          | 0.0249         | 0.0215           | 0.6064             | 0.0034          | -55.4560       | -58.4780     | -0.0692         | -0.1146       |
-| 0.6908        | 0.2645 | 60   | 0.6907          | 0.0344         | 0.0292           | 0.6038             | 0.0051          | -55.3790       | -58.3838     | -0.0694         | -0.1148       |
-| 0.691         | 0.3526 | 80   | 0.6901          | 0.0421         | 0.0358           | 0.6012             | 0.0063          | -55.3130       | -58.3061     | -0.0691         | -0.1144       |
-| 0.6895        | 0.4408 | 100  | 0.6901          | 0.0478         | 0.0414           | 0.5986             | 0.0065          | -55.2578       | -58.2493     | -0.0691         | -0.1144       |
+| 0.6929        | 0.0882 | 20   | 0.6923          | 0.0239         | 0.0221           | 0.6109             | 0.0018          | -52.4151       | -56.1451     | -0.2993         | -0.3271       |
+| 0.6919        | 0.1763 | 40   | 0.6908          | 0.0490         | 0.0443           | 0.6187             | 0.0047          | -52.1927       | -55.8934     | -0.2991         | -0.3268       |
+| 0.6903        | 0.2645 | 60   | 0.6898          | 0.0587         | 0.0518           | 0.6157             | 0.0068          | -52.1173       | -55.7971     | -0.2991         | -0.3269       |
+| 0.6899        | 0.3526 | 80   | 0.6892          | 0.0595         | 0.0515           | 0.6135             | 0.0081          | -52.1210       | -55.7885     | -0.2991         | -0.3269       |
+| 0.6898        | 0.4408 | 100  | 0.6891          | 0.0621         | 0.0538           | 0.6213             | 0.0083          | -52.0979       | -55.7624     | -0.2991         | -0.3269       |
 
 
 ### Framework versions

reference/adapter_config.json

@@ -21,11 +21,11 @@
   "revision": null,
   "target_modules": [
     "down_proj",
-    "up_proj",
     "v_proj",
     "o_proj",
-    "k_proj",
     "gate_proj",
+    "up_proj",
+    "k_proj",
     "q_proj"
   ],
   "task_type": "CAUSAL_LM",

reference/adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0993053d1cb8f5126339726b78be278d424a5ab65a62968aae521c6578153dfa
+oid sha256:76482a406513cef1b542263101e87b786e24bba632d3af9ec30ccef79a3ebedd
 size 167832240

runs/Jun24_18-28-36_155-248-201-118/events.out.tfevents.1719253729.155-248-201-118.33150.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:839288b1b507137dac0f5c537a9ae8be443d11739a970609a95d41e0313adc0a
+size 4184

runs/Jun24_18-29-35_155-248-201-118/events.out.tfevents.1719253776.155-248-201-118.33828.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:799a1c4fb98f1aec13a77de44cdb4ef70751ef3a6ad9fdf9a2600d9ca713e12c
+size 13348

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:0a5da2270c2f48ae24e54031e2da497def000d592b75f0b67f1870f62144598e
+oid sha256:6aabc3510676719d749ac928da99449e40a96661d751ee0838c5f43277a60184
 size 5688