Hugging Face's logo

imdatta0
/
Qwen2-7B_magiccoder_default

PEFT
Safetensors
unsloth
Generated from Trainer
Model card Files Community
imdatta0 commited on
Commit
0879910
•
1 Parent(s): 898d6a8

End of training

Browse files
Files changed (2) hide show
  1. README.md +100 -0
  2. adapter_model.safetensors +1 -1
README.md ADDED
@@ -0,0 +1,100 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ base_model: unsloth/qwen2-7b-bnb-4bit
3
+ library_name: peft
4
+ license: apache-2.0
5
+ tags:
6
+ - unsloth
7
+ - generated_from_trainer
8
+ model-index:
9
+ - name: Qwen2-7B_magiccoder_default
10
+ results: []
11
+ ---
12
+
13
+ <!-- This model card has been generated automatically according to the information the Trainer had access to. You
14
+ should probably proofread and complete it, then remove this comment. -->
15
+
16
+ # Qwen2-7B_magiccoder_default
17
+
18
+ This model is a fine-tuned version of [unsloth/qwen2-7b-bnb-4bit](https://huggingface.co/unsloth/qwen2-7b-bnb-4bit) on an unknown dataset.
19
+ It achieves the following results on the evaluation set:
20
+ - Loss: 0.8796
21
+
22
+ ## Model description
23
+
24
+ More information needed
25
+
26
+ ## Intended uses & limitations
27
+
28
+ More information needed
29
+
30
+ ## Training and evaluation data
31
+
32
+ More information needed
33
+
34
+ ## Training procedure
35
+
36
+ ### Training hyperparameters
37
+
38
+ The following hyperparameters were used during training:
39
+ - learning_rate: 0.0001
40
+ - train_batch_size: 8
41
+ - eval_batch_size: 8
42
+ - seed: 42
43
+ - gradient_accumulation_steps: 8
44
+ - total_train_batch_size: 64
45
+ - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
46
+ - lr_scheduler_type: cosine
47
+ - lr_scheduler_warmup_ratio: 0.02
48
+ - num_epochs: 1
49
+
50
+ ### Training results
51
+
52
+ | Training Loss | Epoch | Step | Validation Loss |
53
+ |:-------------:|:------:|:----:|:---------------:|
54
+ | 0.9592 | 0.0261 | 4 | 1.0014 |
55
+ | 0.9246 | 0.0522 | 8 | 0.9342 |
56
+ | 0.9101 | 0.0783 | 12 | 0.9148 |
57
+ | 0.8605 | 0.1044 | 16 | 0.9008 |
58
+ | 0.9093 | 0.1305 | 20 | 0.8958 |
59
+ | 0.7967 | 0.1566 | 24 | 0.8949 |
60
+ | 0.8075 | 0.1827 | 28 | 0.8941 |
61
+ | 0.9522 | 0.2088 | 32 | 0.8917 |
62
+ | 0.8503 | 0.2349 | 36 | 0.8890 |
63
+ | 0.8704 | 0.2610 | 40 | 0.8864 |
64
+ | 0.9286 | 0.2871 | 44 | 0.8852 |
65
+ | 0.894 | 0.3132 | 48 | 0.8839 |
66
+ | 0.8171 | 0.3393 | 52 | 0.8835 |
67
+ | 0.8224 | 0.3654 | 56 | 0.8828 |
68
+ | 0.8392 | 0.3915 | 60 | 0.8822 |
69
+ | 0.897 | 0.4176 | 64 | 0.8817 |
70
+ | 0.8942 | 0.4437 | 68 | 0.8813 |
71
+ | 0.8501 | 0.4698 | 72 | 0.8809 |
72
+ | 0.8768 | 0.4959 | 76 | 0.8807 |
73
+ | 0.8337 | 0.5220 | 80 | 0.8806 |
74
+ | 0.9285 | 0.5481 | 84 | 0.8801 |
75
+ | 0.8301 | 0.5742 | 88 | 0.8803 |
76
+ | 0.9212 | 0.6003 | 92 | 0.8805 |
77
+ | 0.8485 | 0.6264 | 96 | 0.8804 |
78
+ | 0.9419 | 0.6525 | 100 | 0.8801 |
79
+ | 0.8191 | 0.6786 | 104 | 0.8799 |
80
+ | 0.8668 | 0.7047 | 108 | 0.8799 |
81
+ | 0.8335 | 0.7308 | 112 | 0.8798 |
82
+ | 0.8003 | 0.7569 | 116 | 0.8797 |
83
+ | 0.794 | 0.7830 | 120 | 0.8796 |
84
+ | 0.863 | 0.8091 | 124 | 0.8797 |
85
+ | 0.8262 | 0.8352 | 128 | 0.8795 |
86
+ | 0.9269 | 0.8613 | 132 | 0.8796 |
87
+ | 0.876 | 0.8874 | 136 | 0.8795 |
88
+ | 0.8955 | 0.9135 | 140 | 0.8794 |
89
+ | 0.8481 | 0.9396 | 144 | 0.8796 |
90
+ | 0.9111 | 0.9657 | 148 | 0.8795 |
91
+ | 0.8653 | 0.9918 | 152 | 0.8796 |
92
+
93
+
94
+ ### Framework versions
95
+
96
+ - PEFT 0.12.0
97
+ - Transformers 4.44.0
98
+ - Pytorch 2.4.0+cu121
99
+ - Datasets 2.20.0
100
+ - Tokenizers 0.19.1
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:9bd9e53c86b31e5e0e38434b3de9d20e367c22f0a63e302267a2da84f456eee6
3
  size 80792096
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:dc866ed885b81965127546907506c8304e45e1cfc3139f87915530444cadfab7
3
  size 80792096