Hugging Face's logo

sri-lasya
/
mistral_fine_tuned

PEFT
Safetensors
Generated from Trainer
Model card Files Community
sri-lasya commited on
Commit
e3b7c04
•
1 Parent(s): 0eea9e3

sri-lasya/gst-taxing-llm

Browse files
Files changed (4) hide show
  1. README.md +27 -27
  2. adapter_config.json +5 -5
  3. adapter_model.safetensors +1 -1
  4. training_args.bin +1 -1
README.md CHANGED
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
17
 
18
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
- - Loss: 1.7791
21
 
22
  ## Model description
23
 
@@ -36,7 +36,7 @@ More information needed
36
  ### Training hyperparameters
37
 
38
  The following hyperparameters were used during training:
39
- - learning_rate: 5e-05
40
  - train_batch_size: 8
41
  - eval_batch_size: 8
42
  - seed: 42
@@ -49,31 +49,31 @@ The following hyperparameters were used during training:
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
- | 2.9281 | 0.04 | 10 | 2.3786 |
53
- | 2.1861 | 0.08 | 20 | 1.9991 |
54
- | 1.5698 | 0.12 | 30 | 1.9460 |
55
- | 1.6891 | 0.16 | 40 | 1.8457 |
56
- | 1.6863 | 0.2 | 50 | 1.8188 |
57
- | 1.4201 | 0.24 | 60 | 1.7638 |
58
- | 1.6936 | 0.28 | 70 | 1.7349 |
59
- | 1.5696 | 0.32 | 80 | 1.6898 |
60
- | 1.6084 | 0.36 | 90 | 1.7269 |
61
- | 1.5357 | 0.4 | 100 | 1.7332 |
62
- | 1.6684 | 0.44 | 110 | 1.6728 |
63
- | 1.2216 | 0.48 | 120 | 1.6563 |
64
- | 1.5176 | 0.52 | 130 | 1.6376 |
65
- | 1.5256 | 0.56 | 140 | 1.6653 |
66
- | 1.3695 | 0.6 | 150 | 1.6317 |
67
- | 1.3191 | 0.64 | 160 | 1.6261 |
68
- | 1.4091 | 0.68 | 170 | 1.6145 |
69
- | 1.5681 | 0.72 | 180 | 1.6007 |
70
- | 1.7259 | 0.76 | 190 | 1.6076 |
71
- | 1.5123 | 0.8 | 200 | 1.6836 |
72
- | 1.387 | 0.84 | 210 | 1.7292 |
73
- | 1.5837 | 0.88 | 220 | 1.7177 |
74
- | 1.6955 | 0.92 | 230 | 1.7232 |
75
- | 1.7862 | 0.96 | 240 | 1.7714 |
76
- | 1.79 | 1.0 | 250 | 1.7791 |
77
 
78
 
79
  ### Framework versions
 
17
 
18
  This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
19
  It achieves the following results on the evaluation set:
20
+ - Loss: 1.7859
21
 
22
  ## Model description
23
 
 
36
  ### Training hyperparameters
37
 
38
  The following hyperparameters were used during training:
39
+ - learning_rate: 3e-05
40
  - train_batch_size: 8
41
  - eval_batch_size: 8
42
  - seed: 42
 
49
 
50
  | Training Loss | Epoch | Step | Validation Loss |
51
  |:-------------:|:-----:|:----:|:---------------:|
52
+ | 3.1512 | 0.04 | 10 | 2.7003 |
53
+ | 2.5126 | 0.08 | 20 | 2.1770 |
54
+ | 1.696 | 0.12 | 30 | 2.0002 |
55
+ | 1.7231 | 0.16 | 40 | 1.9348 |
56
+ | 1.7336 | 0.2 | 50 | 1.9126 |
57
+ | 1.4891 | 0.24 | 60 | 1.8458 |
58
+ | 1.7379 | 0.28 | 70 | 1.7978 |
59
+ | 1.6231 | 0.32 | 80 | 1.7494 |
60
+ | 1.6705 | 0.36 | 90 | 1.7712 |
61
+ | 1.5771 | 0.4 | 100 | 1.7794 |
62
+ | 1.7315 | 0.44 | 110 | 1.7449 |
63
+ | 1.2828 | 0.48 | 120 | 1.7161 |
64
+ | 1.5608 | 0.52 | 130 | 1.6869 |
65
+ | 1.5608 | 0.56 | 140 | 1.6915 |
66
+ | 1.4046 | 0.6 | 150 | 1.6782 |
67
+ | 1.3853 | 0.64 | 160 | 1.6715 |
68
+ | 1.4398 | 0.68 | 170 | 1.6472 |
69
+ | 1.5917 | 0.72 | 180 | 1.6333 |
70
+ | 1.7207 | 0.76 | 190 | 1.6295 |
71
+ | 1.5135 | 0.8 | 200 | 1.6901 |
72
+ | 1.3951 | 0.84 | 210 | 1.7478 |
73
+ | 1.6068 | 0.88 | 220 | 1.7256 |
74
+ | 1.7094 | 0.92 | 230 | 1.7206 |
75
+ | 1.8219 | 0.96 | 240 | 1.7528 |
76
+ | 1.7888 | 1.0 | 250 | 1.7859 |
77
 
78
 
79
  ### Framework versions
adapter_config.json CHANGED
@@ -20,14 +20,14 @@
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
23
- "lm_head",
24
- "gate_proj",
25
- "o_proj",
26
- "q_proj",
27
  "k_proj",
28
  "down_proj",
 
 
 
29
  "up_proj",
30
- "v_proj"
 
31
  ],
32
  "task_type": "CAUSAL_LM",
33
  "use_dora": false,
 
20
  "rank_pattern": {},
21
  "revision": null,
22
  "target_modules": [
 
 
 
 
23
  "k_proj",
24
  "down_proj",
25
+ "gate_proj",
26
+ "v_proj",
27
+ "q_proj",
28
  "up_proj",
29
+ "o_proj",
30
+ "lm_head"
31
  ],
32
  "task_type": "CAUSAL_LM",
33
  "use_dora": false,
adapter_model.safetensors CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:e8c5f32b7d6fa90f84d910a465422cf94db936e145caeb9b099519e4bd3a5b34
3
  size 1217458040
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:c1a922bdd50f194559d95b2dc6dbc3dea965dde7ddce5072bc9e336d3773945b
3
  size 1217458040
training_args.bin CHANGED
@@ -1,3 +1,3 @@
1
  version https://git-lfs.github.com/spec/v1
2
- oid sha256:8724a5519057acb5e2a9f5c7a800ead5125c04cddc074795c276676fc7cfe0da
3
  size 5048
 
1
  version https://git-lfs.github.com/spec/v1
2
+ oid sha256:5690a186ae8a6f85c974bd1f2517acfc089f339519fa9acbe05f884bb56f47c6
3
  size 5048