sri-lasya/gst-taxing-llm

README.md

@@ -1,9 +1,9 @@
 ---
-license: apache-2.0
+base_model: mistralai/Mistral-7B-v0.3
 library_name: peft
+license: apache-2.0
 tags:
 - generated_from_trainer
-base_model: mistralai/Mistral-7B-v0.3
 model-index:
 - name: mistral_fine_tuned
   results: []
@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6741
+- Loss: 1.7477
 
 ## Model description
 
@@ -47,18 +47,18 @@ The following hyperparameters were used during training:
 
 ### Training results
 
-| Training Loss | Epoch  | Step | Validation Loss |
-|:-------------:|:------:|:----:|:---------------:|
-| 2.7903        | 0.0926 | 10   | 2.8396          |
-| 2.4022        | 0.1852 | 20   | 2.3709          |
-| 1.9902        | 0.2778 | 30   | 2.0631          |
-| 1.8025        | 0.3704 | 40   | 1.9687          |
-| 1.7349        | 0.4630 | 50   | 1.8904          |
-| 1.718         | 0.5556 | 60   | 1.8402          |
-| 1.7084        | 0.6481 | 70   | 1.8026          |
-| 1.6992        | 0.7407 | 80   | 1.7643          |
-| 1.5539        | 0.8333 | 90   | 1.7414          |
-| 1.5368        | 0.9259 | 100  | 1.6741          |
+| Training Loss | Epoch | Step | Validation Loss |
+|:-------------:|:-----:|:----:|:---------------:|
+| 3.3406        | 0.1   | 10   | 2.8564          |
+| 2.3377        | 0.2   | 20   | 2.3722          |
+| 1.9307        | 0.3   | 30   | 2.0729          |
+| 1.7952        | 0.4   | 40   | 1.9599          |
+| 1.8246        | 0.5   | 50   | 1.9037          |
+| 1.7709        | 0.6   | 60   | 1.9209          |
+| 1.5977        | 0.7   | 70   | 1.8391          |
+| 1.5913        | 0.8   | 80   | 1.8199          |
+| 1.5449        | 0.9   | 90   | 1.7803          |
+| 1.6924        | 1.0   | 100  | 1.7477          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,14 +20,14 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "up_proj",
     "lm_head",
+    "o_proj",
     "gate_proj",
     "v_proj",
-    "down_proj",
-    "q_proj",
     "k_proj",
-    "o_proj"
+    "down_proj",
+    "up_proj",
+    "q_proj"
   ],
   "task_type": "CAUSAL_LM",
   "use_dora": false,

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5dc20fe0466e8feac51efc369010453b2f3eeab64d9c7ee19d21d8d33e9978ca
+oid sha256:1d5097c1245b1e01baa1df0fcdcff0daeee1661038085dff3ecce50f2403dfa4
 size 1217458040