sri-lasya/gst-taxing-llm

README.md

@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.7859
+- Loss: 1.7777
 
 ## Model description
 
@@ -43,37 +43,22 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_steps: 0.03
-- training_steps: 250
+- training_steps: 100
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 3.1512        | 0.04  | 10   | 2.7003          |
-| 2.5126        | 0.08  | 20   | 2.1770          |
-| 1.696         | 0.12  | 30   | 2.0002          |
-| 1.7231        | 0.16  | 40   | 1.9348          |
-| 1.7336        | 0.2   | 50   | 1.9126          |
-| 1.4891        | 0.24  | 60   | 1.8458          |
-| 1.7379        | 0.28  | 70   | 1.7978          |
-| 1.6231        | 0.32  | 80   | 1.7494          |
-| 1.6705        | 0.36  | 90   | 1.7712          |
-| 1.5771        | 0.4   | 100  | 1.7794          |
-| 1.7315        | 0.44  | 110  | 1.7449          |
-| 1.2828        | 0.48  | 120  | 1.7161          |
-| 1.5608        | 0.52  | 130  | 1.6869          |
-| 1.5608        | 0.56  | 140  | 1.6915          |
-| 1.4046        | 0.6   | 150  | 1.6782          |
-| 1.3853        | 0.64  | 160  | 1.6715          |
-| 1.4398        | 0.68  | 170  | 1.6472          |
-| 1.5917        | 0.72  | 180  | 1.6333          |
-| 1.7207        | 0.76  | 190  | 1.6295          |
-| 1.5135        | 0.8   | 200  | 1.6901          |
-| 1.3951        | 0.84  | 210  | 1.7478          |
-| 1.6068        | 0.88  | 220  | 1.7256          |
-| 1.7094        | 0.92  | 230  | 1.7206          |
-| 1.8219        | 0.96  | 240  | 1.7528          |
-| 1.7888        | 1.0   | 250  | 1.7859          |
+| 3.1448        | 0.1   | 10   | 2.6930          |
+| 2.5114        | 0.2   | 20   | 2.1804          |
+| 1.6951        | 0.3   | 30   | 2.0022          |
+| 1.7202        | 0.4   | 40   | 1.9292          |
+| 1.7303        | 0.5   | 50   | 1.9083          |
+| 1.4861        | 0.6   | 60   | 1.8414          |
+| 1.7349        | 0.7   | 70   | 1.7941          |
+| 1.6208        | 0.8   | 80   | 1.7469          |
+| 1.6669        | 0.9   | 90   | 1.7685          |
+| 1.5728        | 1.0   | 100  | 1.7777          |
 
 
 ### Framework versions

adapter_config.json

@@ -21,11 +21,11 @@
   "revision": null,
   "target_modules": [
     "k_proj",
-    "down_proj",
-    "gate_proj",
+    "up_proj",
     "v_proj",
     "q_proj",
-    "up_proj",
+    "down_proj",
+    "gate_proj",
     "o_proj",
     "lm_head"
   ],

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c1a922bdd50f194559d95b2dc6dbc3dea965dde7ddce5072bc9e336d3773945b
+oid sha256:882aa51c16706785857813f589e45856756f4e694354b248a3bbd8616a6d5a7d
 size 1217458040

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:5690a186ae8a6f85c974bd1f2517acfc089f339519fa9acbe05f884bb56f47c6
+oid sha256:5c4bcac94e7d8a069e668bc9663b1ba2feffdb9f1ba69641a598006ebedc3773
 size 5048