sri-lasya/gst-taxing-llm

README.md

@@ -17,7 +17,7 @@ should probably proofread and complete it, then remove this comment. -->
 
 This model is a fine-tuned version of [mistralai/Mistral-7B-v0.3](https://huggingface.co/mistralai/Mistral-7B-v0.3) on an unknown dataset.
 It achieves the following results on the evaluation set:
-- Loss: 1.6832
+- Loss: 1.7791
 
 ## Model description
 
@@ -36,29 +36,44 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 3e-05
-- train_batch_size: 15
+- learning_rate: 5e-05
+- train_batch_size: 8
 - eval_batch_size: 8
 - seed: 42
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_steps: 0.03
-- training_steps: 100
+- training_steps: 250
 
 ### Training results
 
 | Training Loss | Epoch | Step | Validation Loss |
 |:-------------:|:-----:|:----:|:---------------:|
-| 3.1857        | 0.1   | 10   | 2.6298          |
-| 2.0791        | 0.2   | 20   | 2.1259          |
-| 1.7399        | 0.3   | 30   | 1.9620          |
-| 1.7326        | 0.4   | 40   | 1.8837          |
-| 1.7447        | 0.5   | 50   | 1.8289          |
-| 1.7049        | 0.6   | 60   | 1.8458          |
-| 1.5188        | 0.7   | 70   | 1.7563          |
-| 1.5249        | 0.8   | 80   | 1.7390          |
-| 1.4711        | 0.9   | 90   | 1.7106          |
-| 1.6522        | 1.0   | 100  | 1.6832          |
+| 2.9281        | 0.04  | 10   | 2.3786          |
+| 2.1861        | 0.08  | 20   | 1.9991          |
+| 1.5698        | 0.12  | 30   | 1.9460          |
+| 1.6891        | 0.16  | 40   | 1.8457          |
+| 1.6863        | 0.2   | 50   | 1.8188          |
+| 1.4201        | 0.24  | 60   | 1.7638          |
+| 1.6936        | 0.28  | 70   | 1.7349          |
+| 1.5696        | 0.32  | 80   | 1.6898          |
+| 1.6084        | 0.36  | 90   | 1.7269          |
+| 1.5357        | 0.4   | 100  | 1.7332          |
+| 1.6684        | 0.44  | 110  | 1.6728          |
+| 1.2216        | 0.48  | 120  | 1.6563          |
+| 1.5176        | 0.52  | 130  | 1.6376          |
+| 1.5256        | 0.56  | 140  | 1.6653          |
+| 1.3695        | 0.6   | 150  | 1.6317          |
+| 1.3191        | 0.64  | 160  | 1.6261          |
+| 1.4091        | 0.68  | 170  | 1.6145          |
+| 1.5681        | 0.72  | 180  | 1.6007          |
+| 1.7259        | 0.76  | 190  | 1.6076          |
+| 1.5123        | 0.8   | 200  | 1.6836          |
+| 1.387         | 0.84  | 210  | 1.7292          |
+| 1.5837        | 0.88  | 220  | 1.7177          |
+| 1.6955        | 0.92  | 230  | 1.7232          |
+| 1.7862        | 0.96  | 240  | 1.7714          |
+| 1.79          | 1.0   | 250  | 1.7791          |
 
 
 ### Framework versions

adapter_config.json

@@ -20,12 +20,12 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
-    "k_proj",
-    "q_proj",
     "lm_head",
-    "down_proj",
     "gate_proj",
     "o_proj",
+    "q_proj",
+    "k_proj",
+    "down_proj",
     "up_proj",
     "v_proj"
   ],

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:82bb7bc6d79cfbd21fba3c1a21acdf0f856c2e5069d4ebb1a072f1c5800d170e
+oid sha256:e8c5f32b7d6fa90f84d910a465422cf94db936e145caeb9b099519e4bd3a5b34
 size 1217458040

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:d42c39e2b210ffe4ce74535dcd290c0e6bc67e651401bfa50cf9285e071e672c
-size 5112
+oid sha256:8724a5519057acb5e2a9f5c7a800ead5125c04cddc074795c276676fc7cfe0da
+size 5048