falcon-7b-instruct-ft-instagram-adapters-trainer-2

README.md

@@ -34,7 +34,7 @@ More information needed
 ### Training hyperparameters
 
 The following hyperparameters were used during training:
-- learning_rate: 0.0002
+- learning_rate: 0.02
 - train_batch_size: 4
 - eval_batch_size: 8
 - seed: 42
@@ -43,7 +43,7 @@ The following hyperparameters were used during training:
 - optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
 - lr_scheduler_type: constant
 - lr_scheduler_warmup_ratio: 0.03
-- training_steps: 500
+- training_steps: 20
 - mixed_precision_training: Native AMP
 
 ### Training results

adapter_config.json

@@ -19,10 +19,10 @@
   "rank_pattern": {},
   "revision": null,
   "target_modules": [
+    "dense_4h_to_h",
     "query_key_value",
-    "dense_h_to_4h",
     "dense",
-    "dense_4h_to_h"
+    "dense_h_to_4h"
   ],
   "task_type": "CAUSAL_LM",
   "use_rslora": false

adapter_model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:4699f11ee549e908e5de2f10521979aa21f6175d1deacfba28c89e8a49c4e225
+oid sha256:effcd3c9f4daf872ebb52d9d050d58f127c2d742267eaeab77f5726ef0046825
 size 522227376

runs/Jan18_10-17-25_4e5acb56f0db/events.out.tfevents.1705573053.4e5acb56f0db.4020.0

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe7b31b4390f14f0f2743f6d1c1576bcab114bbd573e7d56d637df8c40bdc34a
+size 5522

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:c1f6f70619b34e35719bed8cbe1ba864b41e2510a83f204a94c745e8a6d5b64f
+oid sha256:928ae454a3915a9895db74def7092addf02f2e7e75b2351a522b166c23d0d42b
 size 4664