Update README.md
Browse files
README.md
CHANGED
@@ -30,10 +30,10 @@ Breaking it down further, each epoch took only 5.8 hours and cost a mere `$19.25
|
|
30 |
- Total finetuning Cost: $57.75
|
31 |
- Model Path: meta-llama/Llama-2-70b-hf
|
32 |
- Dataset: databricks/databricks-dolly-15k
|
33 |
-
- Learning rate:
|
34 |
- Number of epochs: 3
|
35 |
-
- Data split:
|
36 |
-
- Gradient accumulation steps:
|
37 |
|
38 |
license: apache-2.0
|
39 |
---
|
@@ -52,3 +52,9 @@ Prompt Used:
|
|
52 |
[response]
|
53 |
```
|
54 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
30 |
- Total finetuning Cost: $57.75
|
31 |
- Model Path: meta-llama/Llama-2-70b-hf
|
32 |
- Dataset: databricks/databricks-dolly-15k
|
33 |
+
- Learning rate: 0.0002
|
34 |
- Number of epochs: 3
|
35 |
+
- Data split: Training 90% / Validation 10%
|
36 |
+
- Gradient accumulation steps: 4
|
37 |
|
38 |
license: apache-2.0
|
39 |
---
|
|
|
52 |
[response]
|
53 |
```
|
54 |
|
55 |
+
Loss metrics
|
56 |
+
|
57 |
+
Training loss (Blue) Validation Loss (orange):
|
58 |
+
![training loss](train-loss.png "Training loss")
|
59 |
+
|
60 |
+
|