Update README.md
Browse files
README.md
CHANGED
|
@@ -19,9 +19,13 @@ This repository provides a fine-tuned version of the **Llama-3-1-8b base model**
|
|
| 19 |
- **Parameters**: 8 Billion (8B)
|
| 20 |
- **Precision**: bf16 (Brain Floating Point 16-bit)
|
| 21 |
- **Fine-Tuning Method**: LoRa (Low-Rank Adaptation)
|
|
|
|
|
|
|
|
|
|
|
|
|
| 22 |
- **Datasets Used**:
|
| 23 |
-
- Roleplay Dataset
|
| 24 |
-
- Reasoning and Logic Dataset
|
| 25 |
- **Fine-Tuning Approach**: Iterative Fine-Tuning using self-chat logs
|
| 26 |
|
| 27 |
## Recommended Settings
|
|
|
|
| 19 |
- **Parameters**: 8 Billion (8B)
|
| 20 |
- **Precision**: bf16 (Brain Floating Point 16-bit)
|
| 21 |
- **Fine-Tuning Method**: LoRa (Low-Rank Adaptation)
|
| 22 |
+
- **Lora Rank**: 32
|
| 23 |
+
- **Lora Alpha**: 64
|
| 24 |
+
- **Learning Rate**: 0.0008
|
| 25 |
+
- **Training Epochs**: 4
|
| 26 |
- **Datasets Used**:
|
| 27 |
+
- cat1.0 Roleplay Dataset
|
| 28 |
+
- cat1.0 Reasoning and Logic Dataset
|
| 29 |
- **Fine-Tuning Approach**: Iterative Fine-Tuning using self-chat logs
|
| 30 |
|
| 31 |
## Recommended Settings
|