Update README.md
Browse files
README.md
CHANGED
@@ -28,7 +28,7 @@ The model was fine-tuned on `vanessasml/cybersecurity_32k_instruction_input_outp
|
|
28 |
No special data format was applied as [recommended](https://huggingface.co/blog/llama3#fine-tuning-with-%F0%9F%A4%97-trl)
|
29 |
|
30 |
## Training Procedure
|
31 |
-
- **Preprocessing**: Text data were tokenized using the tokenizer corresponding to the base model `
|
32 |
- **Hardware**: The training was performed on GPUs with mixed precision (FP16/BF16) enabled.
|
33 |
- **Optimizer**: Paged AdamW with a cosine learning rate schedule.
|
34 |
- **Epochs**: The model was trained for 1 epoch.
|
|
|
28 |
No special data format was applied as [recommended](https://huggingface.co/blog/llama3#fine-tuning-with-%F0%9F%A4%97-trl)
|
29 |
|
30 |
## Training Procedure
|
31 |
+
- **Preprocessing**: Text data were tokenized using the tokenizer corresponding to the base model `meta-llama/Meta-Llama-3-8B`.
|
32 |
- **Hardware**: The training was performed on GPUs with mixed precision (FP16/BF16) enabled.
|
33 |
- **Optimizer**: Paged AdamW with a cosine learning rate schedule.
|
34 |
- **Epochs**: The model was trained for 1 epoch.
|