Update README.md
Browse files
README.md
CHANGED
|
@@ -11,7 +11,7 @@ pipeline_tag: text-generation
|
|
| 11 |
|
| 12 |
A continued pretrained version of unsloth/Qwen2.5-7B model using unsloth's low rank adaptation on a dataset of [DTF](dtf.ru) posts. The adapter is already merged with the model.
|
| 13 |
|
| 14 |
-
For pretraining, posts from [SubMaroon/DTF_comments_Responses_Counts](https://huggingface.co/datasets/SubMaroon/DTF_Comments_Responses_Counts) were selected, deduplicated by simple `df.unique` and filtered by length of 1000 < x < 128000 tokens.
|
| 15 |
|
| 16 |
LoRA hyperparameters:
|
| 17 |
|
|
|
|
| 11 |
|
| 12 |
A continued pretrained version of unsloth/Qwen2.5-7B model using unsloth's low rank adaptation on a dataset of [DTF](dtf.ru) posts. The adapter is already merged with the model.
|
| 13 |
|
| 14 |
+
For pretraining, posts from [SubMaroon/DTF_comments_Responses_Counts](https://huggingface.co/datasets/SubMaroon/DTF_Comments_Responses_Counts) were selected, deduplicated by simple `df.unique` and filtered by length of 1000 < x < 128000 tokens. The training dataset size was roughly 75M tokens.
|
| 15 |
|
| 16 |
LoRA hyperparameters:
|
| 17 |
|