Fix typos in tool_response special token and text
Browse files
bonus-unit1/bonus-unit1.ipynb
CHANGED
@@ -619,8 +619,8 @@
|
|
619 |
" eothink = \"</think>\"\n",
|
620 |
" tool_call=\"<tool_call>\"\n",
|
621 |
" eotool_call=\"</tool_call>\"\n",
|
622 |
-
" tool_response=\"<
|
623 |
-
" eotool_response=\"</
|
624 |
" pad_token = \"<pad>\"\n",
|
625 |
" eos_token = \"<eos>\"\n",
|
626 |
" @classmethod\n",
|
@@ -650,7 +650,7 @@
|
|
650 |
"source": [
|
651 |
"## Step 9: Let's configure the LoRA\n",
|
652 |
"\n",
|
653 |
-
"This is we are going to define the
|
654 |
]
|
655 |
},
|
656 |
{
|
@@ -676,7 +676,7 @@
|
|
676 |
"peft_config = LoraConfig(r=rank_dimension,\n",
|
677 |
" lora_alpha=lora_alpha,\n",
|
678 |
" lora_dropout=lora_dropout,\n",
|
679 |
-
" target_modules=[\"gate_proj\",\"q_proj\",\"lm_head\",\"o_proj\",\"k_proj\",\"embed_tokens\",\"down_proj\",\"up_proj\",\"v_proj\"], #
|
680 |
" task_type=TaskType.CAUSAL_LM)"
|
681 |
]
|
682 |
},
|
@@ -702,7 +702,7 @@
|
|
702 |
},
|
703 |
"outputs": [],
|
704 |
"source": [
|
705 |
-
"username=\"Jofthomas\"#
|
706 |
"output_dir = \"gemma-2-2B-it-thinking-function_calling-V0\" # The directory where the trained model checkpoints, logs, and other artifacts will be saved. It will also be the default name of the model when pushed to the hub if not redefined later.\n",
|
707 |
"per_device_train_batch_size = 1\n",
|
708 |
"per_device_eval_batch_size = 1\n",
|
@@ -1639,7 +1639,7 @@
|
|
1639 |
"\n",
|
1640 |
"You've just **mastered what Function-Calling is and how to fine-tune your model to do Function-Calling**!\n",
|
1641 |
"\n",
|
1642 |
-
"If it's the first time you
|
1643 |
"\n",
|
1644 |
"Also, don't hesitate to try to **fine-tune different models**. The **best way to learn is by trying.**\n",
|
1645 |
"\n",
|
|
|
619 |
" eothink = \"</think>\"\n",
|
620 |
" tool_call=\"<tool_call>\"\n",
|
621 |
" eotool_call=\"</tool_call>\"\n",
|
622 |
+
" tool_response=\"<tool_response>\"\n",
|
623 |
+
" eotool_response=\"</tool_response>\"\n",
|
624 |
" pad_token = \"<pad>\"\n",
|
625 |
" eos_token = \"<eos>\"\n",
|
626 |
" @classmethod\n",
|
|
|
650 |
"source": [
|
651 |
"## Step 9: Let's configure the LoRA\n",
|
652 |
"\n",
|
653 |
+
"This is where we are going to define the parameters of our adapter. These are the most important parameters in LoRA as they define the size and importance of the adapters we are training."
|
654 |
]
|
655 |
},
|
656 |
{
|
|
|
676 |
"peft_config = LoraConfig(r=rank_dimension,\n",
|
677 |
" lora_alpha=lora_alpha,\n",
|
678 |
" lora_dropout=lora_dropout,\n",
|
679 |
+
" target_modules=[\"gate_proj\",\"q_proj\",\"lm_head\",\"o_proj\",\"k_proj\",\"embed_tokens\",\"down_proj\",\"up_proj\",\"v_proj\"], # which layer in the transformers do we target ?\n",
|
680 |
" task_type=TaskType.CAUSAL_LM)"
|
681 |
]
|
682 |
},
|
|
|
702 |
},
|
703 |
"outputs": [],
|
704 |
"source": [
|
705 |
+
"username=\"Jofthomas\"# REPLACE with your Hugging Face username\n",
|
706 |
"output_dir = \"gemma-2-2B-it-thinking-function_calling-V0\" # The directory where the trained model checkpoints, logs, and other artifacts will be saved. It will also be the default name of the model when pushed to the hub if not redefined later.\n",
|
707 |
"per_device_train_batch_size = 1\n",
|
708 |
"per_device_eval_batch_size = 1\n",
|
|
|
1639 |
"\n",
|
1640 |
"You've just **mastered what Function-Calling is and how to fine-tune your model to do Function-Calling**!\n",
|
1641 |
"\n",
|
1642 |
+
"If it's the first time you've done this, it's normal that you're feeling puzzled. Take time to check the documentation and understand each part of the code and why we did it this way.\n",
|
1643 |
"\n",
|
1644 |
"Also, don't hesitate to try to **fine-tune different models**. The **best way to learn is by trying.**\n",
|
1645 |
"\n",
|