Use of [INST] Tokens in Fine-Tuning with Custom Prompt Structure
Hello,
I'm currently working on fine-tuning the Mistral-7B-Instruct-v0.2 model and I have a question regarding the use of [INST] tokens in the prompts.
In my project, I'm using a custom prompt structure that includes an instruction followed by a "HUMAN: " and "AI: " format. This differs from the one provided in the instruction format, which uses the [INST] and [/INST] tokens.
Given this, I'm curious about the motivation behind adding [INST] to the prompt. Is it a must to use these tokens even if my prompts are structured differently? As I'm doing fine-tuning for the model, I'm wondering if it's still necessary to add the [INST] tokens or if there's a way to adapt this to my custom prompt structure.
Thank you in advance for your guidance.
Hi,
After finetuning the model, my observation is that the model can be used either way. You may simply add an [/INST] token after your prompt. If you skip this token on your prompt, the finetuned model itself adds the token and start generating. You just need the exclude this token on your application.
Hope this helps.