Failure to Properly Append EOS Token
#1
by
LoafyLemon
- opened
Hi, first of all, thanks for the quant!
There's a small issue; The model keeps adding the word "assistant" instead of properly ending the text with an end-of-sequence (EOS) token, the config files might need updating to reflect the proper EOS id of 128009.
To users: To resolve this on your end, you need to adjust the configuration file generation_config.json. Specifically, you should locate the parameter eos_token_id within this file and change its value from 128001 to 128009. By doing this, you're instructing the model to correctly recognise the end of the generated text and stop appending additional words.