The generated response cannot stop.
#50
by
shaohuay
- opened
for long prompt, this model will generate non-sense info to fill until the max_new_tokens, is there any way to fix it?
Are you calling apply_chat_template()? This will help ensure your model is better behaved....