EOS issues
#1
by
dillfrescott
- opened
@migtissera
In tokenizer_config.json
you have "eos_token": "</s>",
- is this correct?
Yep! That’s how I trained it.
Maybe its a llama.cpp setting of mine causing it?
#!/bin/bash
./main -n -1 -ins -m "$1" --color -c 4096 --multiline-input -t 14 -ngl 48 --log-disable
Thats my boot script
The GPTQ version works well. I had to add EOS and BOS tokens to the tokenizer.
Can you check here @dillfrescott ? https://github.com/ggerganov/llama.cpp/issues/1776
Hmm interesting. What should I set that line to before compiling it? Im bad at this
I think the issue is that its not throwing the EOS token when it should, and just rambling on until forced to stop.
Maybe @wolfram can help here?