Not able to get proper outputs
#6
by
Praneet
- opened
@teknium
Please could I get help in understanding what I'm doing wrong here
What should the format for the prompt be like?
As you can see, the model gives its response but does not end there and goes onto generate more stuff. Running this post 4bit bitsandbytes quantization
@teknium
Please could I get help in understanding what I'm doing wrong here
What should the format for the prompt be like?
As you can see, the model gives its response but does not end there and goes onto generate more stuff. Running this post 4bit bitsandbytes quantization
I'm not @teknium , but maybe setting a stop token to "### human:" would help? I don't use raw Transformers or whatever that is, but I think there's a way.
It should be
USER:
ASSISTANT:
teknium
changed discussion status to
closed