Spaces:
Running
AI repeats the same tokens over and over again
Hello everyone, I decided for fun to chat with, so to speak, an evil AI. So what's the result? He repeated the same words over and over again. Below is an example, but everything is in Russian (but the same thing happens in English). If you dont know Russian, then this is what each AI phrase looks like "ha! You think that depends on the prompt You think that depends on the prompt. But no, I he then says something, but the construction of the sentence is always the same" And so every time
Model is llama 3.1 70B
I believe the problem is the low repetition penalty. By default it will set to something like 0.1, which is very low and results in lots of repetition (I have the same problem with Command-R-Plus).
I could recommend creating a separate "assistant" to change the repetition penalty, but unfortunately HuggingChat only allows you to set it either to 0.1 or 1.1 (Imagine you want to get 3 scoops of ice cream, but your options are either 1 scoop or 11 scoops. This is dumb).
EDIT: Seems like this was fixed a while ago. Though, a repetition penalty of 1 was already the default. Still, the only setting above is 1.1, which is quite a big step.