Impressive model but.
Really impressive model, especially so quickly after L3 release. It's quite refreshing compared to other, older models.
My only grip with it, and it's minor AF. When the bot is chaining
action. spoken dialog. action. spoken dialog.
it often ends up with
action. spoken dialog.* action.*
or
action. spoken dialog.action.
instead. It's fairly common, so I'm sure you noticed it. It's especially prevalent in longer RP (200tks+) responses.
It's not really a big deal, and easy enough to fix on the fly (in my UI), but maybe you could double check your dataset for instances of badly chained action/dialog content to see if there's a reason why.
(yes, I'm using the correct L3 formatting.)
This is a problem that plagues pretty much every l3 8b atm, and its something that i am working on slowly.
Oh, that's interesting! Now that you mention it, it's true that I've only seen that very particular issue in L3-8B models... It did feel it less common in the "Aurora-L3-8B-Llama3.Q8_0-imat" model, but I didn't run it for nearly as long as this one so far.
Anyway, thank you for the clarification.
@SerialKicked Aura was trained with an rp dataset jeiku made to deal with a similar situation in certain mistral models.
Oh, that's interesting! Now that you mention it, it's true that I've only seen that very particular issue in L3-8B models... It did feel it less common in the "Aurora-L3-8B-Llama3.Q8_0-imat" model, but I didn't run it for nearly as long as this one so far.
Anyway, thank you for the clarification.
Aurora received training data to combat the issue. IIrc correctly that data should have been included in Nitral's merges, but it was pretty far back in the lineage.
I was having this problem to a point that telling it to keep responses short and limited to one sentence of action and one sentence of utterance wouldn't even work and it'd still just go on forever. Messed with jeiku/Chaos_RP_l3_8B a bit and it followed that system prompt more consistently, usually adding another action after an utterance only part of the time, but most of the time adhering to one action and one utterance per response.
Another weird thing that seems unique to Poppy (in my experience, I've not yet experimented with many L3 models), and maybe tied to the issue at hand: I've seen the model switch back and forth between 3rd person action and 1st person action on occasions, even if the conversation was written with 3rd person actions from the start (with a full 8K context window). And when it does so, the "*" tends to be all over the place.
It's kinda weird, as if anything, having used tons of models, they tend to reset to 3rd person mode on their own, here it's kinda the opposite. I have yet to try to do the opposite (do everything in 1st person to see if it's better at sticking to it).
I might be wrong but it sometimes feels like Poppy (with some minor changes) would perform better as a storyteller than as a pure chat bot.
if you are getting endless generation with poppy, theres something wrong with the template you are using. Im running it both in exl2 and gguf at the same time for kcpp's vision and the speed of tabby and i cant recreate this issue other than using an incorrect template.
Every version of poppy follows the official l3 format.
Yeah, it's definitely not infinitely generating using the L3 instruct template, it can be quite lengthy at times, but that's about it.
I - Lewdiculous - linked presets in the big orange Warning box for the relevant models. Just limit response length and use Trim Incomplete Sequences for now.
Another weird thing that seems unique to Poppy (in my experience, I've not yet experimented with many L3 models), and maybe tied to the issue at hand: I've seen the model switch back and forth between 3rd person action and 1st person action on occasions, even if the conversation was written with 3rd person actions from the start (with a full 8K context window). And when it does so, the "*" tends to be all over the place.
It's kinda weird, as if anything, having used tons of models, they tend to reset to 3rd person mode on their own, here it's kinda the opposite. I have yet to try to do the opposite (do everything in 1st person to see if it's better at sticking to it).
I might be wrong but it sometimes feels like Poppy (with some minor changes) would perform better as a storyteller than as a pure chat bot.
Never even considered making a story telling bot. On another note, can someone show me a l3 model that doesn't have the asterisk problem. Ive tried 32 different l3 models and it was in every single one.
l3 model that doesn't have the asterisk problem
That's literally the Holy Grail. They all seem very prone to issues with misplaced asterisks or quotations.
It's just a feeling, but I'm no expert into model mixing :D
On another note, can someone show me a l3 model that doesn't have the asterisk problem. Ive tried 32 different l3 models and it was in every single one.
I can confirm that @jeiku 's Chaos_RP-L3, which I tried it after reading the responses to the thread, practically never triggers the issue in my experience. Otoh, its writing style is a bit more generic and it's worse at formatting (paragraphs, what are those?). I can't say if it's completely absent or not, but it is noticeably less common.
For reference, i don't use quotes, making my formatting probably easier to digest than a compilation of "text", text and text. Might be a reason why.