Best 72B Model so far... But for only 6K CTX
Did you train this model for only 6K ctx? I fucking love it, but after 6K it's looking its mind and quality drastically decreases.
Huh, that's odd
It was trained at 8192 seqlen, see https://huggingface.co./estrogen/TQ-2.5-72b-RP-Ink-ep2-adpt
Qwen usually generalizes context though
Do you use quant kv cache? It has proven to have a very negative effect on Qwen2.5
I run Qwen2.5-72b-RP-Ink-Q4_K_S with no quantkv, at 11k ctx with flash attention. I notice a heavy loss of performance at 6k ctx and notice loss of personality at 4-5k. I am genuinly saddened. Is it because of my quant? This is by far the best model with rejections. It actually is fun again to chat with an llm.
Okay, I fixed it by adding the character description to authors note at depth 4.
Okay, I fixed it by adding the character description to authors note at depth 4.
Did you blank/remove the default char description and add it to AN? Or keep default as well so it's present twice?
Tried both, having it twice helped a bit.