General discussion/feedback sharing.
Q5_K_S | CTX: 10240 and 8192 | Very prone to talking about bonds and futures.
Wack
Im getting mixed signals.
Well then that's good to hear, can you request their presets and character cards if possible please?
I'm probably messing with her and I want to solve this.
This is from the settings file from st they sent:
{
"temp": 2,
"temperature_last": true,
"top_p": 1,
"top_k": 0,
"top_a": 0,
"tfs": 1,
"epsilon_cutoff": 0,
"eta_cutoff": 0,
"typical_p": 1,
"min_p": 0.1,
"rep_pen": 1,
"rep_pen_range": 0,
"no_repeat_ngram_size": 0,
"penalty_alpha": 0,
"num_beams": 1,
"length_penalty": 1,
"min_length": 0,
"encoder_rep_pen": 1,
"freq_pen": 0,
"presence_pen": 0,
"do_sample": true,
"early_stopping": false,
"add_bos_token": true,
"truncation_length": 2048,
"ban_eos_token": false,
"skip_special_tokens": true,
"streaming": true,
"mirostat_mode": 0,
"mirostat_tau": 8,
"mirostat_eta": 0.1,
"guidance_scale": 1,
"negative_prompt": "",
"grammar_string": "",
"banned_tokens": "",
"type": "ooba",
"rep_pen_size": 0,
"genamt": 1024,
"max_length": 8192
}
Can you get Context and System presets?
Should be useful to also have testing in a similar condition.
If you can upload them to https://catbox.moe
took a big hit overall like i thought though
Can you get Context and System presets?
Should be useful to also have testing in a similar condition.
If you can upload them to https://catbox.moe
Just asked them to when they have a moment.
Geez, ugh, the thing is benchmarks don't reflect the full experience with the model, as long as it not too low, it should be fine, if the people are happy with it, hell, I am happy too.
Oh it definitely performs still, if anything its been getting more stable with each iteration away from 3.0
3.05 was a freak that performed well but also benched super high. (models scoring that high typically drop out in rp performance, have gptisms or have issues following format)
why no Q6_K?
@Bit101 These are just quants I made to test the model. In that regard, Q5_K_M will give you very close quality.
You can make a request at Model-Requests if you need all the usual quants.