bartowski/Cat-Llama-3-70B-instruct-GGUF · It's an interesting one for sure

I'm a beginner, I run AI model in oobabooga' UI with pretty much default settings and profiles, and under these circumstances this one's impressed me. I've tried some light RP and then some translating from one language to another and the results were very stable - no hallucinating, great attention to detail etc. The only problem - it talks a lot (compared to some other models I tried with default settings). :)
And it's kinda hard to run on my 24+24 Gb of VRAM - I've tried the Q4_K_M version so far. Tho it crashes pretty soon at n_ctx 1536, so I tried around 1200, but, tbh, without any understanding of what value would actually be good.