[Thanks]
#1
by
Darkknight535
- opened
hehe was looking for this. best i can fit in 30gb vram.
30GB VRAM is probably cutting it quite close, but if you can run, you may want to try out this: https://huggingface.co./Jellon/Llama-3.1-Saoirse-70B-exl2-3bpw
I personally think it's better from the tests I've done.