8b?
this is the 8b version? what lexi model did you use? the 8b model is only 5bg this is around 15. i couldnt run this on my system and i have 32gb of ram, 8gb of gpu ram and 8gb of vram. might be great for a cloud or a big server but not for a mid tier pc locally. was hoping to replace the 3.1 8b with this :(
@h3xkore I think you misunderstood how it works. This is the 8B model yes. But this is the safetensors file format as it is originally.
If you want to run t his locally for inference, you can use a software like for example LM Studio, and you search for the GGUF format of this model instead (https://huggingface.co./Orenguteng/Llama-3.1-8B-Lexi-Uncensored-GGUF) which is also available from other people who quantize it.
You download a smaller size quantization (meaning it's been compressed to become smaller) and run it locally and you should be able to run it on your PC. You can try different quantization sizes and see which works best for you. Check youtube for resources for LM Studio it's very easy to set up.
i really appreciate you taking the time to educate me in this process and sending me in the right direction. thank you very much.