Hugging Face's logo

LoneStriker
/
miquella-120b-3.0bpw-h6-exl2

Text Generation
Transformers
Safetensors
llama
mergekit
Merge
text-generation-inference
Inference Endpoints
Model card Files Community
1
New discussion

Will any 120b model currently fit on a single 24GB VRAM card through any app I can run on PC? (aka 4090)

15
#1 opened 9 months ago by clevnumb