RuntimeError: Unrecognized tensor type ID: AutocastCUDA
#4
by
Rohith1016
- opened
While using branch - gptq-8bit-128g-actorder_True getting this error "RuntimeError: Unrecognized tensor type ID: AutocastCUDA"
model = AutoModelForCausalLM.from_pretrained(model_name_or_path,
device_map="auto",
trust_remote_code=False,
revision="gptq-8bit-128g-actorder_True")
Issue is caused by using PyTorch 2.1
Either build AutoGPTQ 0.4.2 from source, or build the latest AutoGPTQ from source, or wait a little while until the new AutoGPTQ 0.5 release is made, which will have pre-built wheels for PyTorch 2.1. Not sure when AutoGPTQ 0.5.0 is releasing, but it should be pretty soon - maybe today or tomorrow.