Hugging Face's logo

TheBloke
/
WizardCoder-Python-34B-V1.0-GPTQ

Text Generation
Transformers
Safetensors
llama
code
Eval Results
text-generation-inference
4-bit precision
gptq
Model card Files Community
6

ValueError: not enough values to unpack (expected 3, got 2)

#2
by rhamnett - opened
Discussion
rhamnett

Hello getting the following error when generating a response using the example in the Model card.

rhamnett

Fixed with inject_fused_attention=False 👍👍

lucasjin

@rhamnett hello, got same error, where did u injeet this?

TheBloke
 Owner

it goes in the .from_quantized() call

lucasjin

@TheBloke hello, I quantized a model, but can not load it with ctransformers

ValueError: No model file found in directory out/quantized_openbuddy-coder-34b-v11-bf16'

but your model is OK, why? am using safetensors

TheBloke
 Owner

ctransformers? ctransformers is for GGML/GGUF models, not GPTQ models. You want normal Huggingface Transformers to load GPTQ models

lucasjin
•
edited Aug 31, 2023

but ctransformers also supports GPTQ via exallama

Sign up or log in to comment