bartowski

/

DeepSeek-R1-Distill-Qwen-32B-GGUF

like 73

I just tried to load Q5_K_M in Oobabooga. I updated Oobabooga a few minute ago. It's still not having it. Every other model works, so I'm guessing this one is going to be like Nemotron 51b, with Oobabooga users having to wait for an update.

ERROR DUMP:
llama_model_load: error loading model: error loading model vocabulary: unknown pre-tokenizer type: 'deepseek-r1-qwen'
llama_model_load_from_file: failed to load model
17:14:52-135613 ERROR Failed to load the model.
Traceback (most recent call last):
File "C:\AI\text-generation-webui-main\modules\ui_model_menu.py", line 214, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\text-generation-webui-main\modules\models.py", line 90, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\text-generation-webui-main\modules\models.py", line 280, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\AI\text-generation-webui-main\modules\llamacpp_model.py", line 111, in from_pretrained
result.model = Llama(**params)
^^^^^^^^^^^^^^^
File "C:\AI\text-generation-webui-main\installer_files\env\Lib\site-packages\llama_cpp_cuda_tensorcores\llama.py", line 369, in init
internals.LlamaModel(
File "C:\AI\text-generation-webui-main\installer_files\env\Lib\site-packages\llama_cpp_cuda_tensorcores_internals.py", line 56, in init
raise ValueError(f"Failed to load model from file: {path_model}")
ValueError: Failed to load model from file: models\Deepseek-R1-Qwen-32b-Q5_K_M_GGUF\DeepSeek-R1-Distill-Qwen-32B-Q5_K_M.gguf

Exception ignored in: <function LlamaCppModel.__del__ at 0x000002363D489120>
Traceback (most recent call last):
File "C:\AI\text-generation-webui-main\modules\llamacpp_model.py", line 62, in del
del self.model
^^^^^^^^^^
AttributeError: 'LlamaCppModel' object has no attribute 'model'