bartowski

/

DeepSeek-R1-Distill-Qwen-14B-GGUF

like 80

I've got the same issue, it just doesn't load.

19:21:56-731601 INFO Loading "DeepSeek-R1-Distill-Qwen-14B-Q8_0.gguf"
19:21:57-055526 INFO llama.cpp weights detected: "models\DeepSeek-R1-Distill-Qwen-14B-Q8_0.gguf"
llama_model_load_from_file: using device CUDA0 (NVIDIA GeForce RTX 4070 Ti SUPER) - 15089 MiB free
llama_model_load: error loading model: tensor 'blk.46.ffn_gate.weight' data is not within the file bounds, model is corrupted or incomplete
llama_model_load_from_file: failed to load model
19:21:57-108380 ERROR Failed to load the model.
Traceback (most recent call last):
File "X:\Ai\Text-Generation-Webui\modules\ui_model_menu.py", line 214, in load_model_wrapper
shared.model, shared.tokenizer = load_model(selected_model, loader)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\Ai\Text-Generation-Webui\modules\models.py", line 90, in load_model
output = load_func_maploader
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\Ai\Text-Generation-Webui\modules\models.py", line 280, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_file)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "X:\Ai\Text-Generation-Webui\modules\llamacpp_model.py", line 111, in from_pretrained
result.model = Llama(**params)
^^^^^^^^^^^^^^^
File "X:\Ai\Text-Generation-Webui\installer_files\env\Lib\site-packages\llama_cpp_cuda_tensorcores\llama.py", line 369, in init
internals.LlamaModel(
File "X:\Ai\Text-Generation-Webui\installer_files\env\Lib\site-packages\llama_cpp_cuda_tensorcores_internals.py", line 56, in init
raise ValueError(f"Failed to load model from file: {path_model}")
ValueError: Failed to load model from file: models\DeepSeek-R1-Distill-Qwen-14B-Q8_0.gguf

Exception ignored in: <function LlamaCppModel.__del__ at 0x000001E5804FF920>
Traceback (most recent call last):
File "X:\Ai\Text-Generation-Webui\modules\llamacpp_model.py", line 62, in del
del self.model
^^^^^^^^^^
AttributeError: 'LlamaCppModel' object has no attribute 'model'

Text generation webui relies on llama-cpp-python which hasn't been updated to support the DeepSeek distills yet