unsloth

/

Qwen2.5-VL-7B-Instruct

like 7

Follow

Unsloth AI 3.36k

I tried to run the qwen 2 vl notebook with the change of the model name to unsloth/Qwen2.5-VL-7B-Instruct, and it does not load the model.

%%capture
import os
if "COLAB_" not in "".join(os.environ.keys()):
    !pip install unsloth
else:
    # Do this only in Colab and Kaggle notebooks! Otherwise use pip install unsloth
    !pip install git+https://github.com/huggingface/transformers accelerate
    !pip install qwen-vl-utils[decord]==0.0.8
    !pip install --no-deps bitsandbytes xformers peft trl triton
    !pip install --no-deps cut_cross_entropy unsloth_zoo
    !pip install sentencepiece protobuf datasets huggingface_hub hf_transfer
    !pip install --no-deps unsloth

from unsloth import FastVisionModel # FastLanguageModel for LLMs
import torch

model, tokenizer = FastVisionModel.from_pretrained(
    "unsloth/Qwen2.5-VL-7B-Instruct",
    load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.
    use_gradient_checkpointing = "unsloth", # True or "unsloth" for long context
)

#ERROR MESSAGE

RuntimeError Traceback (most recent call last)
in <cell line: 0>()
20 ] # More models at https://huggingface.co./unsloth
21
---> 22 model, tokenizer = FastVisionModel.from_pretrained(
23 "unsloth/Qwen2.5-VL-7B-Instruct",
24 load_in_4bit = True, # Use 4bit to reduce memory use. False for 16bit LoRA.

8 frames
/usr/local/lib/python3.11/dist-packages/torch/nn/modules/module.py in load_state_dict(self, state_dict, strict, assign)
2582
2583 if len(error_msgs) > 0:
-> 2584 raise RuntimeError(
2585 "Error(s) in loading state_dict for {}:\n\t{}".format(
2586 self.class.name, "\n\t".join(error_msgs)

RuntimeError: Error(s) in loading state_dict for Linear4bit:
size mismatch for weight: copying a param with shape torch.Size([33947648, 1]) from checkpoint, the shape in current model is torch.Size([3584, 18944]).