nicolinho/QRM-Llama3.1-8B-v2 · dtype error

18 days ago

•

Not sure if I'm missing a line or an update, but getting this error with v2, while v1 works

File "hf/modules/transformers_modules/nicolinho/QRM-Llama3.1-8B-v2/d862cde4955388681ef443c5d2775b4c3c3fcaa0/modeling_custom.py", line 156, in forward
    rewards = self.regression_layer(hidden_states)

File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)

File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/module.py", line 1747, in _call_impl
    return forward_call(*args, **kwargs)

File "/usr/local/lib/python3.10/dist-packages/torch/nn/modules/linear.py", line 125, in forward
    return F.linear(input, self.weight, self.bias)

RuntimeError: expected mat1 and mat2 to have the same dtype, but got: c10::BFloat16 != float

nicolinho

Owner 17 days ago

That is weird. I just tested and for me it works fine. What are the versions of the related packages like transformers that you are using?

sanderland

17 days ago

transformers 4.47.1
Looking at the logs, the actual error is probably

Some weights of the model checkpoint at nicolinho/QRM-Llama3.1-8B-v2 were not used when initializing LlamaForRewardModelWithGating: ['score.weight']

sanderland

11 days ago

Trying the example on cpu also gives

Some weights of the model checkpoint at nicolinho/QRM-Llama3.1-8B-v2 were not used when initializing LlamaForRewardModelWithGating: ['score.weight']

sanderland changed discussion title from dtype error to dtype error - weights not used? 11 days ago