I was trying to fine-tune llama3 8b but getting following error - TypeError: LlamaForCausalLM.forward() got an unexpected keyword argument 'decoder_input_ids'
Using latest version for these libraries-
!pip install -q -U bitsandbytes
!pip install -q -U git+https://github.com/huggingface/transformers.git
!pip install -q -U git+https://github.com/huggingface/peft.git
!pip install -q -U git+https://github.com/huggingface/accelerate.git
!pip install -q datasets
Traceback (most recent call last):
File "/localhome/local-anbhoite/llama/finetune_llm/Finetuning.py", line 101, in
trainer.train()
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/trl/trainer/sft_trainer.py", line 361, in train
output = super().train(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/transformers/trainer.py", line 1645, in train
return inner_training_loop(
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/transformers/trainer.py", line 1929, in _inner_training_loop
tr_loss_step = self.training_step(model, inputs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/transformers/trainer.py", line 2750, in training_step
loss = self.compute_loss(model, inputs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/transformers/trainer.py", line 2775, in compute_loss
outputs = model(**inputs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/accelerate/utils/operations.py", line 822, in forward
return model_forward(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/accelerate/utils/operations.py", line 810, in call
return convert_to_fp32(self.model_forward(*args, **kwargs))
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/torch/amp/autocast_mode.py", line 16, in decorate_autocast
return func(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/peft/peft_model.py", line 1592, in forward
return self.base_model(
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1532, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1541, in _call_impl
return forward_call(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/peft/tuners/tuners_utils.py", line 179, in forward
return self.model.forward(*args, **kwargs)
File "/localhome/local-anbhoite/.local/lib/python3.10/site-packages/accelerate/hooks.py", line 166, in new_forward
output = module._old_forward(*args, **kwargs)
TypeError: LlamaForCausalLM.forward() got an unexpected keyword argument 'decoder_input_ids'
0%|
I've seen something like this. Look at the JSON structure. The tokenizer takes text and produces input ids, but the structure for training is a very long column of integers.
Yes I solved it. In my case problem was with format of the input.
Yes I solved it. In my case problem was with format of the input.
Could you provide more details? What kind of format of the input is uncorrect.
My inputs contains input_ids and attention_mask and all of them are number, I meet the same problems.
I met the same problem and it turns out that I used the wrong task_type in LoraConfig. For LlamaForCausalLM, the task_type should be TaskType.CAUSAL_LM instead of TaskType.SEQ_2_SEQ_LM.