Prompt with <image> in the prompt leads to 'RuntimeError: CUDA error: device-side assert triggered'
#6
by
vicaran
- opened
Not an issue but good to know in case anyone runs into this issue. In other multimodal models an image token placeholder is needed in the prompt so it could come up when using those same prompts on this model as it was my case. Once I remove the <image> placeholder from the prompt, the issue when away.
I think this is because the processor for paligemma adds image tokens on its own. This is kind of inconvenient for when you want to input multiple images