Fine-tuning AIDC-AI/Ovis1.6-Llama3.2-3B
#4
by
dedi25
- opened
Has anyone fine-tuned this model? Ovis1.6-Gemma2-9B can be fine -tuned on ms-swift, but I couldn't find anything on this one. I have found some fine-tuning tutorials for Image-Text-toText type VLMs, but those models have the function AutoProcessor, while Ovis doesn't. It uses model.get_text_tokenizer() and model.get_visual_tokenizer() instead. If you find anything relevant or related to this, please send it here.
Thanks in advance!
I recommend trying the latest Ovis2-2B/4B models, which are comparable in size to the Ovis1.6-Llama3.2-3B, and Ovis2 delivers better performance. Additionally, ms-swift now supports fine-tuning for Ovis2.