is it possible to replace my image processor 'openai/clip-vit-large-patch14-336'?

#13
by hmhanna - opened

Hello Team!

I see the following settings in the config.json. Is there a way to replace 'openai/clip-vit-large-patch14-336' with 'openai/clip-vit-base-patch32'? I want a faster smaller model that I think is good enough for my task

"img_processor": {
"image_dim_out": 1024,
"model_name": "openai/clip-vit-large-patch14-336",
"name": "clip_vision_model",
"num_img_tokens": 144
},

hmhanna changed discussion title from is it possible to replace 'openai/clip-vit-large-patch14-336'? to is it possible to replace my image processor 'openai/clip-vit-large-patch14-336'?
Microsoft org

hi @hmhanna , note we train vision encoders as well, so any replacement would need a retrain for the model to work.

haipingwu changed discussion status to closed

Sign up or log in to comment