Inquiry about reasoning speed
#3
by
dingguofeng
- opened
Hello, I would like to ask how is the inference speed of this HF version? For example, how much time does it take to generate a caption for an image?
3 mins per image on four 3090 gpus, and 1.5 mins per image on three 4090 gpus
2024.08.29.upd:
After a series of optimizations, it now takes eight seconds to generate a caption per image.