ljnlonoljpiljm

/

florence-2-large-interleaved-docci-keywords

like 0

A bit of a long story, but short awnser is DOCCI & a custom dataset (nsfw). Custom dataset has 20+ "anchor" words & phrases per image in an effort to make it output specific text. I then concatenate and shuffle DOCCI and the custom dataset and then train with a low learning rate with gradient accumulation (512 accumulation steps with a batch size of 2). If you use a "task" of <KEYWORDS> you'll get an idea of what the custom training data looks like.

oh, I see what you did...i was thinking (and I'm in the progress of) doing the same thing but with OD + region proposal + keywords + more_detailed_caption(docci) (I don't actually know how much region proposal helps but it is easy to "label")