leoxiaobin haipingwu commited on
Commit
37070d5
1 Parent(s): a15cd1f

add_grounding_example (#2)

Browse files

- add grounding example (b9f04dd36709c6a9c921b97770a440eaad1686ad)
- add grounding example (6060dccbb4333867f062c0e8e2ed8eae48ce5755)


Co-authored-by: Haiping Wu <[email protected]>

Files changed (1) hide show
  1. README.md +14 -1
README.md CHANGED
@@ -85,7 +85,10 @@ processor = AutoProcessor.from_pretrained("microsoft/Florence-2-base", trust_rem
85
  url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true"
86
  image = Image.open(requests.get(url, stream=True).raw)
87
 
88
- def run_example(prompt):
 
 
 
89
 
90
  inputs = processor(text=prompt, images=image, return_tensors="pt")
91
  generated_ids = model.generate(
@@ -169,6 +172,16 @@ prompt = <REGION_PROPOSAL>
169
  run_example(prompt)
170
  ```
171
 
 
 
 
 
 
 
 
 
 
 
172
  for More detailed examples, please refer to [notebook](https://huggingface.co/microsoft/Florence-2-large/blob/main/sample_inference.ipynb)
173
  </details>
174
 
 
85
  url = "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/transformers/tasks/car.jpg?download=true"
86
  image = Image.open(requests.get(url, stream=True).raw)
87
 
88
+ def run_example(prompt, text_input=None):
89
+
90
+ if text_input is not None:
91
+ prompt = prompt + text_input
92
 
93
  inputs = processor(text=prompt, images=image, return_tensors="pt")
94
  generated_ids = model.generate(
 
172
  run_example(prompt)
173
  ```
174
 
175
+ ### Caption to Phrase Grounding
176
+ caption to phrase grounding task requires additional text input, i.e. caption.
177
+
178
+ Caption to phrase grounding results format:
179
+ {'\<CAPTION_TO_PHRASE_GROUNDING>': {'bboxes': [[x1, y1, x2, y2], ...], 'labels': ['', '', ...]}}
180
+ ```python
181
+ task_prompt = '<CAPTION_TO_PHRASE_GROUNDING>'
182
+ results = run_example(task_prompt, text_input="A green car parked in front of a yellow building.")
183
+ ```
184
+
185
  for More detailed examples, please refer to [notebook](https://huggingface.co/microsoft/Florence-2-large/blob/main/sample_inference.ipynb)
186
  </details>
187