Update README.md

README.md

@@ -11,73 +11,79 @@ pipeline_tag: object-detection
 
 # Model Card for Model ID
 
-<!-- Provide a quick summary of what the model is/does. -->
 
+## Table of Contents
+
+1. [Model Details](#model-details)
+2. [Model Sources](#model-sources)
+3. [How to Get Started with the Model](#how-to-get-started-with-the-model)
+4. [Training Details](#training-details)
+5. [Evaluation](#evaluation)
+6. [Model Architecture and Objective](#model-architecture-and-objective)
+7. [Citation](#citation)
 
 
 ## Model Details
 
+We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR. This new formulation directly uses box coordinates as queries in Transformer decoders and dynamically updates them layer-by-layer. Using box coordinates not only helps using explicit positional priors to improve the query-to-feature similarity and eliminate the slow training convergence issue in DETR, but also allows us to modulate the positional attention map using the box width and height information. Such a design makes it clear that queries in DETR can be implemented as performing soft ROI pooling layer-by-layer in a cascade manner. As a result, it leads to the best performance on MS-COCO benchmark among the DETR-like detection models under the same setting, e.g., AP 45.7\% using ResNet50-DC5 as backbone trained in 50 epochs. We also conducted extensive experiments to confirm our analysis and verify the effectiveness of our methods.
+
 ### Model Description
 
 <!-- Provide a longer summary of what this model is. -->
 
 This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated.
 
-- **Developed by:** [More Information Needed]
-- **Funded by [optional]:** [More Information Needed]
-- **Shared by [optional]:** [More Information Needed]
-- **Model type:** [More Information Needed]
-- **Language(s) (NLP):** [More Information Needed]
-- **License:** [More Information Needed]
-- **Finetuned from model [optional]:** [More Information Needed]
-
-### Model Sources [optional]
+- **Developed by:** Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang
+- **Funded by:** IDEA-Research
+- **Shared by:** David Hajdu
+- **Model type:** DAB-DETR
+- **License:**  Apache-2.0
+- 
+### Model Sources 
 
 <!-- Provide the basic links for the model. -->
 
-- **Repository:** [More Information Needed]
-- **Paper [optional]:** [More Information Needed]
-- **Demo [optional]:** [More Information Needed]
-
-## Uses
-
-<!-- Address questions around how the model is intended to be used, including the foreseeable users of the model and those affected by the model. -->
-
-### Direct Use
-
-<!-- This section is for the model use without fine-tuning or plugging into a larger ecosystem/app. -->
-
-[More Information Needed]
-
-### Downstream Use [optional]
-
-<!-- This section is for the model use when fine-tuned for a task, or when plugged into a larger ecosystem/app -->
-
-[More Information Needed]
-
-### Out-of-Scope Use
+- **Repository:** https://github.com/IDEA-Research/DAB-DETR
+- **Paper:** https://arxiv.org/abs/2201.12329
 
-<!-- This section addresses misuse, malicious use, and uses that the model will not work well for. -->
+## How to Get Started with the Model
 
-[More Information Needed]
+Use the code below to get started with the model.
 
-## Bias, Risks, and Limitations
+```python
+import torch
+import requests
 
-<!-- This section is meant to convey both technical and sociotechnical limitations. -->
+from PIL import Image
+from transformers import AutoModelForObjectDetection, AutoImageProcessor
 
-[More Information Needed]
+url = 'http://images.cocodataset.org/val2017/000000039769.jpg' 
+image = Image.open(requests.get(url, stream=True).raw)
 
-### Recommendations
+image_processor = AutoImageProcessor.from_pretrained("davidhajdu/dab-detr-resnet-50")
+model = AutoModelForObjectDetection.from_pretrained("davidhajdu/dab-detr-resnet-50")
 
-<!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. -->
+inputs = image_processor(images=image, return_tensors="pt")
 
-Users (both direct and downstream) should be made aware of the risks, biases and limitations of the model. More information needed for further recommendations.
+with torch.no_grad():
+    outputs = model(**inputs)
 
-## How to Get Started with the Model
-
-Use the code below to get started with the model.
+results = image_processor.post_process_object_detection(outputs, target_sizes=torch.tensor([image.size[::-1]]), threshold=0.3)
 
-[More Information Needed]
+for result in results:
+    for score, label_id, box in zip(result["scores"], result["labels"], result["boxes"]):
+        score, label = score.item(), label_id.item()
+        box = [round(i, 2) for i in box.tolist()]
+        print(f"{model.config.id2label[label]}: {score:.2f} {box}")
+```
+This should output
+```
+cat: 0.87 [14.7, 49.39, 320.52, 469.28]
+remote: 0.86 [41.08, 72.37, 173.39, 117.2]
+cat: 0.86 [344.45, 19.43, 639.85, 367.86]
+remote: 0.61 [334.27, 75.93, 367.92, 188.81]
+couch: 0.59 [-0.04, 1.34, 639.9, 477.09]
+```
 
 ## Training Details
 
@@ -85,15 +91,15 @@ Use the code below to get started with the model.
 
 <!-- This should link to a Dataset Card, perhaps with a short stub of information on what the training data is all about as well as documentation related to data pre-processing or additional filtering. -->
 
-[More Information Needed]
+The DAB-DETR model was trained on [COCO 2017 object detection](https://cocodataset.org/#download), a dataset consisting of 118k/5k annotated images for training/validation respectively. 
 
 ### Training Procedure
 
 <!-- This relates heavily to the Technical Specifications. Content here should link to that section when it is relevant to the training procedure. -->
 
-#### Preprocessing [optional]
+#### Preprocessing
 
-[More Information Needed]
+Images are resized/rescaled such that the shortest side is at least 480 and at most 800 pixels and the long size is at most 1333 pixels, and normalized across the RGB channels with the ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225).
 
 
 #### Training Hyperparameters
@@ -174,7 +180,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 
 [More Information Needed]
 
-## Citation [optional]
+## Citation
 
 <!-- If there is a paper or blog post introducing the model, the APA and Bibtex information for that should go in this section. -->
 
@@ -192,7 +198,7 @@ Carbon emissions can be estimated using the [Machine Learning Impact calculator]
 ```
 
 
-## Model Card Authors [optional]
+## Model Card Authors
 
 [David Hajdu](https://huggingface.co/davidhajdu)