--- library_name: transformers tags: - vision - object-detection license: apache-2.0 language: - en pipeline_tag: object-detection --- # Model Card for Model ID ## Table of Contents 1. [Model Details](#model-details) 2. [Model Sources](#model-sources) 3. [How to Get Started with the Model](#how-to-get-started-with-the-model) 4. [Training Details](#training-details) 5. [Evaluation](#evaluation) 6. [Model Architecture and Objective](#model-architecture-and-objective) 7. [Citation](#citation) ## Model Details ![image/png](https://github.com/conditionedstimulus/hf_media/blob/main/dab_detr_convergence_plot.png) > We present in this paper a novel query formulation using dynamic anchor boxes for DETR (DEtection TRansformer) and offer a deeper understanding of the role of queries in DETR. This new formulation directly uses box coordinates as queries in Transformer decoders and dynamically updates them layer-by-layer. Using box coordinates not only helps using explicit positional priors to improve the query-to-feature similarity and eliminate the slow training convergence issue in DETR, but also allows us to modulate the positional attention map using the box width and height information. Such a design makes it clear that queries in DETR can be implemented as performing soft ROI pooling layer-by-layer in a cascade manner. As a result, it leads to the best performance on MS-COCO benchmark among the DETR-like detection models under the same setting, e.g., AP 45.7\% using ResNet50-DC5 as backbone trained in 50 epochs. We also conducted extensive experiments to confirm our analysis and verify the effectiveness of our methods. ### Model Description This is the model card of a 🤗 transformers model that has been pushed on the Hub. This model card has been automatically generated. - **Developed by:** Shilong Liu, Feng Li, Hao Zhang, Xiao Yang, Xianbiao Qi, Hang Su, Jun Zhu, Lei Zhang - **Funded by:** IDEA-Research - **Shared by:** David Hajdu - **Model type:** DAB-DETR - **License:** Apache-2.0 - ### Model Sources - **Repository:** https://github.com/IDEA-Research/DAB-DETR - **Paper:** https://arxiv.org/abs/2201.12329 ## How to Get Started with the Model Use the code below to get started with the model. ```python import torch import requests from PIL import Image from transformers import AutoModelForObjectDetection, AutoImageProcessor url = 'http://images.cocodataset.org/val2017/000000039769.jpg' image = Image.open(requests.get(url, stream=True).raw) image_processor = AutoImageProcessor.from_pretrained("davidhajdu/dab-detr-resnet-50") model = AutoModelForObjectDetection.from_pretrained("davidhajdu/dab-detr-resnet-50") inputs = image_processor(images=image, return_tensors="pt") with torch.no_grad(): outputs = model(**inputs) results = image_processor.post_process_object_detection(outputs, target_sizes=torch.tensor([image.size[::-1]]), threshold=0.3) for result in results: for score, label_id, box in zip(result["scores"], result["labels"], result["boxes"]): score, label = score.item(), label_id.item() box = [round(i, 2) for i in box.tolist()] print(f"{model.config.id2label[label]}: {score:.2f} {box}") ``` This should output ``` cat: 0.87 [14.7, 49.39, 320.52, 469.28] remote: 0.86 [41.08, 72.37, 173.39, 117.2] cat: 0.86 [344.45, 19.43, 639.85, 367.86] remote: 0.61 [334.27, 75.93, 367.92, 188.81] couch: 0.59 [-0.04, 1.34, 639.9, 477.09] ``` ## Training Details ### Training Data The DAB-DETR model was trained on [COCO 2017 object detection](https://cocodataset.org/#download), a dataset consisting of 118k/5k annotated images for training/validation respectively. ### Training Procedure #### Preprocessing Images are resized/rescaled such that the shortest side is at least 480 and at most 800 pixels and the long size is at most 1333 pixels, and normalized across the RGB channels with the ImageNet mean (0.485, 0.456, 0.406) and standard deviation (0.229, 0.224, 0.225). #### Training Hyperparameters - **Training regime:** [More Information Needed] #### Speeds, Sizes, Times [optional] [More Information Needed] ## Evaluation ### Testing Data, Factors & Metrics #### Testing Data [More Information Needed] #### Factors [More Information Needed] #### Metrics [More Information Needed] ### Results [More Information Needed] #### Summary ## Model Examination [optional] [More Information Needed] ## Environmental Impact Carbon emissions can be estimated using the [Machine Learning Impact calculator](https://mlco2.github.io/impact#compute) presented in [Lacoste et al. (2019)](https://arxiv.org/abs/1910.09700). - **Hardware Type:** [More Information Needed] - **Hours used:** [More Information Needed] - **Cloud Provider:** [More Information Needed] - **Compute Region:** [More Information Needed] - **Carbon Emitted:** [More Information Needed] ## Technical Specifications [optional] ### Model Architecture and Objective [More Information Needed] ### Compute Infrastructure [More Information Needed] #### Hardware [More Information Needed] #### Software [More Information Needed] ## Citation **BibTeX:** ```bibtex @inproceedings{ liu2022dabdetr, title={{DAB}-{DETR}: Dynamic Anchor Boxes are Better Queries for {DETR}}, author={Shilong Liu and Feng Li and Hao Zhang and Xiao Yang and Xianbiao Qi and Hang Su and Jun Zhu and Lei Zhang}, booktitle={International Conference on Learning Representations}, year={2022}, url={https://openreview.net/forum?id=oMI9PjOb9Jl} } ``` ## Model Card Authors [David Hajdu](https://huggingface.co./davidhajdu)