Hugging Face's logo

iulia-elisa
/
XAMI-model

Instance Segmentation
Vision Transformers
CNN
Optical Space Missions
Artefact Mapping
ESA
Model card Files Community
XAMI-model / README.md
iulia-elisa's picture
iulia-elisa
Update README.md
91cf61a verified
preview code
|
raw
history blame
5.73 kB
metadata
tags:
  - Instance Segmentation
  - Vision Transformers
  - CNN
  - Optical Space Missions
  - Artefact Mapping
  - ESA
pretty_name: XAMI-model
license: mit
datasets:
  - iulia-elisa/XAMI-dataset

XAMI-model: XMM-Newton optical Artefact Mapping for astronomical Instance segmentation

This repository contains the weights of the XAMI model. The model is trained on images from the XAMI dataset (available on Github and HuggingFace). The images are astronomical observations from the Optical Monitor (XMM-OM) onboard the XMM-Newton X-ray mission.

Information about the XMM-OM can be found here:

The XAMI model combining a detector and segmentor, while freezing the detector model previously trained on the XAMI dataset.

Figure 1: The XAMI model combining a detector and segmentor, while freezing the detector model previously trained on the XAMI dataset.

Cloning the repository 

git clone https://github.com/ESA-Datalabs/XAMI-model.git
cd XAMI-model

# creating the environment
conda env create -f environment.yaml
conda activate xami_model_env

# Install the package in editable mode
pip install -e .

Downloading the dataset and model checkpoints from HuggingFace

The dataset is splited into train and validation categories and contains annotated artefacts in COCO format for Instance Segmentation. We use multilabel Stratified K-fold (k=4) to balance class distributions across splits. We choose to work with a single dataset splits version (out of 4) but also provide means to work with all 4 versions.

The Dataset-Structure.md offers more details about the dataset structure. We provide the following dataset formats: COCO format for Instance Segmentation (commonly used by Detectron2 models) and YOLOv8-Seg format used by ultralytics.

Check the dataset_and_model.ipynb for downloading the dataset and model weights.

Model Inference

After cloning the repository and setting up the environment, use the following code for model loading and inference:

from xami_model.inference.xami_inference import InferXami

det_type = 'rtdetr' # 'rtdetr'  'yolov8'

detr_checkpoint = f'./xami_model/train/weights/{det_type}_sam_weights/{det_type}_detect_300e_best.pt'
sam_checkpoint = f'./xami_model/train/weights/{det_type}_sam_weights/{det_type}_sam.pth'

detr_sam_pipeline = InferXami(
    device='cuda:0',
    detr_checkpoint=detr_checkpoint,
    sam_checkpoint=sam_checkpoint,
    model_type='vit_t', # the SAM checkpoint and model_type (vit_h, vit_t, etc.) must be compatible
    use_detr_masks=True,
    detr_type=det_type)

masks = detr_sam_pipeline.run_predict('./example_images/S0893811101_M.png', show_masks=True)

For training the model, check the training README.md.

Performance metrics

Cumulative distribution of IoUs between predicted and true masks using RT-DETR as detector.
Figure 2: Cumulative distribution of IoUs between predicted and true masks using RT-DETR as detector.
Category Precision Recall
YOLO-v8 RT-DETR YOLO-v8 RT-DETR
Overall 84.3 62.7 72.1 78.3
Central-Ring 89.3 89.1 94.0 97.0
Read-out-Streak 71.1 68.3 73.3 95.3
Smoke-Ring 80.6 78.1 85.6 93.8
Star-Loop 80.5 71.6 74.1 83.3
Other 100.0 6.2 33.3 22.2

Table 1: Metrics per object detector used.

© Licence

This project is licensed under MIT license.