ahsanMah commited on
Commit
98f9427
·
1 Parent(s): 19b39bc

Updated readme with some info

Browse files
Files changed (2) hide show
  1. README.md +43 -10
  2. assets/heatmap.webp +0 -0
README.md CHANGED
@@ -1,8 +1,5 @@
1
  ---
2
- title: Ano Edm Gr
3
- emoji: 👀
4
- colorFrom: yellow
5
- colorTo: red
6
  sdk: gradio
7
  sdk_version: 4.36.1
8
  app_file: hfapp.py
@@ -10,14 +7,50 @@ pinned: false
10
  ---
11
 
12
  # Anomaly Localization with Score-Based Diffusion Models
 
 
 
 
 
13
 
14
- Detect anomalies in natural images!
15
 
16
- ### Background
17
- This project aims to add anomaly detection capabilites to score-based diffusion models. The main idea is to learn the distribution of the typical score vectors for each patch position. One can train a postion-condiditoned normalizing flow model to estimate the likelihood of the outputs of the score vector. The upshot is that you can visualize the per-patch likelihood and inspect the resulting heatmap of anomalies! This idea comes from work done as part of my doctoral dissertation, with the underlying methodology described in (this paper)[https://arxiv.org/abs/2010.13132].
18
 
 
19
 
20
- The code builds upon the excellent (EDM2 repository)[https://github.com/NVlabs/edm2/] by NVLabs. Please refer to the original work form more information on the diffusion models.
21
 
22
- ### Caveats
23
- As the underlying models are trained on Imagenet-1k, you may have better success when the subject belongs to one of the 1000 classes it was trianed on.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
+ title: Localizing Anomalies
 
 
 
3
  sdk: gradio
4
  sdk_version: 4.36.1
5
  app_file: hfapp.py
 
7
  ---
8
 
9
  # Anomaly Localization with Score-Based Diffusion Models
10
+ <p align="middle">
11
+ <img src="samples/duckelephant.jpeg" alt="input elephant" width="32%"/>
12
+ <img src="assets/heatmap.webp" alt="heatmap image" width="32%"/>
13
+ </p>
14
+ ## Introduction
15
 
16
+ This project aims to add anomaly detection capabilities to score-based diffusion models, enabling the detection of anomalies in natural images. The core idea is to learn the distribution of typical score vectors for each patch position in an image. By training a position-conditioned normalizing flow model, we can estimate the likelihood of the score vectors' outputs. This allows for visualizing per-patch likelihoods and generating heatmaps of anomalies.
17
 
18
+ ## Background
 
19
 
20
+ The methodology for this project is based on research conducted as part of my doctoral dissertation. The underlying concepts are detailed in the following paper: [Multiscale Score Matching for Out-of-Distribution Detection](https://arxiv.org/abs/2010.13132).
21
 
22
+ This code builds upon the excellent [EDM2 repository](https://github.com/NVlabs/edm2) by NVLabs. For more information on the diffusion models, please refer to the original work.
23
 
24
+ ## Features
25
+
26
+ - **Anomaly Detection**: Identify and localize anomalies in natural images.
27
+ - **Score-Based Diffusion Models**: Utilizes state-of-the-art (non-latent) diffusion models.
28
+ - **Heatmap Visualization**: Generate heatmaps to visualize anomaly likelihoods.
29
+
30
+ ## Installation
31
+
32
+ You can run the gradio app locally. To get started, clone the project and install the dependencies.
33
+
34
+ ```bash
35
+ git clone https://github.com/ahsanMah/localizing-anomalies.git
36
+ cd localizing-anomalies
37
+ pip install -r requirements.txt
38
+ ```
39
+
40
+ The model will work without a GPU but may take 15-30 seconds given your resources. If you have a NVIDIA GPU, install PyTorch with CUDA.
41
+
42
+ ## Usage
43
+ ```bash
44
+ python app.py
45
+ ```
46
+ Then go to [http://localhost:7860](localhost:7860)
47
+
48
+
49
+ ### Notes
50
+ - As the underlying models are trained on Imagenet-1k, you may have better success when the subject belongs to one of the 1000 classes it was trianed on
51
+ - It helps to have the subject centered in the middle of the image. This is due to the resize-center-crop that was used to train the score models
52
+
53
+
54
+ ### Acknowledgements
55
+
56
+ Special thanks to NVLabs for their [EDM2 repository](https://github.com/NVlabs/edm2).
assets/heatmap.webp ADDED