1aurent commited on
Commit
a4eb74d
1 Parent(s): 978ce1a

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +145 -2
README.md CHANGED
@@ -1,8 +1,151 @@
1
  ---
2
  tags:
3
- - image-classification
4
  - timm
 
 
5
  library_name: timm
6
- license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
7
  ---
 
8
  # Model card for vit_base_patch8_224.kaiko_ai_towards_large_pathology_fms
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  tags:
 
3
  - timm
4
+ - feature-extraction
5
+ - image-classification
6
  library_name: timm
7
+ license: other
8
+ license_name: kaiko-non-commercial
9
+ license_link: https://github.com/kaiko-ai/towards_large_pathology_fms/blob/a62a0c54719d858371aefa0fcab6ec4b34c86c4c/LICENSE
10
+ metrics:
11
+ - accuracy
12
+ model-index:
13
+ - name: kaiko
14
+ results:
15
+ - task:
16
+ type: image-classification
17
+ name: Image Classification
18
+ dataset:
19
+ name: BACH
20
+ type: image-classification
21
+ metrics:
22
+ - type: accuracy
23
+ value: 0.865
24
+ name: Accuracy
25
+ verified: false
26
+ - task:
27
+ type: image-classification
28
+ name: Image Classification
29
+ dataset:
30
+ name: CRC-NCT-HE
31
+ type: image-classification
32
+ metrics:
33
+ - type: accuracy
34
+ value: 0.956
35
+ name: Accuracy
36
+ verified: false
37
+ - task:
38
+ type: image-classification
39
+ name: Image Classification
40
+ dataset:
41
+ name: MHIST
42
+ type: image-classification
43
+ metrics:
44
+ - type: accuracy
45
+ value: 0.809
46
+ name: Accuracy
47
+ verified: false
48
+ - task:
49
+ type: image-classification
50
+ name: Image Classification
51
+ dataset:
52
+ name: PCam
53
+ type: image-classification
54
+ metrics:
55
+ - type: accuracy
56
+ value: 0.921
57
+ name: Accuracy
58
+ verified: false
59
+ - task:
60
+ type: image-classification
61
+ name: Image Classification
62
+ dataset:
63
+ name: TP53
64
+ type: image-classification
65
+ metrics:
66
+ - type: accuracy
67
+ value: 0.659
68
+ name: Accuracy
69
+ verified: false
70
+ - task:
71
+ type: image-classification
72
+ name: Image Classification
73
+ dataset:
74
+ name: CoNSeP
75
+ type: image-classification
76
+ metrics:
77
+ - type: accuracy
78
+ value: 0.741
79
+ name: Accuracy
80
+ verified: false
81
  ---
82
+
83
  # Model card for vit_base_patch8_224.kaiko_ai_towards_large_pathology_fms
84
+
85
+ ![](https://github.com/kaiko-ai/towards_large_pathology_fms/blob/a62a0c54719d858371aefa0fcab6ec4b34c86c4c/docs/images/kaiko-logo.png?raw=true)
86
+
87
+ ## Model Details
88
+
89
+ - **Model Type:** Feature backbone
90
+ - **Model Stats:**
91
+ - Params: 86M (base)
92
+ - Image size: 224 x 224 x 3
93
+ - Patch size: 8 x 8 x 3
94
+ - **Repository:** [github.com:kaiko-ai/towards_large_pathology_fms](https://github.com/kaiko-ai/towards_large_pathology_fms)
95
+ - **Original Weights:** [github.com:kaiko-ai/towards_large_pathology_fms/0.0.1](https://github.com/kaiko-ai/towards_large_pathology_fms/releases/tag/0.0.1)
96
+ - **Papers:**
97
+ - [Towards Large-Scale Training of Pathology Foundation Models](https://arxiv.org/abs/2404.15217)
98
+
99
+ ## Model Usage
100
+
101
+ ### Image Embeddings
102
+
103
+ ```python
104
+ from torchvision.transforms import v2
105
+ from PIL import Image
106
+ import requests
107
+ import torch
108
+ import timm
109
+ import io
110
+
111
+ # get example histology image
112
+ url = "https://encrypted-tbn0.gstatic.com/images?q=tbn:ANd9GcQc7_xZpGOfQT7sxKwf2w5lL4GAq6IX_CbTzP1NGeenzA&s"
113
+ image = Image.open(io.BytesIO(requests.get(url).content))
114
+
115
+ # load model from the hub
116
+ model = timm.create_model(
117
+ model_name="hf-hub:1aurent/vit_base_patch8_224.kaiko_ai_towards_large_pathology_fms",
118
+ dynamic_img_size=True,
119
+ pretrained=True,
120
+ ).eval()
121
+
122
+ # get image transform
123
+ preprocessing = v2.Compose(
124
+ [
125
+ v2.ToImage(),
126
+ v2.Resize(size=224),
127
+ v2.CenterCrop(size=224),
128
+ v2.ToDtype(torch.float32, scale=True),
129
+ v2.Normalize(
130
+ mean=(0.5, 0.5, 0.5),
131
+ std=(0.5, 0.5, 0.5),
132
+ ),
133
+ ]
134
+ )
135
+
136
+ data = preprocessing(image).unsqueeze(0) # input is a (batch_size, num_channels, img_size, img_size) shaped tensor
137
+ output = model(data) # output is a (batch_size, num_features) shaped tensor
138
+ ```
139
+
140
+ ## Citation
141
+
142
+ ```bibtex
143
+ @misc{ai2024largescale,
144
+ title = {Towards Large-Scale Training of Pathology Foundation Models},
145
+ author = {kaiko.ai and Nanne Aben and Edwin D. de Jong and Ioannis Gatopoulos and Nicolas Känzig and Mikhail Karasikov and Axel Lagré and Roman Moser and Joost van Doorn and Fei Tang},
146
+ year = {2024},
147
+ eprint = {2404.15217},
148
+ archivePrefix = {arXiv},
149
+ primaryClass = {cs.CV}
150
+ }
151
+ ```