Add README
Browse files
README.md
CHANGED
@@ -1,8 +1,75 @@
|
|
1 |
---
|
2 |
tags:
|
3 |
-
- image
|
4 |
- timm
|
|
|
|
|
|
|
|
|
|
|
|
|
5 |
library_name: timm
|
6 |
license: apache-2.0
|
7 |
---
|
|
|
8 |
# Model card for H-optimus-0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
---
|
2 |
tags:
|
3 |
+
- image feature extraction
|
4 |
- timm
|
5 |
+
- pathology
|
6 |
+
- histology
|
7 |
+
- medical imaging
|
8 |
+
- self-supervised learning
|
9 |
+
- vision transformer
|
10 |
+
- foundation model
|
11 |
library_name: timm
|
12 |
license: apache-2.0
|
13 |
---
|
14 |
+
|
15 |
# Model card for H-optimus-0
|
16 |
+
|
17 |
+
<p align="center">
|
18 |
+
<img src="./logo.png" width="500" height="180" />
|
19 |
+
</p>
|
20 |
+
|
21 |
+
`H-optimus-0` is an open-source foundation model for histology, developed by [Bioptimus](https://www.bioptimus.com/).
|
22 |
+
The model is a 1.1B parameter vision transformer trained on a proprietary collection of more than 500,000 H&E stained whole slide histology images.
|
23 |
+
For more information, please refer to our GitHub repository [here](https://github.com/bioptimus/releases/tree/main/models/h-optimus/v0?utm_source=owkin&utm_medium=referral&utm_campaign=h-bioptimus-o).
|
24 |
+
|
25 |
+
`H-optimus-0` can be used to extract powerful features from histology images for various downstream applications, such as mutation prediction, survival analysis, or tissue classification.
|
26 |
+
|
27 |
+
## How to use it to extract features.
|
28 |
+
|
29 |
+
The code below can be used to run inference; `H-optimus-0` expects images of size 224x224 that were extracted at 0.5 microns per pixel.
|
30 |
+
```python
|
31 |
+
from huggingface_hub import login
|
32 |
+
import torch
|
33 |
+
import timm
|
34 |
+
from torchvision import transforms
|
35 |
+
|
36 |
+
# Login to the Hugging Face hub, using your user access token that can be found here:
|
37 |
+
# https://huggingface.co/settings/tokens.
|
38 |
+
login()
|
39 |
+
|
40 |
+
model = timm.create_model(
|
41 |
+
"hf-hub:bioptimus/H-optimus-0", pretrained=True, init_values=1e-5, dynamic_img_size=False
|
42 |
+
)
|
43 |
+
model.to("cuda")
|
44 |
+
model.eval()
|
45 |
+
|
46 |
+
transform = transforms.Compose([
|
47 |
+
transforms.ToTensor(),
|
48 |
+
transforms.Normalize(
|
49 |
+
mean=(0.707223, 0.578729, 0.703617),
|
50 |
+
std=(0.211883, 0.230117, 0.177517)
|
51 |
+
),
|
52 |
+
])
|
53 |
+
|
54 |
+
input = torch.rand(3, 224, 224)
|
55 |
+
input = transforms.ToPILImage()(input)
|
56 |
+
|
57 |
+
# We recommend using mixed precision for faster inference.
|
58 |
+
with torch.autocast(device_type="cuda", dtype=torch.float16):
|
59 |
+
with torch.inference_mode():
|
60 |
+
features = model(transform(input).unsqueeze(0).to("cuda"))
|
61 |
+
|
62 |
+
assert features.shape == (1, 1536)
|
63 |
+
```
|
64 |
+
|
65 |
+
## BibTeX entry and citation info.
|
66 |
+
|
67 |
+
If you find this repository useful, please consider citing our work:
|
68 |
+
```
|
69 |
+
@software{hoptimus0,
|
70 |
+
author = {Saillard, Charlie and Jenatton, Rodolphe and Llinares-López, Felipe and Mariet, Zelda and Cahané, David and Durand, Eric and Vert, Jean-Philippe},
|
71 |
+
title = {H-optimus-0},
|
72 |
+
url = {https://github.com/bioptimus/releases/tree/main/models/h-optimus/v0},
|
73 |
+
year = {2024},
|
74 |
+
}
|
75 |
+
```
|
logo.png
ADDED