Push model using huggingface_hub.
Browse files
README.md
CHANGED
@@ -1,138 +1,9 @@
|
|
1 |
---
|
2 |
-
|
3 |
-
|
4 |
-
-
|
5 |
-
metrics:
|
6 |
-
- accuracy
|
7 |
---
|
8 |
-
# VGG-like Kolmogorov-Arnold Convolutional network with Gram polynomials
|
9 |
|
10 |
-
This model
|
11 |
-
|
12 |
-
|
13 |
-
|
14 |
-
The model consists of consecutive 10 Bottleneck Gram ConvKAN Layers with BatchNorm, polynomial degree equal to 5, GlobalAveragePooling and Linear classification head:
|
15 |
-
|
16 |
-
1. KAGN Convolution, 32 filters, 3x3
|
17 |
-
2. Max pooling, 2x2
|
18 |
-
3. KAGN Convolution, 64 filters, 3x3
|
19 |
-
4. Max pooling, 2x2
|
20 |
-
5. KAGN Convolution, 128 filters, 3x3
|
21 |
-
6. KAGN Convolution, 128 filters, 3x3
|
22 |
-
7. Max pooling, 2x2
|
23 |
-
8. KAGN Convolution, 256 filters, 3x3
|
24 |
-
9. KAGN Convolution, 256 filters, 3x3
|
25 |
-
10 Max pooling, 2x2
|
26 |
-
11. KAGN Convolution, 256 filters, 3x3
|
27 |
-
12. KAGN Convolution, 256 filters, 3x3
|
28 |
-
13. Max pooling, 2x2
|
29 |
-
14. KAGN Convolution, 512 filters, 3x3
|
30 |
-
15. KAGN Convolution, 512 filters, 3x3
|
31 |
-
16. Global Average pooling
|
32 |
-
17. Output layer, 1000 nodes.
|
33 |
-
|
34 |
-

|
35 |
-
|
36 |
-
|
37 |
-
## Intended uses & limitations
|
38 |
-
|
39 |
-
You can use the raw model for image classification or use it as pretrained model for further finetuning.
|
40 |
-
|
41 |
-
### How to use
|
42 |
-
|
43 |
-
First, clone the repository:
|
44 |
-
|
45 |
-
```
|
46 |
-
git clone https://github.com/IvanDrokin/torch-conv-kan.git
|
47 |
-
cd torch-conv-kan
|
48 |
-
pip install -r requirements.txt
|
49 |
-
```
|
50 |
-
Then you can initialize the model and load weights.
|
51 |
-
|
52 |
-
```python
|
53 |
-
import torch
|
54 |
-
from models import vggkagn_bn
|
55 |
-
model = vggkagn_bn(3,
|
56 |
-
1000,
|
57 |
-
groups=1,
|
58 |
-
degree=5,
|
59 |
-
dropout=0.05,
|
60 |
-
l1_decay=0,
|
61 |
-
width_scale=2,
|
62 |
-
affine=True,
|
63 |
-
norm_layer=nn.BatchNorm2d,
|
64 |
-
expected_feature_shape=(1, 1),
|
65 |
-
vgg_type='VGG11v4')
|
66 |
-
model.from_pretrained('brivangl/vgg_kagn_bn11_v4')
|
67 |
-
```
|
68 |
-
|
69 |
-
Transforms, used for validation on Imagenet1k:
|
70 |
-
|
71 |
-
```python
|
72 |
-
from torchvision.transforms import v2
|
73 |
-
transforms_val = v2.Compose([
|
74 |
-
v2.ToImage(),
|
75 |
-
v2.Resize(256, antialias=True),
|
76 |
-
v2.CenterCrop(224),
|
77 |
-
v2.ToDtype(torch.float32, scale=True),
|
78 |
-
v2.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
|
79 |
-
])
|
80 |
-
```
|
81 |
-
|
82 |
-
|
83 |
-
|
84 |
-
## Training data
|
85 |
-
This model trained on Imagenet1k dataset (1281167 images in train set)
|
86 |
-
|
87 |
-
## Training procedure
|
88 |
-
|
89 |
-
Model was trained during 200 full epochs with AdamW optimizer, with following parameters:
|
90 |
-
```python
|
91 |
-
{'learning_rate': 0.0005, 'adam_beta1': 0.9, 'adam_beta2': 0.999, 'adam_weight_decay': 5e-06,
|
92 |
-
'adam_epsilon': 1e-08, 'lr_warmup_steps': 7500, 'lr_power': 0.3, 'lr_end': 1e-07, 'set_grads_to_none': False}
|
93 |
-
```
|
94 |
-
And this augmnetations:
|
95 |
-
```python
|
96 |
-
transforms_train = v2.Compose([
|
97 |
-
v2.ToImage(),
|
98 |
-
v2.RandomHorizontalFlip(p=0.5),
|
99 |
-
v2.RandomResizedCrop(224, antialias=True),
|
100 |
-
v2.RandomChoice([v2.AutoAugment(AutoAugmentPolicy.CIFAR10),
|
101 |
-
v2.AutoAugment(AutoAugmentPolicy.IMAGENET)
|
102 |
-
]),
|
103 |
-
v2.ToDtype(torch.float32, scale=True),
|
104 |
-
v2.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225]),
|
105 |
-
])
|
106 |
-
```
|
107 |
-
|
108 |
-
## Evaluation results
|
109 |
-
|
110 |
-
On Imagenet1k Validation:
|
111 |
-
|
112 |
-
| Accuracy, top1 | Accuracy, top5 | AUC (ovo) | AUC (ovr) |
|
113 |
-
|:--------------:|:--------------:|:---------:|:---------:|
|
114 |
-
| 68.50 | 88.46 | 99.61 | 99.61 |
|
115 |
-
|
116 |
-
On Imagenet1k Test:
|
117 |
-
Coming soon
|
118 |
-
|
119 |
-
### BibTeX entry and citation info
|
120 |
-
|
121 |
-
If you use this project in your research or wish to refer to the baseline results, please use the following BibTeX entry.
|
122 |
-
|
123 |
-
```bibtex
|
124 |
-
@misc{torch-conv-kan,
|
125 |
-
author = {Ivan Drokin},
|
126 |
-
title = {Torch Conv KAN},
|
127 |
-
year = {2024},
|
128 |
-
publisher = {GitHub},
|
129 |
-
journal = {GitHub repository},
|
130 |
-
howpublished = {\url{https://github.com/IvanDrokin/torch-conv-kan}}
|
131 |
-
}
|
132 |
-
```
|
133 |
-
|
134 |
-
## References
|
135 |
-
|
136 |
-
- [1] Ziming Liu et al., "KAN: Kolmogorov-Arnold Networks", 2024, arXiv. https://arxiv.org/abs/2404.19756
|
137 |
-
- [2] https://github.com/KindXiaoming/pykan
|
138 |
-
- [3] https://github.com/Khochawongwat/GRAMKAN
|
|
|
1 |
---
|
2 |
+
tags:
|
3 |
+
- pytorch_model_hub_mixin
|
4 |
+
- model_hub_mixin
|
|
|
|
|
5 |
---
|
|
|
6 |
|
7 |
+
This model has been pushed to the Hub using the [PytorchModelHubMixin](https://huggingface.co/docs/huggingface_hub/package_reference/mixins#huggingface_hub.PyTorchModelHubMixin) integration:
|
8 |
+
- Library: [More Information Needed]
|
9 |
+
- Docs: [More Information Needed]
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|