SWv2-DMAE-H-6-ps-clean-fix-U-40-Cross-6

This model is a fine-tuned version of microsoft/swinv2-tiny-patch4-window8-256 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.4696
• Accuracy: 0.8269

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 4e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 40

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.3847	0.96	11	1.3617	0.3462
1.3874	2.0	23	1.3383	0.3462
1.3594	2.96	34	1.2585	0.5577
1.2304	4.0	46	1.1284	0.4231
1.0706	4.96	57	0.9184	0.5769
0.9507	6.0	69	0.8399	0.6346
0.7358	6.96	80	0.8507	0.5
0.6664	8.0	92	0.6480	0.75
0.5935	8.96	103	0.5550	0.7692
0.565	10.0	115	0.6149	0.6731
0.4619	10.96	126	0.5001	0.8077
0.4954	12.0	138	0.4696	0.8269
0.4313	12.96	149	0.5613	0.7308
0.4299	14.0	161	0.5546	0.7885
0.4213	14.96	172	0.5084	0.7692
0.3764	16.0	184	0.6273	0.7115
0.3745	16.96	195	0.8686	0.6346
0.3935	18.0	207	0.5955	0.7692
0.3005	18.96	218	0.5678	0.8077
0.2867	20.0	230	0.5452	0.8077
0.2801	20.96	241	0.5286	0.8077
0.3213	22.0	253	0.6355	0.7692
0.273	22.96	264	0.6812	0.75
0.2667	24.0	276	0.5957	0.7692
0.2859	24.96	287	0.6318	0.75
0.2629	26.0	299	0.5229	0.7885
0.2452	26.96	310	0.7775	0.7308
0.2177	28.0	322	0.6332	0.7692
0.2421	28.96	333	0.7003	0.75
0.2114	30.0	345	0.5935	0.7885
0.204	30.96	356	0.6657	0.7692
0.2027	32.0	368	0.6497	0.7692
0.2348	32.96	379	0.6382	0.7692
0.2081	34.0	391	0.5660	0.8269
0.1688	34.96	402	0.6342	0.7692
0.1752	36.0	414	0.6113	0.7692
0.1845	36.96	425	0.5766	0.7885
0.1938	38.0	437	0.5906	0.7692
0.1804	38.26	440	0.5907	0.7885

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0