swiftformer-xs-ve-U13-b-80e

This model is a fine-tuned version of MBZUAI/swiftformer-xs on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.6618
• Accuracy: 0.8478

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0003
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 128
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.15
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.92	6	1.3859	0.2391
1.3857	2.0	13	1.3834	0.3261
1.3857	2.92	19	1.3789	0.1957
1.3767	4.0	26	1.3666	0.1739
1.3227	4.92	32	1.3565	0.1522
1.3227	6.0	39	1.3887	0.1087
1.1987	6.92	45	1.3719	0.2174
1.1071	8.0	52	1.3271	0.3043
1.1071	8.92	58	1.3562	0.2609
0.9926	10.0	65	1.2306	0.4130
0.8721	10.92	71	1.1953	0.4565
0.8721	12.0	78	1.0754	0.5652
0.7746	12.92	84	0.9931	0.6739
0.6859	14.0	91	0.9979	0.6739
0.6859	14.92	97	0.8964	0.6957
0.5777	16.0	104	0.9186	0.6522
0.5136	16.92	110	0.7950	0.7609
0.5136	18.0	117	0.7794	0.7391
0.5019	18.92	123	0.8645	0.7174
0.3879	20.0	130	0.8773	0.6957
0.3879	20.92	136	0.7304	0.7609
0.3532	22.0	143	0.6918	0.7609
0.3532	22.92	149	0.7882	0.7609
0.3288	24.0	156	0.7132	0.7609
0.2573	24.92	162	0.6645	0.8043
0.2573	26.0	169	0.6618	0.8478
0.239	26.92	175	0.6780	0.8043
0.2018	28.0	182	0.8138	0.6957
0.2018	28.92	188	0.8797	0.6957
0.1961	30.0	195	0.8602	0.7174
0.214	30.92	201	0.8188	0.7391
0.214	32.0	208	0.6956	0.7609
0.1596	32.92	214	0.7981	0.7391
0.172	34.0	221	0.6845	0.7609
0.172	34.92	227	0.9340	0.7174
0.1852	36.0	234	0.9548	0.6522
0.1492	36.92	240	0.7747	0.7609
0.1492	38.0	247	0.9907	0.6304
0.1735	38.92	253	0.8040	0.7174
0.1405	40.0	260	0.6946	0.7609
0.1405	40.92	266	0.7019	0.7609
0.1269	42.0	273	0.8246	0.7174
0.1269	42.92	279	0.9238	0.6739
0.1237	44.0	286	0.9354	0.6957
0.1201	44.92	292	0.7543	0.7391
0.1201	46.0	299	0.7151	0.7174
0.1134	46.92	305	0.7284	0.7174
0.1141	48.0	312	0.7791	0.7609
0.1141	48.92	318	0.7824	0.7391
0.1253	50.0	325	0.7319	0.7609
0.0968	50.92	331	0.7151	0.7609
0.0968	52.0	338	0.7662	0.7609
0.0996	52.92	344	0.8086	0.7826
0.0844	54.0	351	0.8921	0.7609
0.0844	54.92	357	0.8782	0.7609
0.1141	56.0	364	0.7864	0.7391
0.1263	56.92	370	0.7125	0.7609
0.1263	58.0	377	0.6758	0.7609
0.0966	58.92	383	0.7243	0.7609
0.0771	60.0	390	0.7090	0.7609
0.0771	60.92	396	0.7157	0.7609
0.0497	62.0	403	0.7549	0.7609
0.0497	62.92	409	0.7806	0.7609
0.0848	64.0	416	0.7902	0.7391
0.0477	64.92	422	0.7684	0.7391
0.0477	66.0	429	0.8038	0.6957
0.0823	66.92	435	0.7503	0.6957
0.0726	68.0	442	0.7634	0.7609
0.0726	68.92	448	0.7860	0.7826
0.0799	70.0	455	0.7630	0.7609
0.067	70.92	461	0.8094	0.7391
0.067	72.0	468	0.7511	0.7391
0.0893	72.92	474	0.7738	0.7391
0.0738	73.85	480	0.7971	0.7391

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0