vit-base-patch16-224-in21k-finetuned-inaturalist

This model is a fine-tuned version of google/vit-base-patch16-224-in21k on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.7703
• Accuracy: 0.8542

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 128
• eval_batch_size: 128
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 512
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_ratio: 0.1
• num_epochs: 100

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	0.8421	4	3.1793	0.0347
No log	1.8947	9	3.1647	0.0486
3.1648	2.9474	14	3.1382	0.0944
3.1648	4.0	19	3.0995	0.1556
3.0817	4.8421	23	3.0555	0.2639
3.0817	5.8947	28	2.9849	0.3889
2.9167	6.9474	33	2.8932	0.5139
2.9167	8.0	38	2.7775	0.5972
2.6682	8.8421	42	2.6706	0.6528
2.6682	9.8947	47	2.5233	0.7069
2.3659	10.9474	52	2.3859	0.7375
2.3659	12.0	57	2.2546	0.75
2.079	12.8421	61	2.1531	0.7528
2.079	13.8947	66	2.0372	0.75
1.828	14.9474	71	1.9339	0.7597
1.828	16.0	76	1.8403	0.7694
1.6253	16.8421	80	1.7733	0.7764
1.6253	17.8947	85	1.6914	0.7903
1.4502	18.9474	90	1.6153	0.7875
1.4502	20.0	95	1.5510	0.7986
1.4502	20.8421	99	1.5016	0.8
1.2959	21.8947	104	1.4454	0.8222
1.2959	22.9474	109	1.3912	0.8181
1.1802	24.0	114	1.3390	0.8333
1.1802	24.8421	118	1.2995	0.8333
1.0629	25.8947	123	1.2707	0.8389
1.0629	26.9474	128	1.2335	0.8361
0.9801	28.0	133	1.1975	0.8444
0.9801	28.8421	137	1.1672	0.8389
0.9076	29.8947	142	1.1338	0.8444
0.9076	30.9474	147	1.1137	0.8472
0.8349	32.0	152	1.0855	0.8528
0.8349	32.8421	156	1.0717	0.8542
0.7782	33.8947	161	1.0483	0.8514
0.7782	34.9474	166	1.0352	0.85
0.7208	36.0	171	1.0202	0.8556
0.7208	36.8421	175	0.9994	0.8486
0.6708	37.8947	180	0.9814	0.8556
0.6708	38.9474	185	0.9691	0.8542
0.6303	40.0	190	0.9599	0.8486
0.6303	40.8421	194	0.9422	0.8472
0.6303	41.8947	199	0.9278	0.8486
0.6018	42.9474	204	0.9172	0.8528
0.6018	44.0	209	0.9093	0.8514
0.5622	44.8421	213	0.9030	0.8583
0.5622	45.8947	218	0.8972	0.8625
0.5474	46.9474	223	0.8859	0.8569
0.5474	48.0	228	0.8858	0.8653
0.5254	48.8421	232	0.8779	0.8556
0.5254	49.8947	237	0.8635	0.8569
0.5036	50.9474	242	0.8563	0.8611
0.5036	52.0	247	0.8613	0.8542
0.4855	52.8421	251	0.8546	0.8625
0.4855	53.8947	256	0.8469	0.8597
0.4697	54.9474	261	0.8327	0.8528
0.4697	56.0	266	0.8268	0.8597
0.4482	56.8421	270	0.8188	0.8556
0.4482	57.8947	275	0.8171	0.8653
0.4436	58.9474	280	0.8133	0.8486
0.4436	60.0	285	0.8070	0.8639
0.4436	60.8421	289	0.7986	0.8542
0.4211	61.8947	294	0.7937	0.8597
0.4211	62.9474	299	0.7908	0.8611
0.4228	64.0	304	0.7952	0.8625
0.4228	64.8421	308	0.8010	0.8514
0.4046	65.8947	313	0.7975	0.8472
0.4046	66.9474	318	0.7927	0.8417
0.4048	68.0	323	0.7880	0.8556
0.4048	68.8421	327	0.7860	0.8514
0.3925	69.8947	332	0.7899	0.8403
0.3925	70.9474	337	0.7883	0.8417
0.3936	72.0	342	0.7885	0.8417
0.3936	72.8421	346	0.7874	0.8361
0.3985	73.8947	351	0.7832	0.8417
0.3985	74.9474	356	0.7787	0.8514
0.3849	76.0	361	0.7753	0.8486
0.3849	76.8421	365	0.7746	0.8514
0.3796	77.8947	370	0.7736	0.8542
0.3796	78.9474	375	0.7731	0.8528
0.3717	80.0	380	0.7715	0.8556
0.3717	80.8421	384	0.7709	0.8556
0.3717	81.8947	389	0.7706	0.8569
0.3802	82.9474	394	0.7704	0.8556
0.3802	84.0	399	0.7704	0.8542
0.3782	84.2105	400	0.7703	0.8542

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.1+cu121
• Datasets 2.21.0
• Tokenizers 0.20.1