BEiT-DMAE-DA-REVAL-80-35e-05

This model is a fine-tuned version of microsoft/beit-base-patch16-224 on the imagefolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.4981
• Accuracy: 0.8913

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 3.5e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 4
• total_train_batch_size: 64
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 80

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
1.528	0.96	11	1.2944	0.4565
1.3955	2.0	23	1.3459	0.5
1.3036	2.96	34	1.2487	0.3696
1.1708	4.0	46	1.0742	0.5217
1.0545	4.96	57	0.9917	0.5435
0.9352	6.0	69	0.9326	0.5435
0.7648	6.96	80	0.9982	0.4348
0.7161	8.0	92	0.8967	0.5870
0.6157	8.96	103	0.9164	0.6087
0.6128	10.0	115	0.9261	0.6522
0.4791	10.96	126	0.7729	0.7391
0.4494	12.0	138	0.6894	0.8043
0.4213	12.96	149	0.7138	0.8043
0.3341	14.0	161	0.9143	0.7174
0.3114	14.96	172	0.6721	0.7609
0.322	16.0	184	0.7821	0.7609
0.2975	16.96	195	0.7998	0.7826
0.2694	18.0	207	0.6825	0.8043
0.2104	18.96	218	0.6983	0.8043
0.2309	20.0	230	0.7812	0.7174
0.1867	20.96	241	0.7511	0.7609
0.17	22.0	253	0.9706	0.7391
0.1915	22.96	264	0.7702	0.8043
0.2309	24.0	276	0.7152	0.8043
0.1813	24.96	287	0.7917	0.7826
0.1399	26.0	299	0.7918	0.7391
0.1481	26.96	310	0.7052	0.8261
0.2046	28.0	322	0.5693	0.8478
0.1912	28.96	333	0.6074	0.8261
0.1467	30.0	345	0.9355	0.7609
0.1263	30.96	356	0.6719	0.8261
0.1357	32.0	368	0.7006	0.7826
0.1117	32.96	379	0.7690	0.7609
0.1294	34.0	391	0.9282	0.7609
0.139	34.96	402	0.7608	0.7609
0.131	36.0	414	0.8221	0.8043
0.1237	36.96	425	0.9204	0.7391
0.1148	38.0	437	0.5724	0.8261
0.1131	38.96	448	0.9197	0.7609
0.1171	40.0	460	0.8922	0.7174
0.0833	40.96	471	0.6172	0.8043
0.1026	42.0	483	0.6637	0.7609
0.09	42.96	494	0.8515	0.7826
0.0894	44.0	506	0.5513	0.8696
0.0842	44.96	517	0.8008	0.8261
0.0824	46.0	529	0.6873	0.8696
0.1004	46.96	540	1.0546	0.7826
0.0915	48.0	552	0.8237	0.7609
0.0642	48.96	563	0.4981	0.8913
0.0872	50.0	575	0.7128	0.8696
0.0755	50.96	586	0.7991	0.8043
0.0773	52.0	598	0.8565	0.7826
0.0853	52.96	609	0.7463	0.8478
0.0717	54.0	621	0.7527	0.8043
0.0919	54.96	632	0.6984	0.8478
0.0913	56.0	644	0.7035	0.8043
0.0672	56.96	655	0.6481	0.8696
0.0691	58.0	667	0.7666	0.8696
0.0733	58.96	678	0.7665	0.8478
0.0982	60.0	690	0.8573	0.8478
0.0791	60.96	701	0.7859	0.8696
0.0559	62.0	713	0.7715	0.8261
0.0725	62.96	724	0.8548	0.8478
0.065	64.0	736	0.8533	0.8478
0.0844	64.96	747	0.8175	0.8478
0.0664	66.0	759	0.7743	0.8261
0.0696	66.96	770	0.8106	0.8696
0.0567	68.0	782	0.7904	0.8478
0.083	68.96	793	0.8550	0.8043
0.0385	70.0	805	0.8915	0.8261
0.0714	70.96	816	0.8631	0.8261
0.0444	72.0	828	0.8231	0.8478
0.0496	72.96	839	0.8199	0.8478
0.0887	74.0	851	0.8304	0.8478
0.0475	74.96	862	0.8529	0.8261
0.0639	76.0	874	0.8580	0.8261
0.0704	76.52	880	0.8589	0.8261

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0