fine-tuned-bert-CBT

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.9580

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 3

Training results

Training Loss	Epoch	Step	Validation Loss
2.6996	0.0278	10	2.5932
2.5659	0.0556	20	2.4949
2.5146	0.0833	30	2.3969
2.4268	0.1111	40	2.3803
2.3637	0.1389	50	2.2703
2.2599	0.1667	60	2.2429
2.2562	0.1944	70	2.1078
2.0763	0.2222	80	2.0618
2.0285	0.25	90	2.0001
2.0964	0.2778	100	1.9234
1.8736	0.3056	110	1.8622
1.9762	0.3333	120	1.8228
1.8524	0.3611	130	1.7528
1.7582	0.3889	140	1.6850
1.666	0.4167	150	1.6393
1.6081	0.4444	160	1.5788
1.5763	0.4722	170	1.5264
1.5569	0.5	180	1.5205
1.5338	0.5278	190	1.4746
1.3994	0.5556	200	1.4834
1.4633	0.5833	210	1.4261
1.5346	0.6111	220	1.3530
1.4419	0.6389	230	1.3255
1.2908	0.6667	240	1.3178
1.4339	0.6944	250	1.3066
1.2745	0.7222	260	1.2494
1.3522	0.75	270	1.2489
1.2186	0.7778	280	1.2322
1.2945	0.8056	290	1.1780
1.3717	0.8333	300	1.2243
1.2835	0.8611	310	1.2065
1.2402	0.8889	320	1.1678
1.0937	0.9167	330	1.1325
1.2552	0.9444	340	1.1202
1.1272	0.9722	350	1.1032
1.1746	1.0	360	1.0993
1.1314	1.0278	370	1.0908
1.1143	1.0556	380	1.0849
0.9685	1.0833	390	1.0890
0.8647	1.1111	400	1.1004
0.8987	1.1389	410	1.0898
0.8619	1.1667	420	1.1206
0.8827	1.1944	430	1.0759
0.9164	1.2222	440	1.0875
1.1161	1.25	450	1.0763
0.8902	1.2778	460	1.0525
0.9228	1.3056	470	1.0476
0.789	1.3333	480	1.0377
0.8984	1.3611	490	1.0522
1.065	1.3889	500	1.0215
0.7046	1.4167	510	1.0186
0.9612	1.4444	520	1.0143
0.7705	1.4722	530	1.0207
0.7768	1.5	540	1.0060
1.0041	1.5278	550	1.0296
1.0711	1.5556	560	1.0030
0.6894	1.5833	570	1.0044
1.1434	1.6111	580	0.9944
0.9194	1.6389	590	0.9822
0.8456	1.6667	600	0.9993
0.8691	1.6944	610	0.9917
0.8099	1.7222	620	0.9836
0.9797	1.75	630	0.9814
0.9307	1.7778	640	0.9790
0.6684	1.8056	650	0.9704
0.8533	1.8333	660	0.9869
0.8324	1.8611	670	0.9658
0.7872	1.8889	680	0.9683
0.6827	1.9167	690	0.9857
1.0274	1.9444	700	0.9893
0.8892	1.9722	710	0.9738
0.7447	2.0	720	0.9732
0.7072	2.0278	730	0.9748
0.7205	2.0556	740	0.9637
0.6102	2.0833	750	0.9647
0.7017	2.1111	760	0.9520
0.7374	2.1389	770	0.9495
0.6856	2.1667	780	0.9455
0.4585	2.1944	790	0.9481
0.5872	2.2222	800	0.9494
0.7054	2.25	810	0.9457
0.7	2.2778	820	0.9557
0.5781	2.3056	830	0.9579
0.6319	2.3333	840	0.9677
0.5814	2.3611	850	0.9705
0.5985	2.3889	860	0.9614
0.729	2.4167	870	0.9536
0.6084	2.4444	880	0.9499
0.755	2.4722	890	0.9594
0.4991	2.5	900	0.9782
0.5469	2.5278	910	0.9928
0.6299	2.5556	920	0.9875
0.5911	2.5833	930	0.9720
0.4386	2.6111	940	0.9701
0.5477	2.6389	950	0.9695
0.6532	2.6667	960	0.9723
0.6322	2.6944	970	0.9710
0.4968	2.7222	980	0.9701
0.5498	2.75	990	0.9708
0.746	2.7778	1000	0.9697
0.5654	2.8056	1010	0.9698
0.5468	2.8333	1020	0.9655
0.6086	2.8611	1030	0.9643
0.6928	2.8889	1040	0.9612
0.4775	2.9167	1050	0.9585
0.5934	2.9444	1060	0.9579
0.6645	2.9722	1070	0.9580
0.5721	3.0	1080	0.9580

Framework versions

• Transformers 4.47.0
• Pytorch 2.5.1+cu121
• Datasets 3.1.0
• Tokenizers 0.21.0