bert-base-uncased-sst-2-64-13-smoothed

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6180
• Accuracy: 0.8281

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 75
• label_smoothing_factor: 0.45

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	4	0.6942	0.5234
No log	2.0	8	0.6933	0.5547
0.6914	3.0	12	0.6922	0.5547
0.6914	4.0	16	0.6906	0.5469
0.692	5.0	20	0.6887	0.5781
0.692	6.0	24	0.6867	0.6328
0.692	7.0	28	0.6841	0.6406
0.6784	8.0	32	0.6813	0.6406
0.6784	9.0	36	0.6780	0.6484
0.6647	10.0	40	0.6731	0.6484
0.6647	11.0	44	0.6674	0.7188
0.6647	12.0	48	0.6615	0.7344
0.6336	13.0	52	0.6535	0.7109
0.6336	14.0	56	0.6475	0.7422
0.5851	15.0	60	0.6394	0.7812
0.5851	16.0	64	0.6344	0.7656
0.5851	17.0	68	0.6371	0.7578
0.5508	18.0	72	0.6343	0.7656
0.5508	19.0	76	0.6337	0.7734
0.5412	20.0	80	0.6377	0.7578
0.5412	21.0	84	0.6349	0.7812
0.5412	22.0	88	0.6269	0.7891
0.5393	23.0	92	0.6227	0.8281
0.5393	24.0	96	0.6209	0.8203
0.5375	25.0	100	0.6198	0.8203
0.5375	26.0	104	0.6194	0.8359
0.5375	27.0	108	0.6205	0.8203
0.5377	28.0	112	0.6223	0.8125
0.5377	29.0	116	0.6236	0.8047
0.537	30.0	120	0.6235	0.8047
0.537	31.0	124	0.6250	0.7891
0.537	32.0	128	0.6243	0.7969
0.5375	33.0	132	0.6215	0.8281
0.5375	34.0	136	0.6206	0.8203
0.5368	35.0	140	0.6201	0.8281
0.5368	36.0	144	0.6200	0.8359
0.5368	37.0	148	0.6198	0.8359
0.5363	38.0	152	0.6199	0.8281
0.5363	39.0	156	0.6202	0.8203
0.5365	40.0	160	0.6198	0.8281
0.5365	41.0	164	0.6192	0.8359
0.5365	42.0	168	0.6192	0.8438
0.5369	43.0	172	0.6188	0.8281
0.5369	44.0	176	0.6192	0.8203
0.5363	45.0	180	0.6196	0.8203
0.5363	46.0	184	0.6186	0.8203
0.5363	47.0	188	0.6182	0.8359
0.5362	48.0	192	0.6181	0.8359
0.5362	49.0	196	0.6182	0.8203
0.5365	50.0	200	0.6182	0.8203
0.5365	51.0	204	0.6179	0.8359
0.5365	52.0	208	0.6177	0.8359
0.5359	53.0	212	0.6175	0.8359
0.5359	54.0	216	0.6174	0.8281
0.5366	55.0	220	0.6174	0.8359
0.5366	56.0	224	0.6175	0.8359
0.5366	57.0	228	0.6176	0.8359
0.5362	58.0	232	0.6177	0.8359
0.5362	59.0	236	0.6179	0.8359
0.5359	60.0	240	0.6179	0.8359
0.5359	61.0	244	0.6178	0.8359
0.5359	62.0	248	0.6177	0.8359
0.5358	63.0	252	0.6178	0.8359
0.5358	64.0	256	0.6179	0.8359
0.5361	65.0	260	0.6181	0.8281
0.5361	66.0	264	0.6182	0.8281
0.5361	67.0	268	0.6181	0.8281
0.5358	68.0	272	0.6181	0.8281
0.5358	69.0	276	0.6181	0.8281
0.5362	70.0	280	0.6180	0.8281
0.5362	71.0	284	0.6180	0.8359
0.5362	72.0	288	0.6180	0.8359
0.5356	73.0	292	0.6180	0.8359
0.5356	74.0	296	0.6180	0.8281
0.5358	75.0	300	0.6180	0.8281

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3