bert-base-uncased-sst-2-64-13-30

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5262
• Accuracy: 0.7812

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1.5e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 5
• num_epochs: 30

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	4	0.7282	0.5
No log	2.0	8	0.6974	0.5391
0.7077	3.0	12	0.6873	0.5312
0.7077	4.0	16	0.6799	0.5078
0.6218	5.0	20	0.6651	0.5469
0.6218	6.0	24	0.6520	0.5938
0.6218	7.0	28	0.6537	0.5781
0.5204	8.0	32	0.6387	0.625
0.5204	9.0	36	0.6147	0.6562
0.3954	10.0	40	0.5967	0.6719
0.3954	11.0	44	0.5932	0.6719
0.3954	12.0	48	0.6011	0.6641
0.2891	13.0	52	0.5855	0.6797
0.2891	14.0	56	0.5345	0.7266
0.223	15.0	60	0.5222	0.7734
0.223	16.0	64	0.5274	0.7422
0.223	17.0	68	0.5238	0.75
0.1672	18.0	72	0.5203	0.7812
0.1672	19.0	76	0.5166	0.7969
0.1316	20.0	80	0.5132	0.7891
0.1316	21.0	84	0.5118	0.7969
0.1316	22.0	88	0.5129	0.7969
0.1103	23.0	92	0.5170	0.8047
0.1103	24.0	96	0.5216	0.7812
0.09	25.0	100	0.5242	0.7891
0.09	26.0	104	0.5268	0.7734
0.09	27.0	108	0.5272	0.7656
0.0819	28.0	112	0.5266	0.7734
0.0819	29.0	116	0.5263	0.7812
0.0753	30.0	120	0.5262	0.7812

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3