bert-base-uncased-sst-2-32-13-smoothed

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6688
• Accuracy: 0.7188

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 75
• label_smoothing_factor: 0.45

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.7003	0.4844
No log	2.0	4	0.6998	0.5
No log	3.0	6	0.6991	0.5
No log	4.0	8	0.6980	0.5
0.6958	5.0	10	0.6967	0.4844
0.6958	6.0	12	0.6953	0.4844
0.6958	7.0	14	0.6940	0.5312
0.6958	8.0	16	0.6928	0.5312
0.6958	9.0	18	0.6915	0.5625
0.6939	10.0	20	0.6904	0.625
0.6939	11.0	22	0.6894	0.625
0.6939	12.0	24	0.6885	0.6094
0.6939	13.0	26	0.6881	0.5938
0.6939	14.0	28	0.6874	0.5938
0.6707	15.0	30	0.6870	0.6094
0.6707	16.0	32	0.6869	0.6562
0.6707	17.0	34	0.6871	0.6562
0.6707	18.0	36	0.6867	0.625
0.6707	19.0	38	0.6858	0.625
0.6407	20.0	40	0.6848	0.6094
0.6407	21.0	42	0.6840	0.5781
0.6407	22.0	44	0.6850	0.5781
0.6407	23.0	46	0.6852	0.5625
0.6407	24.0	48	0.6845	0.5625
0.6016	25.0	50	0.6853	0.5938
0.6016	26.0	52	0.6859	0.5938
0.6016	27.0	54	0.6851	0.5781
0.6016	28.0	56	0.6812	0.5938
0.6016	29.0	58	0.6793	0.6094
0.5645	30.0	60	0.6786	0.6094
0.5645	31.0	62	0.6774	0.625
0.5645	32.0	64	0.6763	0.6719
0.5645	33.0	66	0.6754	0.6719
0.5645	34.0	68	0.6751	0.6562
0.5434	35.0	70	0.6748	0.6719
0.5434	36.0	72	0.6741	0.7031
0.5434	37.0	74	0.6745	0.6875
0.5434	38.0	76	0.6752	0.7031
0.5434	39.0	78	0.6756	0.6719
0.5383	40.0	80	0.6755	0.6719
0.5383	41.0	82	0.6760	0.6875
0.5383	42.0	84	0.6778	0.6406
0.5383	43.0	86	0.6802	0.6406
0.5383	44.0	88	0.6823	0.6406
0.5379	45.0	90	0.6827	0.5938
0.5379	46.0	92	0.6815	0.6094
0.5379	47.0	94	0.6804	0.625
0.5379	48.0	96	0.6790	0.6406
0.5379	49.0	98	0.6756	0.6562
0.5371	50.0	100	0.6739	0.6562
0.5371	51.0	102	0.6726	0.6406
0.5371	52.0	104	0.6718	0.6719
0.5371	53.0	106	0.6710	0.7031
0.5371	54.0	108	0.6705	0.6875
0.5365	55.0	110	0.6701	0.6875
0.5365	56.0	112	0.6698	0.6875
0.5365	57.0	114	0.6696	0.7031
0.5365	58.0	116	0.6694	0.6875
0.5365	59.0	118	0.6693	0.6875
0.5369	60.0	120	0.6690	0.7031
0.5369	61.0	122	0.6687	0.7188
0.5369	62.0	124	0.6687	0.7344
0.5369	63.0	126	0.6688	0.7031
0.5369	64.0	128	0.6688	0.6875
0.5365	65.0	130	0.6688	0.7031
0.5365	66.0	132	0.6688	0.7188
0.5365	67.0	134	0.6688	0.7188
0.5365	68.0	136	0.6688	0.7188
0.5365	69.0	138	0.6688	0.7344
0.5364	70.0	140	0.6687	0.7344
0.5364	71.0	142	0.6687	0.7188
0.5364	72.0	144	0.6687	0.7188
0.5364	73.0	146	0.6687	0.7188
0.5364	74.0	148	0.6688	0.7188
0.5366	75.0	150	0.6688	0.7188

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3