bert-base-uncased-sst-2-16-13

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.5049
• Accuracy: 0.8125

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	1	0.7051	0.5312
No log	2.0	2	0.7048	0.5312
No log	3.0	3	0.7041	0.5312
No log	4.0	4	0.7029	0.5312
No log	5.0	5	0.7014	0.5625
No log	6.0	6	0.6996	0.5625
No log	7.0	7	0.6975	0.5625
No log	8.0	8	0.6951	0.5625
No log	9.0	9	0.6922	0.5625
0.7427	10.0	10	0.6891	0.5625
0.7427	11.0	11	0.6857	0.5625
0.7427	12.0	12	0.6820	0.5625
0.7427	13.0	13	0.6781	0.5625
0.7427	14.0	14	0.6740	0.5625
0.7427	15.0	15	0.6698	0.5312
0.7427	16.0	16	0.6658	0.5938
0.7427	17.0	17	0.6621	0.5625
0.7427	18.0	18	0.6585	0.5625
0.7427	19.0	19	0.6550	0.6562
0.6663	20.0	20	0.6516	0.6562
0.6663	21.0	21	0.6489	0.7188
0.6663	22.0	22	0.6471	0.7188
0.6663	23.0	23	0.6465	0.75
0.6663	24.0	24	0.6465	0.75
0.6663	25.0	25	0.6461	0.7188
0.6663	26.0	26	0.6450	0.7188
0.6663	27.0	27	0.6427	0.6875
0.6663	28.0	28	0.6394	0.6875
0.6663	29.0	29	0.6358	0.7188
0.5394	30.0	30	0.6319	0.75
0.5394	31.0	31	0.6279	0.75
0.5394	32.0	32	0.6244	0.7812
0.5394	33.0	33	0.6207	0.7812
0.5394	34.0	34	0.6169	0.7812
0.5394	35.0	35	0.6131	0.7812
0.5394	36.0	36	0.6096	0.7812
0.5394	37.0	37	0.6057	0.7812
0.5394	38.0	38	0.6028	0.7812
0.5394	39.0	39	0.6010	0.75
0.3922	40.0	40	0.5975	0.75
0.3922	41.0	41	0.5941	0.75
0.3922	42.0	42	0.5902	0.75
0.3922	43.0	43	0.5854	0.75
0.3922	44.0	44	0.5800	0.75
0.3922	45.0	45	0.5768	0.7188
0.3922	46.0	46	0.5747	0.7188
0.3922	47.0	47	0.5743	0.7188
0.3922	48.0	48	0.5765	0.7188
0.3922	49.0	49	0.5779	0.6875
0.2715	50.0	50	0.5813	0.7188
0.2715	51.0	51	0.5839	0.6875
0.2715	52.0	52	0.5857	0.7188
0.2715	53.0	53	0.5916	0.7188
0.2715	54.0	54	0.5986	0.75
0.2715	55.0	55	0.6033	0.75
0.2715	56.0	56	0.6016	0.75
0.2715	57.0	57	0.6004	0.75
0.2715	58.0	58	0.5928	0.75
0.2715	59.0	59	0.5860	0.7812
0.174	60.0	60	0.5795	0.75
0.174	61.0	61	0.5707	0.75
0.174	62.0	62	0.5629	0.7188
0.174	63.0	63	0.5578	0.6875
0.174	64.0	64	0.5535	0.7188
0.174	65.0	65	0.5498	0.7188
0.174	66.0	66	0.5468	0.7188
0.174	67.0	67	0.5436	0.7188
0.174	68.0	68	0.5404	0.7188
0.174	69.0	69	0.5373	0.7188
0.1107	70.0	70	0.5353	0.7188
0.1107	71.0	71	0.5327	0.7188
0.1107	72.0	72	0.5292	0.7188
0.1107	73.0	73	0.5243	0.75
0.1107	74.0	74	0.5187	0.75
0.1107	75.0	75	0.5131	0.75
0.1107	76.0	76	0.5081	0.75
0.1107	77.0	77	0.5036	0.75
0.1107	78.0	78	0.5005	0.75
0.1107	79.0	79	0.4982	0.7812
0.0742	80.0	80	0.4970	0.8438
0.0742	81.0	81	0.4958	0.8438
0.0742	82.0	82	0.4939	0.8438
0.0742	83.0	83	0.4908	0.8438
0.0742	84.0	84	0.4873	0.8125
0.0742	85.0	85	0.4840	0.8125
0.0742	86.0	86	0.4814	0.8125
0.0742	87.0	87	0.4790	0.8125
0.0742	88.0	88	0.4769	0.8125
0.0742	89.0	89	0.4750	0.8125
0.0494	90.0	90	0.4742	0.8125
0.0494	91.0	91	0.4737	0.8125
0.0494	92.0	92	0.4731	0.8125
0.0494	93.0	93	0.4726	0.8125
0.0494	94.0	94	0.4722	0.8125
0.0494	95.0	95	0.4720	0.8125
0.0494	96.0	96	0.4720	0.8125
0.0494	97.0	97	0.4715	0.8125
0.0494	98.0	98	0.4712	0.8125
0.0494	99.0	99	0.4710	0.8125
0.0331	100.0	100	0.4709	0.8125
0.0331	101.0	101	0.4711	0.8125
0.0331	102.0	102	0.4715	0.8125
0.0331	103.0	103	0.4725	0.8125
0.0331	104.0	104	0.4734	0.8125
0.0331	105.0	105	0.4742	0.8125
0.0331	106.0	106	0.4752	0.8125
0.0331	107.0	107	0.4761	0.8125
0.0331	108.0	108	0.4770	0.8125
0.0331	109.0	109	0.4780	0.8125
0.0246	110.0	110	0.4789	0.8125
0.0246	111.0	111	0.4804	0.8125
0.0246	112.0	112	0.4817	0.8125
0.0246	113.0	113	0.4829	0.8125
0.0246	114.0	114	0.4842	0.8125
0.0246	115.0	115	0.4851	0.8125
0.0246	116.0	116	0.4863	0.8125
0.0246	117.0	117	0.4880	0.8125
0.0246	118.0	118	0.4897	0.8125
0.0246	119.0	119	0.4913	0.8125
0.0191	120.0	120	0.4930	0.8125
0.0191	121.0	121	0.4945	0.8125
0.0191	122.0	122	0.4959	0.8125
0.0191	123.0	123	0.4971	0.8125
0.0191	124.0	124	0.4984	0.8125
0.0191	125.0	125	0.4995	0.8125
0.0191	126.0	126	0.5004	0.8125
0.0191	127.0	127	0.5014	0.8125
0.0191	128.0	128	0.5021	0.8125
0.0191	129.0	129	0.5027	0.8125
0.0163	130.0	130	0.5031	0.8125
0.0163	131.0	131	0.5031	0.8125
0.0163	132.0	132	0.5034	0.8125
0.0163	133.0	133	0.5035	0.8125
0.0163	134.0	134	0.5036	0.8125
0.0163	135.0	135	0.5036	0.8125
0.0163	136.0	136	0.5037	0.8125
0.0163	137.0	137	0.5038	0.8125
0.0163	138.0	138	0.5040	0.8125
0.0163	139.0	139	0.5043	0.8125
0.0147	140.0	140	0.5044	0.8125
0.0147	141.0	141	0.5046	0.8125
0.0147	142.0	142	0.5047	0.8125
0.0147	143.0	143	0.5049	0.8125
0.0147	144.0	144	0.5049	0.8125
0.0147	145.0	145	0.5049	0.8125
0.0147	146.0	146	0.5049	0.8125
0.0147	147.0	147	0.5049	0.8125
0.0147	148.0	148	0.5049	0.8125
0.0147	149.0	149	0.5049	0.8125
0.0138	150.0	150	0.5049	0.8125

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3