bert-base-uncased-sst-2-32-100

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.4379
• Accuracy: 0.9219

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	0.5385	0.9219
No log	2.0	4	0.5392	0.9219
No log	3.0	6	0.5398	0.9219
No log	4.0	8	0.5410	0.9219
0.733	5.0	10	0.5426	0.9219
0.733	6.0	12	0.5443	0.9062
0.733	7.0	14	0.5461	0.9062
0.733	8.0	16	0.5481	0.9062
0.733	9.0	18	0.5487	0.9062
0.6383	10.0	20	0.5495	0.9062
0.6383	11.0	22	0.5546	0.8906
0.6383	12.0	24	0.5643	0.9062
0.6383	13.0	26	0.5742	0.9062
0.6383	14.0	28	0.5875	0.9062
0.4993	15.0	30	0.5982	0.9062
0.4993	16.0	32	0.6100	0.9062
0.4993	17.0	34	0.6222	0.9062
0.4993	18.0	36	0.6263	0.9062
0.4993	19.0	38	0.6305	0.9062
0.4891	20.0	40	0.6335	0.9062
0.4891	21.0	42	0.6368	0.9062
0.4891	22.0	44	0.6351	0.9062
0.4891	23.0	46	0.6301	0.9062
0.4891	24.0	48	0.6212	0.9062
0.377	25.0	50	0.6100	0.9062
0.377	26.0	52	0.5999	0.9062
0.377	27.0	54	0.5852	0.9062
0.377	28.0	56	0.5737	0.9062
0.377	29.0	58	0.5606	0.9219
0.3369	30.0	60	0.5466	0.9062
0.3369	31.0	62	0.5319	0.9062
0.3369	32.0	64	0.5205	0.9062
0.3369	33.0	66	0.5074	0.9219
0.3369	34.0	68	0.5025	0.9219
0.19	35.0	70	0.4984	0.9219
0.19	36.0	72	0.4934	0.9219
0.19	37.0	74	0.4927	0.9375
0.19	38.0	76	0.4955	0.9375
0.19	39.0	78	0.4968	0.9375
0.0507	40.0	80	0.4956	0.9375
0.0507	41.0	82	0.4882	0.9375
0.0507	42.0	84	0.4784	0.9375
0.0507	43.0	86	0.4710	0.9219
0.0507	44.0	88	0.4650	0.9219
0.0102	45.0	90	0.4578	0.9219
0.0102	46.0	92	0.4540	0.9219
0.0102	47.0	94	0.4566	0.9062
0.0102	48.0	96	0.4682	0.9062
0.0102	49.0	98	0.4831	0.9219
0.0026	50.0	100	0.4922	0.9219
0.0026	51.0	102	0.4985	0.9219
0.0026	52.0	104	0.5029	0.9219
0.0026	53.0	106	0.5062	0.9219
0.0026	54.0	108	0.5087	0.9219
0.001	55.0	110	0.5100	0.9219
0.001	56.0	112	0.5110	0.9219
0.001	57.0	114	0.5112	0.9219
0.001	58.0	116	0.5112	0.9219
0.001	59.0	118	0.5110	0.9219
0.0004	60.0	120	0.5087	0.9219
0.0004	61.0	122	0.5028	0.9219
0.0004	62.0	124	0.4965	0.9219
0.0004	63.0	126	0.4903	0.9219
0.0004	64.0	128	0.4848	0.9219
0.0003	65.0	130	0.4802	0.9219
0.0003	66.0	132	0.4767	0.9219
0.0003	67.0	134	0.4739	0.9219
0.0003	68.0	136	0.4719	0.9219
0.0003	69.0	138	0.4707	0.9219
0.0024	70.0	140	0.4600	0.9219
0.0024	71.0	142	0.4439	0.9219
0.0024	72.0	144	0.4336	0.9062
0.0024	73.0	146	0.4283	0.9062
0.0024	74.0	148	0.4253	0.9219
0.0002	75.0	150	0.4237	0.9219
0.0002	76.0	152	0.4232	0.9375
0.0002	77.0	154	0.4230	0.9375
0.0002	78.0	156	0.4229	0.9375
0.0002	79.0	158	0.4228	0.9375
0.0002	80.0	160	0.4228	0.9375
0.0002	81.0	162	0.4225	0.9375
0.0002	82.0	164	0.4237	0.9062
0.0002	83.0	166	0.4384	0.9219
0.0002	84.0	168	0.4565	0.9219
0.0004	85.0	170	0.4717	0.9219
0.0004	86.0	172	0.4813	0.9219
0.0004	87.0	174	0.4858	0.9219
0.0004	88.0	176	0.4885	0.9219
0.0004	89.0	178	0.4897	0.9219
0.0002	90.0	180	0.4904	0.9219
0.0002	91.0	182	0.4865	0.9219
0.0002	92.0	184	0.4732	0.9219
0.0002	93.0	186	0.4557	0.9219
0.0002	94.0	188	0.4388	0.9219
0.0053	95.0	190	0.4254	0.9219
0.0053	96.0	192	0.4171	0.9219
0.0053	97.0	194	0.4132	0.9375
0.0053	98.0	196	0.4118	0.9375
0.0053	99.0	198	0.4115	0.9219
0.0002	100.0	200	0.4118	0.9219
0.0002	101.0	202	0.4122	0.9219
0.0002	102.0	204	0.4125	0.9219
0.0002	103.0	206	0.4128	0.9219
0.0002	104.0	208	0.4131	0.9219
0.0002	105.0	210	0.4133	0.9219
0.0002	106.0	212	0.4134	0.9219
0.0002	107.0	214	0.4140	0.9219
0.0002	108.0	216	0.4149	0.9219
0.0002	109.0	218	0.4158	0.9219
0.0002	110.0	220	0.4167	0.9219
0.0002	111.0	222	0.4175	0.9219
0.0002	112.0	224	0.4183	0.9375
0.0002	113.0	226	0.4190	0.9375
0.0002	114.0	228	0.4197	0.9375
0.0001	115.0	230	0.4203	0.9375
0.0001	116.0	232	0.4208	0.9375
0.0001	117.0	234	0.4218	0.9219
0.0001	118.0	236	0.4228	0.9219
0.0001	119.0	238	0.4237	0.9219
0.0002	120.0	240	0.4244	0.9219
0.0002	121.0	242	0.4251	0.9219
0.0002	122.0	244	0.4257	0.9219
0.0002	123.0	246	0.4263	0.9219
0.0002	124.0	248	0.4269	0.9219
0.0002	125.0	250	0.4273	0.9219
0.0002	126.0	252	0.4277	0.9219
0.0002	127.0	254	0.4280	0.9219
0.0002	128.0	256	0.4284	0.9219
0.0002	129.0	258	0.4287	0.9219
0.0008	130.0	260	0.4330	0.9219
0.0008	131.0	262	0.4554	0.9219
0.0008	132.0	264	0.4714	0.9219
0.0008	133.0	266	0.4845	0.9375
0.0008	134.0	268	0.5000	0.9219
0.0001	135.0	270	0.5167	0.9219
0.0001	136.0	272	0.5308	0.9062
0.0001	137.0	274	0.5417	0.9062
0.0001	138.0	276	0.5480	0.9062
0.0001	139.0	278	0.5529	0.9062
0.0001	140.0	280	0.5566	0.9062
0.0001	141.0	282	0.5570	0.9062
0.0001	142.0	284	0.5565	0.9062
0.0001	143.0	286	0.5555	0.9062
0.0001	144.0	288	0.5544	0.9062
0.0001	145.0	290	0.5511	0.9062
0.0001	146.0	292	0.5096	0.9219
0.0001	147.0	294	0.4811	0.9375
0.0001	148.0	296	0.4624	0.9219
0.0001	149.0	298	0.4488	0.9219
0.0002	150.0	300	0.4379	0.9219

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3