bert-base-uncased-sst-2-32-42

This model is a fine-tuned version of bert-base-uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 1.1901
• Accuracy: 0.8438

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 32
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 150

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy
No log	1.0	2	1.0945	0.8438
No log	2.0	4	1.0962	0.8438
No log	3.0	6	1.0975	0.8438
No log	4.0	8	1.0969	0.8438
0.3232	5.0	10	1.0974	0.8438
0.3232	6.0	12	1.0999	0.8438
0.3232	7.0	14	1.1032	0.8438
0.3232	8.0	16	1.1075	0.8438
0.3232	9.0	18	1.1108	0.8438
0.2269	10.0	20	1.1140	0.8438
0.2269	11.0	22	1.1163	0.8438
0.2269	12.0	24	1.1175	0.8438
0.2269	13.0	26	1.1187	0.8438
0.2269	14.0	28	1.1226	0.8438
0.2217	15.0	30	1.1271	0.8438
0.2217	16.0	32	1.1355	0.8281
0.2217	17.0	34	1.1405	0.8281
0.2217	18.0	36	1.1476	0.8281
0.2217	19.0	38	1.1562	0.8281
0.1358	20.0	40	1.1569	0.8281
0.1358	21.0	42	1.1629	0.8281
0.1358	22.0	44	1.1726	0.8281
0.1358	23.0	46	1.1825	0.8281
0.1358	24.0	48	1.1896	0.8281
0.0945	25.0	50	1.1960	0.8281
0.0945	26.0	52	1.1980	0.8281
0.0945	27.0	54	1.2014	0.8281
0.0945	28.0	56	1.2004	0.8281
0.0945	29.0	58	1.1992	0.8281
0.0102	30.0	60	1.2000	0.8281
0.0102	31.0	62	1.1996	0.8281
0.0102	32.0	64	1.1994	0.8281
0.0102	33.0	66	1.1984	0.8281
0.0102	34.0	68	1.1975	0.8281
0.0004	35.0	70	1.1966	0.8281
0.0004	36.0	72	1.1971	0.8281
0.0004	37.0	74	1.1918	0.8281
0.0004	38.0	76	1.1864	0.8281
0.0004	39.0	78	1.1858	0.8281
0.0028	40.0	80	1.1883	0.8281
0.0028	41.0	82	1.1933	0.8281
0.0028	42.0	84	1.2003	0.8281
0.0028	43.0	86	1.2063	0.8281
0.0028	44.0	88	1.2014	0.8281
0.0092	45.0	90	1.1965	0.8281
0.0092	46.0	92	1.1912	0.8281
0.0092	47.0	94	1.1926	0.8281
0.0092	48.0	96	1.1938	0.8281
0.0092	49.0	98	1.1984	0.8281
0.0006	50.0	100	1.1990	0.8281
0.0006	51.0	102	1.2001	0.8281
0.0006	52.0	104	1.2040	0.8281
0.0006	53.0	106	1.2078	0.8281
0.0006	54.0	108	1.2113	0.8281
0.0003	55.0	110	1.2143	0.8281
0.0003	56.0	112	1.2169	0.8281
0.0003	57.0	114	1.2028	0.8281
0.0003	58.0	116	1.1896	0.8281
0.0003	59.0	118	1.1822	0.8281
0.0153	60.0	120	1.1789	0.8281
0.0153	61.0	122	1.1781	0.8281
0.0153	62.0	124	1.1776	0.8281
0.0153	63.0	126	1.1771	0.8281
0.0153	64.0	128	1.1767	0.8281
0.0003	65.0	130	1.1767	0.8281
0.0003	66.0	132	1.1771	0.8281
0.0003	67.0	134	1.1775	0.8281
0.0003	68.0	136	1.1780	0.8281
0.0003	69.0	138	1.1786	0.8281
0.0002	70.0	140	1.1807	0.8281
0.0002	71.0	142	1.1831	0.8281
0.0002	72.0	144	1.1852	0.8281
0.0002	73.0	146	1.1872	0.8281
0.0002	74.0	148	1.1889	0.8281
0.0068	75.0	150	1.2038	0.8281
0.0068	76.0	152	1.2080	0.8281
0.0068	77.0	154	1.2113	0.8281
0.0068	78.0	156	1.2141	0.8281
0.0068	79.0	158	1.2226	0.8281
0.0003	80.0	160	1.2292	0.8281
0.0003	81.0	162	1.2347	0.8281
0.0003	82.0	164	1.2393	0.8281
0.0003	83.0	166	1.2436	0.8281
0.0003	84.0	168	1.2472	0.8281
0.0002	85.0	170	1.2502	0.8281
0.0002	86.0	172	1.2531	0.8281
0.0002	87.0	174	1.2556	0.8281
0.0002	88.0	176	1.2579	0.8281
0.0002	89.0	178	1.2599	0.8281
0.0002	90.0	180	1.2614	0.8281
0.0002	91.0	182	1.2626	0.8281
0.0002	92.0	184	1.2639	0.8281
0.0002	93.0	186	1.2649	0.8281
0.0002	94.0	188	1.2657	0.8281
0.0002	95.0	190	1.2663	0.8281
0.0002	96.0	192	1.2670	0.8281
0.0002	97.0	194	1.2676	0.8281
0.0002	98.0	196	1.2682	0.8281
0.0002	99.0	198	1.2689	0.8281
0.0002	100.0	200	1.2691	0.8281
0.0002	101.0	202	1.2691	0.8281
0.0002	102.0	204	1.2692	0.8281
0.0002	103.0	206	1.2689	0.8281
0.0002	104.0	208	1.2687	0.8281
0.0002	105.0	210	1.2686	0.8281
0.0002	106.0	212	1.2684	0.8281
0.0002	107.0	214	1.2683	0.8281
0.0002	108.0	216	1.2680	0.8281
0.0002	109.0	218	1.2681	0.8281
0.0002	110.0	220	1.2683	0.8281
0.0002	111.0	222	1.2685	0.8281
0.0002	112.0	224	1.2688	0.8281
0.0002	113.0	226	1.2690	0.8281
0.0002	114.0	228	1.2694	0.8281
0.0001	115.0	230	1.2697	0.8281
0.0001	116.0	232	1.2700	0.8281
0.0001	117.0	234	1.2701	0.8281
0.0001	118.0	236	1.2702	0.8281
0.0001	119.0	238	1.2701	0.8281
0.0001	120.0	240	1.2699	0.8281
0.0001	121.0	242	1.2691	0.8281
0.0001	122.0	244	1.2680	0.8281
0.0001	123.0	246	1.2673	0.8281
0.0001	124.0	248	1.2668	0.8281
0.0001	125.0	250	1.2666	0.8281
0.0001	126.0	252	1.2665	0.8281
0.0001	127.0	254	1.2666	0.8281
0.0001	128.0	256	1.2669	0.8281
0.0001	129.0	258	1.2675	0.8281
0.0001	130.0	260	1.2682	0.8281
0.0001	131.0	262	1.2686	0.8281
0.0001	132.0	264	1.2690	0.8281
0.0001	133.0	266	1.2695	0.8281
0.0001	134.0	268	1.2701	0.8281
0.0001	135.0	270	1.2712	0.8281
0.0001	136.0	272	1.2735	0.8281
0.0001	137.0	274	1.2757	0.8281
0.0001	138.0	276	1.2675	0.8281
0.0001	139.0	278	1.2518	0.8281
0.0009	140.0	280	1.2009	0.8438
0.0009	141.0	282	1.1175	0.8594
0.0009	142.0	284	1.1144	0.8594
0.0009	143.0	286	1.1362	0.8594
0.0009	144.0	288	1.1667	0.8438
0.0002	145.0	290	1.1906	0.8438
0.0002	146.0	292	1.2067	0.8438
0.0002	147.0	294	1.2182	0.8438
0.0002	148.0	296	1.2263	0.8438
0.0002	149.0	298	1.2321	0.8438
0.0004	150.0	300	1.1901	0.8438

Framework versions

• Transformers 4.32.0.dev0
• Pytorch 2.0.1+cu118
• Datasets 2.4.0
• Tokenizers 0.13.3