bert-base-uncased_1_16619_token

This model is a fine-tuned version of bert-base-uncased on the bionlp2004 dataset. It achieves the following results on the evaluation set:

• Loss: 0.1660
• Precision: 0.7730
• Recall: 0.8185
• F1: 0.7951
• Accuracy: 0.9478

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Precision	Recall	F1	Accuracy
0.233	1.0	1039	0.1956	0.6860	0.7301	0.7073	0.9331
0.2099	2.0	2078	0.1890	0.7214	0.7508	0.7358	0.9370
0.1991	3.0	3117	0.1862	0.7058	0.7785	0.7404	0.9375
0.19	4.0	4156	0.1736	0.7420	0.7943	0.7673	0.9434
0.1894	5.0	5195	0.1748	0.7319	0.7722	0.7515	0.9417
0.1814	6.0	6234	0.1686	0.7351	0.7952	0.7640	0.9427
0.177	7.0	7273	0.1682	0.7404	0.8086	0.7730	0.9448
0.1756	8.0	8312	0.1740	0.7386	0.7796	0.7585	0.9423
0.1761	9.0	9351	0.1691	0.7442	0.7664	0.7551	0.9430
0.1693	10.0	10390	0.1641	0.7506	0.8113	0.7797	0.9446
0.1697	11.0	11429	0.1669	0.7297	0.7938	0.7604	0.9427
0.1607	12.0	12468	0.1654	0.7593	0.8185	0.7878	0.9454
0.1643	13.0	13507	0.1652	0.7288	0.8035	0.7644	0.9430
0.1618	14.0	14546	0.1592	0.7548	0.7988	0.7762	0.9464
0.1598	15.0	15585	0.1641	0.7575	0.8006	0.7785	0.9454
0.16	16.0	16624	0.1621	0.7440	0.8174	0.7790	0.9456
0.1572	17.0	17663	0.1669	0.7598	0.8015	0.7801	0.9453
0.1528	18.0	18702	0.1680	0.7332	0.8073	0.7685	0.9427
0.1513	19.0	19741	0.1653	0.7630	0.7920	0.7772	0.9453
0.1504	20.0	20780	0.1635	0.7645	0.8073	0.7853	0.9461
0.1491	21.0	21819	0.1591	0.7547	0.8262	0.7889	0.9473
0.1455	22.0	22858	0.1627	0.7634	0.8145	0.7881	0.9457
0.145	23.0	23897	0.1584	0.7529	0.8210	0.7855	0.9464
0.1438	24.0	24936	0.1603	0.7592	0.8012	0.7796	0.9466
0.1413	25.0	25975	0.1614	0.7699	0.8134	0.7911	0.9470
0.1437	26.0	27014	0.1594	0.7557	0.8226	0.7877	0.9465
0.1414	27.0	28053	0.1605	0.7680	0.8183	0.7923	0.9478
0.1385	28.0	29092	0.1631	0.7588	0.8028	0.7802	0.9459
0.1365	29.0	30131	0.1568	0.7701	0.8167	0.7927	0.9482
0.1352	30.0	31170	0.1607	0.7660	0.8271	0.7954	0.9481
0.1331	31.0	32209	0.1646	0.7627	0.8122	0.7867	0.9461
0.1328	32.0	33248	0.1658	0.7560	0.8176	0.7856	0.9464
0.1319	33.0	34287	0.1579	0.7639	0.8228	0.7923	0.9486
0.1309	34.0	35326	0.1595	0.7666	0.8151	0.7901	0.9471
0.1271	35.0	36365	0.1616	0.7645	0.8248	0.7935	0.9476
0.1262	36.0	37404	0.1615	0.7641	0.8104	0.7866	0.9464
0.1227	37.0	38443	0.1614	0.7667	0.8273	0.7958	0.9475
0.1207	38.0	39482	0.1640	0.7763	0.8014	0.7887	0.9472
0.1212	39.0	40521	0.1613	0.7716	0.8142	0.7923	0.9487
0.1192	40.0	41560	0.1596	0.7773	0.8196	0.7979	0.9493
0.1193	41.0	42599	0.1684	0.7769	0.8071	0.7917	0.9473
0.1171	42.0	43638	0.1636	0.7717	0.8183	0.7943	0.9471
0.1146	43.0	44677	0.1613	0.7675	0.8217	0.7937	0.9476
0.1154	44.0	45716	0.1648	0.7725	0.8066	0.7892	0.9467
0.1149	45.0	46755	0.1660	0.7745	0.8172	0.7953	0.9476
0.1133	46.0	47794	0.1655	0.7742	0.8187	0.7958	0.9480
0.1121	47.0	48833	0.1659	0.7768	0.8156	0.7957	0.9481
0.1104	48.0	49872	0.1663	0.7714	0.8129	0.7916	0.9478
0.1069	49.0	50911	0.1659	0.7746	0.8163	0.7949	0.9479
0.11	50.0	51950	0.1660	0.7730	0.8185	0.7951	0.9478

Framework versions

• Transformers 4.29.2
• Pytorch 2.0.1+cu117
• Datasets 2.12.0
• Tokenizers 0.13.3