bert-29

This model is a fine-tuned version of deepset/bert-base-cased-squad2 on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 6.1544

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 4
• eval_batch_size: 4
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss
10.3108	0.02	5	12.3308
11.3676	0.05	10	12.2829
12.0965	0.07	15	12.2339
10.877	0.09	20	12.1855
11.5463	0.11	25	12.1375
10.4439	0.14	30	12.0898
11.3519	0.16	35	12.0421
11.2558	0.18	40	11.9943
11.0568	0.21	45	11.9462
11.3265	0.23	50	11.8980
9.511	0.25	55	11.8505
10.6662	0.28	60	11.8043
10.2726	0.3	65	11.7576
11.1502	0.32	70	11.7108
11.2245	0.34	75	11.6640
10.3183	0.37	80	11.6173
11.3083	0.39	85	11.5707
10.0481	0.41	90	11.5243
10.6689	0.44	95	11.4780
10.299	0.46	100	11.4322
10.9093	0.48	105	11.3863
10.0403	0.5	110	11.3403
10.3065	0.53	115	11.2948
10.4771	0.55	120	11.2493
9.685	0.57	125	11.2040
9.6081	0.6	130	11.1592
9.9776	0.62	135	11.1145
9.8161	0.64	140	11.0705
10.078	0.67	145	11.0263
10.5549	0.69	150	10.9828
9.9488	0.71	155	10.9384
10.1075	0.73	160	10.8946
9.4674	0.76	165	10.8509
10.077	0.78	170	10.8070
9.5688	0.8	175	10.7636
8.9946	0.83	180	10.7205
10.4224	0.85	185	10.6776
8.6883	0.87	190	10.6344
9.5508	0.89	195	10.5919
9.3088	0.92	200	10.5493
9.3799	0.94	205	10.5072
8.8564	0.96	210	10.4656
8.7775	0.99	215	10.4237
9.4815	1.01	220	10.3824
9.2469	1.03	225	10.3411
8.5056	1.06	230	10.3003
9.8006	1.08	235	10.2595
9.185	1.1	240	10.2181
9.3379	1.12	245	10.1769
8.5696	1.15	250	10.1359
8.4981	1.17	255	10.0960
8.9066	1.19	260	10.0554
9.043	1.22	265	10.0155
7.9105	1.24	270	9.9762
8.8268	1.26	275	9.9372
8.5896	1.28	280	9.8979
8.9422	1.31	285	9.8578
8.2276	1.33	290	9.8188
9.1443	1.35	295	9.7795
8.7467	1.38	300	9.7404
9.1588	1.4	305	9.7015
8.9789	1.42	310	9.6630
7.9135	1.44	315	9.6252
8.9771	1.47	320	9.5876
8.7356	1.49	325	9.5499
8.1398	1.51	330	9.5128
8.4803	1.54	335	9.4754
8.5553	1.56	340	9.4385
8.6194	1.58	345	9.4006
8.1879	1.61	350	9.3633
8.1535	1.63	355	9.3269
8.8795	1.65	360	9.2899
8.4935	1.67	365	9.2536
8.5901	1.7	370	9.2170
8.3013	1.72	375	9.1808
8.1959	1.74	380	9.1448
8.1158	1.77	385	9.1093
7.6286	1.79	390	9.0741
8.2353	1.81	395	9.0397
7.822	1.83	400	9.0050
8.0979	1.86	405	8.9706
8.7221	1.88	410	8.9365
7.8976	1.9	415	8.9018
8.1378	1.93	420	8.8677
7.8131	1.95	425	8.8335
8.2754	1.97	430	8.7997
7.6972	2.0	435	8.7658
7.99	2.02	440	8.7324
7.7428	2.04	445	8.6995
8.0031	2.06	450	8.6671
7.9012	2.09	455	8.6346
7.1693	2.11	460	8.6023
7.8746	2.13	465	8.5700
7.3467	2.16	470	8.5386
7.863	2.18	475	8.5073
7.5655	2.2	480	8.4759
7.8374	2.22	485	8.4446
7.731	2.25	490	8.4132
7.6392	2.27	495	8.3820
7.4547	2.29	500	8.3509
7.4642	2.32	505	8.3204
7.9489	2.34	510	8.2902
7.046	2.36	515	8.2605
7.2211	2.39	520	8.2313
7.2287	2.41	525	8.2026
7.0018	2.43	530	8.1745
7.4544	2.45	535	8.1465
7.8701	2.48	540	8.1183
7.2614	2.5	545	8.0897
6.9922	2.52	550	8.0616
7.1671	2.55	555	8.0342
7.7866	2.57	560	8.0062
6.9815	2.59	565	7.9786
6.9534	2.61	570	7.9514
7.2225	2.64	575	7.9251
7.1113	2.66	580	7.8986
6.7271	2.68	585	7.8728
7.132	2.71	590	7.8475
6.8046	2.73	595	7.8227
7.3063	2.75	600	7.7977
6.8367	2.78	605	7.7724
7.161	2.8	610	7.7475
6.6636	2.82	615	7.7231
7.3229	2.84	620	7.6984
6.6655	2.87	625	7.6743
6.8593	2.89	630	7.6502
6.8838	2.91	635	7.6263
7.0462	2.94	640	7.6031
6.7336	2.96	645	7.5800
6.5167	2.98	650	7.5580
6.6102	3.0	655	7.5359
6.8039	3.03	660	7.5140
7.1668	3.05	665	7.4918
6.6394	3.07	670	7.4694
6.8593	3.1	675	7.4473
6.4022	3.12	680	7.4258
6.9173	3.14	685	7.4050
6.2071	3.17	690	7.3850
6.512	3.19	695	7.3654
6.6548	3.21	700	7.3459
6.7666	3.23	705	7.3263
6.3916	3.26	710	7.3066
6.7645	3.28	715	7.2874
6.5965	3.3	720	7.2679
6.5361	3.33	725	7.2490
6.8693	3.35	730	7.2295
6.3229	3.37	735	7.2106
6.7505	3.39	740	7.1919
6.4917	3.42	745	7.1739
6.5649	3.44	750	7.1566
6.5177	3.46	755	7.1392
6.6282	3.49	760	7.1219
6.5035	3.51	765	7.1051
6.5631	3.53	770	7.0880
6.4593	3.56	775	7.0715
6.5314	3.58	780	7.0554
6.2695	3.6	785	7.0400
6.4792	3.62	790	7.0249
6.7222	3.65	795	7.0091
6.4972	3.67	800	6.9931
6.3063	3.69	805	6.9776
6.1834	3.72	810	6.9630
6.3814	3.74	815	6.9485
6.3444	3.76	820	6.9339
6.3784	3.78	825	6.9195
6.4047	3.81	830	6.9052
6.2368	3.83	835	6.8916
6.1245	3.85	840	6.8784
6.3089	3.88	845	6.8657
6.3674	3.9	850	6.8526
6.4337	3.92	855	6.8393
6.2115	3.94	860	6.8270
6.2734	3.97	865	6.8145
6.2301	3.99	870	6.8023
6.0973	4.01	875	6.7905
6.2143	4.04	880	6.7785
6.4512	4.06	885	6.7665
6.1737	4.08	890	6.7545
6.3221	4.11	895	6.7427
6.2879	4.13	900	6.7313
5.9436	4.15	905	6.7214
6.1258	4.17	910	6.7126
6.2819	4.2	915	6.7031
6.1344	4.22	920	6.6934
6.3769	4.24	925	6.6833
6.3609	4.27	930	6.6731
5.9827	4.29	935	6.6632
6.039	4.31	940	6.6541
6.0012	4.33	945	6.6451
6.0147	4.36	950	6.6362
5.8187	4.38	955	6.6275
5.9193	4.4	960	6.6191
6.28	4.43	965	6.6115
6.2678	4.45	970	6.6027
6.0973	4.47	975	6.5940
6.0822	4.5	980	6.5855
6.1009	4.52	985	6.5775
6.1271	4.54	990	6.5701
6.1592	4.56	995	6.5623
6.1096	4.59	1000	6.5551
5.9785	4.61	1005	6.5481
6.3988	4.63	1010	6.5409
6.0417	4.66	1015	6.5335
6.1195	4.68	1020	6.5261
6.0588	4.7	1025	6.5189
6.1183	4.72	1030	6.5121
5.9251	4.75	1035	6.5057
5.9547	4.77	1040	6.5001
6.0658	4.79	1045	6.4946
6.1259	4.82	1050	6.4889
6.0679	4.84	1055	6.4828
6.0607	4.86	1060	6.4769
5.9526	4.89	1065	6.4715
6.0828	4.91	1070	6.4658
5.9547	4.93	1075	6.4605
6.1024	4.95	1080	6.4551
5.9813	4.98	1085	6.4495
5.8972	5.0	1090	6.4440
5.7935	5.02	1095	6.4390
5.9187	5.05	1100	6.4349
5.9726	5.07	1105	6.4305
6.1756	5.09	1110	6.4254
5.8363	5.11	1115	6.4208
5.9026	5.14	1120	6.4165
6.07	5.16	1125	6.4123
5.963	5.18	1130	6.4077
5.8744	5.21	1135	6.4032
5.7556	5.23	1140	6.3994
5.9779	5.25	1145	6.3952
5.972	5.28	1150	6.3913
5.9615	5.3	1155	6.3873
5.996	5.32	1160	6.3834
5.8424	5.34	1165	6.3795
5.7417	5.37	1170	6.3762
5.9022	5.39	1175	6.3727
6.0184	5.41	1180	6.3693
5.749	5.44	1185	6.3659
5.773	5.46	1190	6.3631
6.0517	5.48	1195	6.3601
5.7407	5.5	1200	6.3573
5.9687	5.53	1205	6.3549
5.9979	5.55	1210	6.3518
6.1084	5.57	1215	6.3482
5.8697	5.6	1220	6.3447
6.0638	5.62	1225	6.3409
6.118	5.64	1230	6.3371
5.7951	5.67	1235	6.3334
5.7953	5.69	1240	6.3302
5.9258	5.71	1245	6.3272
6.0077	5.73	1250	6.3240
5.7704	5.76	1255	6.3209
5.8541	5.78	1260	6.3181
5.9699	5.8	1265	6.3153
5.8741	5.83	1270	6.3126
5.917	5.85	1275	6.3100
5.9787	5.87	1280	6.3070
5.9342	5.89	1285	6.3044
6.0153	5.92	1290	6.3018
5.9102	5.94	1295	6.2993
5.8239	5.96	1300	6.2970
5.8519	5.99	1305	6.2946
5.7885	6.01	1310	6.2925
5.7097	6.03	1315	6.2907
5.8986	6.06	1320	6.2885
5.9841	6.08	1325	6.2864
5.706	6.1	1330	6.2843
5.6936	6.12	1335	6.2827
5.8226	6.15	1340	6.2810
5.8315	6.17	1345	6.2791
5.9115	6.19	1350	6.2774
5.8574	6.22	1355	6.2754
5.6731	6.24	1360	6.2736
5.8267	6.26	1365	6.2719
5.9179	6.28	1370	6.2699
5.8623	6.31	1375	6.2682
5.5588	6.33	1380	6.2668
5.675	6.35	1385	6.2656
5.9247	6.38	1390	6.2642
5.9254	6.4	1395	6.2624
5.6931	6.42	1400	6.2608
5.872	6.44	1405	6.2593
5.9024	6.47	1410	6.2574
5.8604	6.49	1415	6.2557
5.7363	6.51	1420	6.2541
5.7869	6.54	1425	6.2528
6.0195	6.56	1430	6.2511
5.8393	6.58	1435	6.2493
5.7697	6.61	1440	6.2476
5.9471	6.63	1445	6.2460
5.9015	6.65	1450	6.2440
5.9454	6.67	1455	6.2419
5.9572	6.7	1460	6.2399
5.8503	6.72	1465	6.2381
5.8685	6.74	1470	6.2361
5.9132	6.77	1475	6.2344
6.0508	6.79	1480	6.2325
5.7752	6.81	1485	6.2307
5.7491	6.83	1490	6.2292
5.8327	6.86	1495	6.2279
5.8021	6.88	1500	6.2266
5.909	6.9	1505	6.2253
5.7635	6.93	1510	6.2237
5.7958	6.95	1515	6.2225
5.7834	6.97	1520	6.2212
5.8064	7.0	1525	6.2202
5.7643	7.02	1530	6.2191
5.7698	7.04	1535	6.2181
5.947	7.06	1540	6.2168
5.651	7.09	1545	6.2156
5.7821	7.11	1550	6.2144
5.9321	7.13	1555	6.2133
5.7556	7.16	1560	6.2122
5.9326	7.18	1565	6.2109
5.8153	7.2	1570	6.2098
5.8886	7.22	1575	6.2086
5.791	7.25	1580	6.2075
5.6872	7.27	1585	6.2066
5.8454	7.29	1590	6.2054
5.6718	7.32	1595	6.2045
5.94	7.34	1600	6.2034
5.7839	7.36	1605	6.2023
5.7479	7.39	1610	6.2014
5.7667	7.41	1615	6.2004
5.6903	7.43	1620	6.1996
5.7733	7.45	1625	6.1988
5.7689	7.48	1630	6.1978
5.6635	7.5	1635	6.1972
5.6859	7.52	1640	6.1965
5.9551	7.55	1645	6.1955
5.9198	7.57	1650	6.1945
5.8671	7.59	1655	6.1935
5.8787	7.61	1660	6.1926
5.672	7.64	1665	6.1916
5.6778	7.66	1670	6.1908
5.7932	7.68	1675	6.1901
5.7728	7.71	1680	6.1893
5.7485	7.73	1685	6.1885
5.7004	7.75	1690	6.1878
5.8689	7.78	1695	6.1871
5.8178	7.8	1700	6.1862
5.7805	7.82	1705	6.1855
5.6899	7.84	1710	6.1847
5.7139	7.87	1715	6.1841
5.8343	7.89	1720	6.1834
5.617	7.91	1725	6.1828
5.6653	7.94	1730	6.1822
5.7244	7.96	1735	6.1817
5.7212	7.98	1740	6.1811
5.8714	8.0	1745	6.1806
5.7405	8.03	1750	6.1798
5.9066	8.05	1755	6.1792
5.8379	8.07	1760	6.1786
5.8899	8.1	1765	6.1778
5.835	8.12	1770	6.1770
5.691	8.14	1775	6.1764
5.9066	8.17	1780	6.1758
5.7688	8.19	1785	6.1751
5.7538	8.21	1790	6.1746
5.6364	8.23	1795	6.1740
5.8411	8.26	1800	6.1734
5.7645	8.28	1805	6.1729
5.8873	8.3	1810	6.1723
5.5752	8.33	1815	6.1718
5.8784	8.35	1820	6.1712
5.7117	8.37	1825	6.1710
5.8186	8.39	1830	6.1704
5.6907	8.42	1835	6.1699
5.7	8.44	1840	6.1694
5.5417	8.46	1845	6.1692
5.8637	8.49	1850	6.1687
5.7281	8.51	1855	6.1682
5.6528	8.53	1860	6.1677
5.7258	8.56	1865	6.1674
5.7621	8.58	1870	6.1669
5.7232	8.6	1875	6.1666
5.5538	8.62	1880	6.1661
5.7917	8.65	1885	6.1658
5.8579	8.67	1890	6.1654
5.6824	8.69	1895	6.1649
5.7336	8.72	1900	6.1646
5.7791	8.74	1905	6.1641
5.5699	8.76	1910	6.1638
5.7826	8.78	1915	6.1634
5.6718	8.81	1920	6.1632
5.829	8.83	1925	6.1629
5.5786	8.85	1930	6.1625
5.7453	8.88	1935	6.1622
5.6309	8.9	1940	6.1620
5.7146	8.92	1945	6.1617
5.7131	8.94	1950	6.1614
5.7432	8.97	1955	6.1609
5.6304	8.99	1960	6.1607
5.6925	9.01	1965	6.1605
5.7406	9.04	1970	6.1601
5.6347	9.06	1975	6.1600
5.6896	9.08	1980	6.1597
5.6159	9.11	1985	6.1594
5.9093	9.13	1990	6.1593
5.7172	9.15	1995	6.1590
5.7223	9.17	2000	6.1587
5.6943	9.2	2005	6.1586
5.7278	9.22	2010	6.1585
5.6541	9.24	2015	6.1583
5.8852	9.27	2020	6.1580
5.8833	9.29	2025	6.1579
5.6182	9.31	2030	6.1577
5.7419	9.33	2035	6.1575
5.8911	9.36	2040	6.1572
5.6314	9.38	2045	6.1571
5.714	9.4	2050	6.1568
5.7446	9.43	2055	6.1566
5.7887	9.45	2060	6.1565
5.8779	9.47	2065	6.1563
5.7857	9.5	2070	6.1561
5.8314	9.52	2075	6.1560
5.584	9.54	2080	6.1558
5.6878	9.56	2085	6.1556
5.9123	9.59	2090	6.1555
5.7777	9.61	2095	6.1554
5.8798	9.63	2100	6.1554
5.7343	9.66	2105	6.1552
5.6734	9.68	2110	6.1550
5.6331	9.7	2115	6.1550
5.5947	9.72	2120	6.1550
5.8934	9.75	2125	6.1549
5.6424	9.77	2130	6.1548
5.4537	9.79	2135	6.1548
5.799	9.82	2140	6.1547
5.5778	9.84	2145	6.1546
5.6188	9.86	2150	6.1546
5.8389	9.89	2155	6.1545
5.7505	9.91	2160	6.1544
5.7237	9.93	2165	6.1544
5.7	9.95	2170	6.1544
5.7322	9.98	2175	6.1544
5.6562	10.0	2180	6.1544

Framework versions

• Transformers 4.34.1
• Pytorch 2.0.1+cu118
• Datasets 2.14.5
• Tokenizers 0.14.1