math_question_grade_detection_v12-16-24_v1

This model is a fine-tuned version of allenai/scibert_scivocab_uncased on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.6301
• Accuracy: 0.8194
• Precision: 0.8228
• Recall: 0.8194
• F1: 0.8200

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• training_steps: 6000

Training results

Training Loss	Epoch	Step	Validation Loss	Accuracy	Precision	Recall	F1
No log	0.0683	50	2.1123	0.1676	0.1211	0.1676	0.1026
No log	0.1366	100	2.0118	0.2613	0.2102	0.2613	0.1941
No log	0.2049	150	1.8750	0.3075	0.3556	0.3075	0.2833
No log	0.2732	200	1.7074	0.3689	0.4076	0.3689	0.3224
No log	0.3415	250	1.5071	0.4612	0.4925	0.4612	0.4492
No log	0.4098	300	1.4983	0.4120	0.5160	0.4120	0.3779
No log	0.4781	350	1.2997	0.5196	0.5526	0.5196	0.5059
No log	0.5464	400	1.1756	0.5849	0.6063	0.5849	0.5731
No log	0.6148	450	1.1104	0.6088	0.6260	0.6088	0.5997
1.654	0.6831	500	1.0897	0.6103	0.6149	0.6103	0.6053
1.654	0.7514	550	1.0162	0.6126	0.6221	0.6126	0.5963
1.654	0.8197	600	1.0077	0.6095	0.6405	0.6095	0.5904
1.654	0.8880	650	0.9427	0.6403	0.6608	0.6403	0.6277
1.654	0.9563	700	0.9067	0.6464	0.6576	0.6464	0.6352
1.654	1.0246	750	0.8812	0.6618	0.6745	0.6618	0.6443
1.654	1.0929	800	0.8706	0.6764	0.6824	0.6764	0.6729
1.654	1.1612	850	0.8650	0.6626	0.6800	0.6626	0.6584
1.654	1.2295	900	0.8226	0.6879	0.7069	0.6879	0.6792
1.654	1.2978	950	0.8039	0.7041	0.7102	0.7041	0.6999
0.9362	1.3661	1000	0.7681	0.7110	0.7194	0.7110	0.7057
0.9362	1.4344	1050	0.7844	0.6941	0.7128	0.6941	0.6916
0.9362	1.5027	1100	0.7334	0.7241	0.7274	0.7241	0.7219
0.9362	1.5710	1150	0.7071	0.7348	0.7371	0.7348	0.7313
0.9362	1.6393	1200	0.6984	0.7487	0.7544	0.7487	0.7486
0.9362	1.7077	1250	0.7166	0.7310	0.7375	0.7310	0.7317
0.9362	1.7760	1300	0.7009	0.7425	0.7476	0.7425	0.7386
0.9362	1.8443	1350	0.6653	0.7533	0.7584	0.7533	0.7521
0.9362	1.9126	1400	0.6670	0.7533	0.7666	0.7533	0.7539
0.9362	1.9809	1450	0.6622	0.7410	0.7482	0.7410	0.7414
0.7205	2.0492	1500	0.6442	0.7479	0.7521	0.7479	0.7420
0.7205	2.1175	1550	0.6465	0.7563	0.7637	0.7563	0.7567
0.7205	2.1858	1600	0.6719	0.7456	0.7684	0.7456	0.7437
0.7205	2.2541	1650	0.6189	0.7694	0.7831	0.7694	0.7721
0.7205	2.3224	1700	0.6196	0.7663	0.7726	0.7663	0.7647
0.7205	2.3907	1750	0.6442	0.7610	0.7612	0.7610	0.7592
0.7205	2.4590	1800	0.6156	0.7733	0.7765	0.7733	0.7736
0.7205	2.5273	1850	0.6003	0.7756	0.7813	0.7756	0.7766
0.7205	2.5956	1900	0.5974	0.7748	0.7781	0.7748	0.7756
0.7205	2.6639	1950	0.6170	0.7633	0.7697	0.7633	0.7609
0.5272	2.7322	2000	0.5920	0.7748	0.7774	0.7748	0.7751
0.5272	2.8005	2050	0.6260	0.7594	0.7754	0.7594	0.7602
0.5272	2.8689	2100	0.5824	0.7932	0.8011	0.7932	0.7929
0.5272	2.9372	2150	0.5796	0.7879	0.7888	0.7879	0.7861
0.5272	3.0055	2200	0.5765	0.7932	0.7959	0.7932	0.7923
0.5272	3.0738	2250	0.5710	0.7940	0.8033	0.7940	0.7956
0.5272	3.1421	2300	0.5902	0.7825	0.7881	0.7825	0.7822
0.5272	3.2104	2350	0.5540	0.7978	0.8007	0.7978	0.7982
0.5272	3.2787	2400	0.5843	0.7863	0.7963	0.7863	0.7869
0.5272	3.3470	2450	0.5719	0.8002	0.8071	0.8002	0.8004
0.4067	3.4153	2500	0.5610	0.8048	0.8115	0.8048	0.8063
0.4067	3.4836	2550	0.5584	0.8009	0.8068	0.8009	0.8023
0.4067	3.5519	2600	0.5661	0.7971	0.8023	0.7971	0.7983
0.4067	3.6202	2650	0.5789	0.7978	0.7996	0.7978	0.7970
0.4067	3.6885	2700	0.6037	0.7848	0.7934	0.7848	0.7856
0.4067	3.7568	2750	0.5666	0.8009	0.8084	0.8009	0.8024
0.4067	3.8251	2800	0.5925	0.7925	0.8055	0.7925	0.7932
0.4067	3.8934	2850	0.5872	0.8055	0.8124	0.8055	0.8073
0.4067	3.9617	2900	0.5637	0.8040	0.8056	0.8040	0.8033
0.4067	4.0301	2950	0.5385	0.8101	0.8129	0.8101	0.8100
0.3331	4.0984	3000	0.5727	0.7955	0.8020	0.7955	0.7972
0.3331	4.1667	3050	0.5755	0.7963	0.8021	0.7963	0.7962
0.3331	4.2350	3100	0.5668	0.8048	0.8097	0.8048	0.8058
0.3331	4.3033	3150	0.5994	0.7986	0.8083	0.7986	0.7999
0.3331	4.3716	3200	0.5886	0.7986	0.8054	0.7986	0.7996
0.3331	4.4399	3250	0.5933	0.7986	0.8091	0.7986	0.8006
0.3331	4.5082	3300	0.6012	0.8002	0.8086	0.8002	0.8017
0.3331	4.5765	3350	0.5947	0.8040	0.8073	0.8040	0.8031
0.3331	4.6448	3400	0.5596	0.8125	0.8132	0.8125	0.8121
0.3331	4.7131	3450	0.5737	0.8048	0.8082	0.8048	0.8054
0.2431	4.7814	3500	0.5822	0.8101	0.8155	0.8101	0.8110
0.2431	4.8497	3550	0.5520	0.8155	0.8177	0.8155	0.8157
0.2431	4.9180	3600	0.5730	0.8125	0.8157	0.8125	0.8127
0.2431	4.9863	3650	0.5790	0.8055	0.8147	0.8055	0.8069
0.2431	5.0546	3700	0.5803	0.8109	0.8139	0.8109	0.8116
0.2431	5.1230	3750	0.5903	0.8132	0.8152	0.8132	0.8130
0.2431	5.1913	3800	0.5632	0.8240	0.8261	0.8240	0.8245
0.2431	5.2596	3850	0.6303	0.8017	0.8077	0.8017	0.8031
0.2431	5.3279	3900	0.5857	0.8148	0.8198	0.8148	0.8158
0.2431	5.3962	3950	0.5705	0.8171	0.8195	0.8171	0.8176
0.1805	5.4645	4000	0.5788	0.8201	0.8204	0.8201	0.8200
0.1805	5.5328	4050	0.5936	0.8101	0.8149	0.8101	0.8104
0.1805	5.6011	4100	0.5875	0.8163	0.8195	0.8163	0.8166
0.1805	5.6694	4150	0.6021	0.8171	0.8224	0.8171	0.8182
0.1805	5.7377	4200	0.5693	0.8186	0.8216	0.8186	0.8192
0.1805	5.8060	4250	0.5950	0.8155	0.8177	0.8155	0.8157
0.1805	5.8743	4300	0.6180	0.8086	0.8143	0.8086	0.8091
0.1805	5.9426	4350	0.5957	0.8155	0.8197	0.8155	0.8162
0.1805	6.0109	4400	0.6080	0.8140	0.8179	0.8140	0.8142
0.1805	6.0792	4450	0.5948	0.8178	0.8197	0.8178	0.8183
0.1547	6.1475	4500	0.5838	0.8217	0.8228	0.8217	0.8219
0.1547	6.2158	4550	0.6166	0.8148	0.8178	0.8148	0.8148
0.1547	6.2842	4600	0.6036	0.8224	0.8264	0.8224	0.8230
0.1547	6.3525	4650	0.6064	0.8232	0.8265	0.8232	0.8229
0.1547	6.4208	4700	0.6158	0.8171	0.8206	0.8171	0.8177
0.1547	6.4891	4750	0.6404	0.8140	0.8185	0.8140	0.8142
0.1547	6.5574	4800	0.6165	0.8171	0.8211	0.8171	0.8179
0.1547	6.6257	4850	0.6126	0.8186	0.8237	0.8186	0.8193
0.1547	6.6940	4900	0.5903	0.8240	0.8251	0.8240	0.8242
0.1547	6.7623	4950	0.6012	0.8155	0.8203	0.8155	0.8165
0.1099	6.8306	5000	0.6131	0.8186	0.8208	0.8186	0.8191
0.1099	6.8989	5050	0.5935	0.8248	0.8262	0.8248	0.8252
0.1099	6.9672	5100	0.6264	0.8186	0.8216	0.8186	0.8189
0.1099	7.0355	5150	0.6274	0.8186	0.8225	0.8186	0.8192
0.1099	7.1038	5200	0.6375	0.8217	0.8233	0.8217	0.8218
0.1099	7.1721	5250	0.6362	0.8148	0.8185	0.8148	0.8154
0.1099	7.2404	5300	0.6180	0.8194	0.8220	0.8194	0.8199
0.1099	7.3087	5350	0.6279	0.8201	0.8252	0.8201	0.8211
0.1099	7.3770	5400	0.6052	0.8217	0.8234	0.8217	0.8219
0.1099	7.4454	5450	0.6075	0.8217	0.8228	0.8217	0.8219
0.0859	7.5137	5500	0.6354	0.8178	0.8220	0.8178	0.8183
0.0859	7.5820	5550	0.6367	0.8163	0.8205	0.8163	0.8170
0.0859	7.6503	5600	0.6088	0.8240	0.8254	0.8240	0.8242
0.0859	7.7186	5650	0.6100	0.8240	0.8269	0.8240	0.8245
0.0859	7.7869	5700	0.6208	0.8232	0.8258	0.8232	0.8239
0.0859	7.8552	5750	0.6302	0.8278	0.8301	0.8278	0.8283
0.0859	7.9235	5800	0.6295	0.8240	0.8268	0.8240	0.8246
0.0859	7.9918	5850	0.6438	0.8240	0.8284	0.8240	0.8247
0.0859	8.0601	5900	0.6334	0.8217	0.8257	0.8217	0.8224
0.0859	8.1284	5950	0.6313	0.8201	0.8237	0.8201	0.8208
0.0733	8.1967	6000	0.6301	0.8194	0.8228	0.8194	0.8200

Framework versions

• Transformers 4.46.3
• Pytorch 2.4.0
• Datasets 3.1.0
• Tokenizers 0.20.3