arabert_cross_relevance_task7_fold2

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.3920
• Qwk: 0.0
• Mse: 0.3925

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.125	2	1.1351	0.0055	1.1338
No log	0.25	4	0.3327	0.0239	0.3327
No log	0.375	6	0.4966	0.0325	0.4972
No log	0.5	8	0.4252	0.0068	0.4257
No log	0.625	10	0.3269	0.0	0.3271
No log	0.75	12	0.3043	0.0	0.3043
No log	0.875	14	0.3478	-0.0164	0.3479
No log	1.0	16	0.5150	-0.1231	0.5155
No log	1.125	18	0.6025	-0.0901	0.6032
No log	1.25	20	0.5883	-0.1348	0.5891
No log	1.375	22	0.4750	-0.0759	0.4757
No log	1.5	24	0.3601	-0.0164	0.3606
No log	1.625	26	0.3455	0.0	0.3460
No log	1.75	28	0.3605	0.0	0.3611
No log	1.875	30	0.4021	-0.0875	0.4027
No log	2.0	32	0.4215	-0.0723	0.4221
No log	2.125	34	0.4273	-0.1017	0.4280
No log	2.25	36	0.4182	-0.0723	0.4188
No log	2.375	38	0.3673	-0.0042	0.3677
No log	2.5	40	0.3484	0.0	0.3488
No log	2.625	42	0.3234	0.0	0.3237
No log	2.75	44	0.3167	0.0	0.3170
No log	2.875	46	0.3171	0.0	0.3173
No log	3.0	48	0.3402	0.0	0.3406
No log	3.125	50	0.3835	-0.0462	0.3840
No log	3.25	52	0.3969	-0.0631	0.3975
No log	3.375	54	0.3965	-0.0631	0.3971
No log	3.5	56	0.3727	-0.0085	0.3732
No log	3.625	58	0.3399	0.0122	0.3403
No log	3.75	60	0.3210	0.0122	0.3213
No log	3.875	62	0.3144	0.0122	0.3145
No log	4.0	64	0.3203	0.0122	0.3205
No log	4.125	66	0.3278	0.0	0.3281
No log	4.25	68	0.3425	0.0	0.3429
No log	4.375	70	0.3631	-0.0085	0.3636
No log	4.5	72	0.3878	-0.0631	0.3884
No log	4.625	74	0.3841	-0.0631	0.3847
No log	4.75	76	0.3554	-0.0164	0.3558
No log	4.875	78	0.3433	0.0	0.3437
No log	5.0	80	0.3407	0.0	0.3410
No log	5.125	82	0.3450	0.0	0.3453
No log	5.25	84	0.3491	0.0	0.3494
No log	5.375	86	0.3528	0.0	0.3532
No log	5.5	88	0.3559	0.0	0.3564
No log	5.625	90	0.3543	0.0	0.3547
No log	5.75	92	0.3489	0.0	0.3493
No log	5.875	94	0.3478	0.0	0.3480
No log	6.0	96	0.3484	0.0	0.3486
No log	6.125	98	0.3566	0.0	0.3568
No log	6.25	100	0.3544	0.0	0.3546
No log	6.375	102	0.3495	0.0	0.3499
No log	6.5	104	0.3529	0.0	0.3534
No log	6.625	106	0.3633	0.0122	0.3638
No log	6.75	108	0.3602	0.0122	0.3608
No log	6.875	110	0.3558	0.0	0.3563
No log	7.0	112	0.3542	0.0	0.3548
No log	7.125	114	0.3543	0.0122	0.3548
No log	7.25	116	0.3536	0.0122	0.3542
No log	7.375	118	0.3534	0.0122	0.3539
No log	7.5	120	0.3538	0.0	0.3543
No log	7.625	122	0.3553	0.0	0.3557
No log	7.75	124	0.3581	0.0	0.3585
No log	7.875	126	0.3601	0.0	0.3605
No log	8.0	128	0.3648	0.0	0.3652
No log	8.125	130	0.3658	0.0	0.3662
No log	8.25	132	0.3655	0.0	0.3659
No log	8.375	134	0.3666	0.0	0.3670
No log	8.5	136	0.3714	0.0	0.3718
No log	8.625	138	0.3763	0.0	0.3767
No log	8.75	140	0.3809	0.0	0.3813
No log	8.875	142	0.3871	0.0	0.3875
No log	9.0	144	0.3938	0.0	0.3941
No log	9.125	146	0.3971	0.0	0.3974
No log	9.25	148	0.3971	0.0	0.3975
No log	9.375	150	0.3950	0.0	0.3953
No log	9.5	152	0.3939	0.0	0.3943
No log	9.625	154	0.3921	0.0	0.3925
No log	9.75	156	0.3913	0.0	0.3918
No log	9.875	158	0.3917	0.0	0.3922
No log	10.0	160	0.3920	0.0	0.3925

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1