arabert_cross_vocabulary_task1_fold5

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.2973
• Qwk: 0.8562
• Mse: 0.2979

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 64
• eval_batch_size: 64
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 10

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.1333	2	1.8520	0.1480	1.8511
No log	0.2667	4	1.0651	0.3341	1.0655
No log	0.4	6	1.1438	0.4927	1.1443
No log	0.5333	8	0.9722	0.6250	0.9725
No log	0.6667	10	0.6730	0.6107	0.6733
No log	0.8	12	0.5341	0.7630	0.5344
No log	0.9333	14	0.4706	0.8016	0.4708
No log	1.0667	16	0.4255	0.8194	0.4256
No log	1.2	18	0.4901	0.8387	0.4901
No log	1.3333	20	0.4642	0.8545	0.4642
No log	1.4667	22	0.3781	0.8345	0.3782
No log	1.6	24	0.3229	0.8175	0.3231
No log	1.7333	26	0.3530	0.8366	0.3532
No log	1.8667	28	0.4548	0.8719	0.4547
No log	2.0	30	0.6182	0.8633	0.6181
No log	2.1333	32	0.5731	0.8699	0.5733
No log	2.2667	34	0.3622	0.8422	0.3626
No log	2.4	36	0.2913	0.8430	0.2916
No log	2.5333	38	0.2887	0.8426	0.2890
No log	2.6667	40	0.2802	0.8370	0.2805
No log	2.8	42	0.3048	0.8363	0.3050
No log	2.9333	44	0.3487	0.8575	0.3490
No log	3.0667	46	0.3168	0.8493	0.3171
No log	3.2	48	0.2820	0.8372	0.2823
No log	3.3333	50	0.3036	0.8603	0.3040
No log	3.4667	52	0.3945	0.8659	0.3949
No log	3.6	54	0.3717	0.8677	0.3721
No log	3.7333	56	0.2969	0.8614	0.2973
No log	3.8667	58	0.2569	0.8385	0.2572
No log	4.0	60	0.2971	0.7270	0.2975
No log	4.1333	62	0.3312	0.6927	0.3316
No log	4.2667	64	0.2800	0.7702	0.2804
No log	4.4	66	0.2896	0.8502	0.2900
No log	4.5333	68	0.4169	0.8694	0.4173
No log	4.6667	70	0.4616	0.8813	0.4620
No log	4.8	72	0.3663	0.8659	0.3668
No log	4.9333	74	0.2765	0.8520	0.2769
No log	5.0667	76	0.3059	0.7201	0.3063
No log	5.2	78	0.3256	0.6901	0.3260
No log	5.3333	80	0.2884	0.7612	0.2888
No log	5.4667	82	0.2705	0.8362	0.2710
No log	5.6	84	0.3435	0.8601	0.3440
No log	5.7333	86	0.4426	0.8795	0.4430
No log	5.8667	88	0.4328	0.8789	0.4333
No log	6.0	90	0.3734	0.8717	0.3738
No log	6.1333	92	0.3017	0.8603	0.3022
No log	6.2667	94	0.2650	0.8366	0.2655
No log	6.4	96	0.2634	0.8175	0.2639
No log	6.5333	98	0.2676	0.8404	0.2681
No log	6.6667	100	0.2966	0.8607	0.2972
No log	6.8	102	0.3292	0.8611	0.3297
No log	6.9333	104	0.3536	0.8685	0.3541
No log	7.0667	106	0.3466	0.8685	0.3470
No log	7.2	108	0.3070	0.8599	0.3075
No log	7.3333	110	0.2886	0.8636	0.2891
No log	7.4667	112	0.2905	0.8636	0.2910
No log	7.6	114	0.2879	0.8636	0.2884
No log	7.7333	116	0.2898	0.8633	0.2904
No log	7.8667	118	0.2981	0.8587	0.2987
No log	8.0	120	0.2958	0.8593	0.2963
No log	8.1333	122	0.3001	0.8607	0.3006
No log	8.2667	124	0.2931	0.8606	0.2936
No log	8.4	126	0.2793	0.8630	0.2798
No log	8.5333	128	0.2685	0.8532	0.2690
No log	8.6667	130	0.2691	0.8518	0.2696
No log	8.8	132	0.2769	0.8571	0.2775
No log	8.9333	134	0.2923	0.8575	0.2929
No log	9.0667	136	0.3156	0.8576	0.3161
No log	9.2	138	0.3326	0.8643	0.3332
No log	9.3333	140	0.3349	0.8643	0.3355
No log	9.4667	142	0.3280	0.8657	0.3286
No log	9.6	144	0.3162	0.8599	0.3168
No log	9.7333	146	0.3062	0.8582	0.3068
No log	9.8667	148	0.2995	0.8549	0.3001
No log	10.0	150	0.2973	0.8562	0.2979

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1