arabert_cross_vocabulary_task2_fold1

This model is a fine-tuned version of aubmindlab/bert-base-arabertv02 on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.6221
• Qwk: 0.0
• Mse: 0.6116

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 2e-05
• train_batch_size: 16
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 1

Training results

Training Loss	Epoch	Step	Validation Loss	Qwk	Mse
No log	0.0317	2	3.8890	0.0036	3.9116
No log	0.0635	4	1.2548	0.0450	1.2615
No log	0.0952	6	1.0365	0.0179	1.0371
No log	0.1270	8	0.7758	0.0128	0.7732
No log	0.1587	10	0.5362	0.0522	0.5302
No log	0.1905	12	0.5544	-0.0096	0.5473
No log	0.2222	14	0.5722	0.0351	0.5645
No log	0.2540	16	0.5726	0.0	0.5645
No log	0.2857	18	0.5582	0.0351	0.5504
No log	0.3175	20	0.5464	0.0351	0.5390
No log	0.3492	22	0.5965	0.0	0.5873
No log	0.3810	24	0.6243	0.0	0.6145
No log	0.4127	26	0.6556	0.0	0.6452
No log	0.4444	28	0.6380	0.0	0.6281
No log	0.4762	30	0.6416	0.0	0.6314
No log	0.5079	32	0.6227	0.0	0.6126
No log	0.5397	34	0.6384	0.0	0.6278
No log	0.5714	36	0.6024	0.0	0.5927
No log	0.6032	38	0.5759	0.0351	0.5670
No log	0.6349	40	0.5609	0.0351	0.5526
No log	0.6667	42	0.5594	0.0351	0.5510
No log	0.6984	44	0.5555	0.0351	0.5473
No log	0.7302	46	0.5577	0.0351	0.5494
No log	0.7619	48	0.5519	0.0351	0.5439
No log	0.7937	50	0.5591	0.0351	0.5506
No log	0.8254	52	0.5659	0.0351	0.5571
No log	0.8571	54	0.5837	0.0	0.5744
No log	0.8889	56	0.5989	0.0	0.5891
No log	0.9206	58	0.6091	0.0	0.5991
No log	0.9524	60	0.6152	0.0	0.6050
No log	0.9841	62	0.6221	0.0	0.6116

Framework versions

• Transformers 4.44.0
• Pytorch 2.4.0
• Datasets 2.21.0
• Tokenizers 0.19.1