Arabic-Morocco-Speech_To_Text-2

This model is a fine-tuned version of smerchi/Arabic-Morocco-Speech_To_Text on the clevelytics-INWI dataset. It achieves the following results on the evaluation set:

• Loss: 0.5924
• Wer: 38.6446

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 50
• num_epochs: 15
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.1426	1.0	30	0.5924	38.6446
0.044	2.0	60	0.6992	40.0865
0.0273	3.0	90	0.7299	41.5285
0.018	4.0	120	0.7187	39.9423
0.0072	5.0	150	0.7263	39.6539
0.0056	6.0	180	0.7556	40.1586
0.0037	7.0	210	0.7415	38.9329
0.0025	8.0	240	0.7665	38.9329
0.0011	9.0	270	0.7764	38.9329
0.0005	10.0	300	0.7848	39.9423
0.0004	11.0	330	0.7912	39.1492
0.0004	12.0	360	0.7953	39.3655
0.0004	13.0	390	0.7980	39.1492
0.0004	14.0	420	0.7998	39.2213
0.0003	15.0	450	0.8003	39.2213

Framework versions

• Transformers 4.41.2
• Pytorch 2.3.1+cu121
• Datasets 2.19.2
• Tokenizers 0.19.1