elgeish-wav2vec2-arabic-fine-tuning_6P

This model is a fine-tuned version of elgeish/wav2vec2-large-xlsr-53-arabic on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.4511
• Wer: 0.4936

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.001
• train_batch_size: 8
• eval_batch_size: 16
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
33.2595	0.8	100	10.7960	1.0
14.3333	1.6	200	8.1978	1.0
4.8209	2.4	300	3.2571	1.0
3.1719	3.2	400	3.1181	1.0
3.0831	4.0	500	3.0458	1.0
2.5752	4.8	600	1.5734	1.0
1.4728	5.6	700	1.2424	0.8933
1.1457	6.4	800	1.0115	0.8471
1.0544	7.2	900	1.1768	0.8726
1.065	8.0	1000	1.1300	0.8232
0.9797	8.8	1100	1.0768	0.8248
0.8787	9.6	1200	1.2050	0.8519
0.7859	10.4	1300	0.8281	0.7564
0.7123	11.2	1400	0.8351	0.7086
0.6248	12.0	1500	0.9252	0.7834
0.5965	12.8	1600	0.6848	0.6879
0.4854	13.6	1700	0.6451	0.6322
0.4371	14.4	1800	0.5714	0.6003
0.3767	15.2	1900	0.6853	0.6178
0.3472	16.0	2000	0.6118	0.6035
0.3105	16.8	2100	0.5476	0.5764
0.2706	17.6	2200	0.4950	0.5446
0.2378	18.4	2300	0.5300	0.5096
0.2028	19.2	2400	0.4686	0.5048
0.1851	20.0	2500	0.4511	0.4936

Framework versions

• Transformers 4.45.2
• Pytorch 2.4.0
• Datasets 3.1.0
• Tokenizers 0.20.3