whisper-large-v3-turbo-OpenSLR-GL

This model is a fine-tuned version of openai/whisper-large-v3-turbo trained on juanjucm/OpenSLR-SpeechT-GL-EN for Galician Text to Speech task. It takes galician speech audios as input and generates the correspondant transcription.

This Automatic Speech Recognition model, was developed to be the first stage of a Speech Translation cascade system for transcribing and translating Galician audios into English texts. After this first STT step, this Galician-to-English MT model can be applied over the generated Galician transcriptions to get English text translations.

The motivation behind this work is to increase the visibility of the Galician language, making it more accessible for non-Galician speakers to understand and engage with Galician audio content.

This model was developed during a 3-week Speech Translation workshop organised by Yasmin Moslem.

Performance and training details

Baseline model achieved a WER score of 20.1 on the evaluation dataset.

After fine-tuning, it achieves the following results on the evaluation set:

• Loss: 0.1613
• WER: 10.6845

The following hyperparameters were used during training:

• learning_rate: 3e-06
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• distributed_type: multi-GPU
• num_devices: 2
• gradient_accumulation_steps: 2
• total_train_batch_size: 64
• total_eval_batch_size: 16
• optimizer: Use adamw_torch with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• num_epochs: 10

Training results

We used WER (Word Error Rate) as our reference transcription metric for selecting the best checkpoint after training.

Training Loss	Epoch	Step	Validation Loss	Wer
0.2739	1.0	75	0.1898	11.4023
0.1841	2.0	150	0.1819	10.3673
0.0542	3.0	225	0.1919	10.6177
0.0399	4.0	300	0.1934	11.1352
0.0264	5.0	375	0.2042	11.2688
0.0143	6.0	450	0.2075	10.3840
0.0056	7.0	525	0.2198	10.8347
0.0063	8.0	600	0.2217	10.9683
0.0037	9.0	675	0.2258	10.5509
0.0042	10.0	750	0.2278	10.6845

Framework versions

• Transformers 4.47.1
• Pytorch 2.4.1+cu121
• Datasets 3.2.0
• Tokenizers 0.21.0