whisper-large-v3-chichewa

This model is a fine-tuned version of openai/whisper-large-v3 on a Chichewa dataset. The dataset details will be provided at a later stage. It achieves the following results on the evaluation set:

• Loss: 3.1427
• Wer: 101.3340

Model description

More information needed

Intended uses & limitations

The model is fine-tuned to perfom transcription of Chichewa. There are several versions of this model, please refer to the usage example notebook to see how to find the commit which has model with best WER. Alternatively, you can explore the Files and Versions tab, go to commits and find the commit with best WER (around 61). Also, its worth noting that the model repo doesnt have a tokenizer.json, as a results the model needs to be loaded using WhisperProcessor instead of AutoModel or other modules in Transformer.

Source of Funding for this Work

The dataset used to fine-tune this model, as well as the compute resources, were provided by Opportunity International.
This was part of a project in Malawi aimed at supporting the deployment of an LLM-based chatbot for agriculture, with the capability to handle voice interactions in the local language, Chichewa. A total of 30 hours was collected for this dataset but due to data quality issues, only 25 hours was used. About 30 minutes was also removed to be used as hold-out for further model evaluation.

Training and evaluation data

More information needed

Training procedure

Most of the training for this model involved trying to varying speech dataset sizes (5 hours, 10 hours up to 24 hours). As such, the different model commits represent different data sizes. More details will be added to each model commit.

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 32
• eval_batch_size: 16
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 10000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
0.9229	7.0423	1000	2.0780	86.3539
0.1427	14.0845	2000	2.5560	83.5493
0.087	21.1268	3000	2.6909	80.8704
0.0742	28.1690	4000	2.8007	81.8982
0.065	35.2113	5000	2.8871	84.3639
0.0627	42.2535	6000	2.9465	84.5334
0.0586	49.2958	7000	3.0451	114.1600
0.063	56.3380	8000	3.0983	82.6964
0.0588	63.3803	9000	3.1352	81.0180
0.0591	70.4225	10000	3.1427	101.3340

Framework versions

• Transformers 4.45.1
• Pytorch 2.0.1
• Datasets 3.0.1
• Tokenizers 0.20.0