wav2vec2-minds14-en

This model is a fine-tuned version of facebook/wav2vec2-base on the minds14 dataset. It achieves the following results on the evaluation set:

• Loss: 5.5729
• Wer: 1.0

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• training_steps: 2000
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.3811	3.5088	100	6.8598	1.0006
4.9442	7.0175	200	6.6217	1.0018
4.6255	10.5263	300	6.2050	1.0
4.4037	14.0351	400	6.1160	1.0
4.1672	17.5439	500	5.7863	1.0
3.8786	21.0526	600	5.6219	1.0
3.6182	24.5614	700	5.4987	1.0
3.654	28.0702	800	5.6024	1.0
3.4135	31.5789	900	5.5648	1.0
3.3532	35.0877	1000	5.6507	1.0
3.344	38.5965	1100	5.5189	1.0
3.3233	42.1053	1200	5.6830	1.0
3.3983	45.6140	1300	5.5447	1.0
3.2433	49.1228	1400	5.5065	1.0
3.2082	52.6316	1500	5.4783	1.0
3.1958	56.1404	1600	5.5747	1.0
3.1756	59.6491	1700	5.5580	1.0
3.1757	63.1579	1800	5.5556	1.0
3.1758	66.6667	1900	5.6747	1.0
3.1373	70.1754	2000	5.5729	1.0

Framework versions

• Transformers 4.44.2
• Pytorch 2.5.0+cu121
• Datasets 3.1.0
• Tokenizers 0.19.1