mms-1b-all-sw-CV_Fleurs_AMMI_ALFFA-1hrs-v1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Wer: 0.2505
• Cer: 0.0866

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Val Wer	Val Cer
16.1327	1.0	37	1.1533	2.6305
14.5509	2.0	74	1.4415	1.7715
11.4505	3.0	111	1.1493	0.8650
7.3358	4.0	148	1.0074	0.8544
4.2752	5.0	185	1.0009	0.8517
3.2479	6.0	222	1.0628	0.6094
2.322	7.0	259	0.7326	0.2337
1.5058	8.0	296	0.4135	0.1305
1.0304	9.0	333	0.3108	0.1034
0.8109	10.0	370	0.2843	0.0967
0.7432	11.0	407	0.2771	0.0948
0.6892	12.0	444	0.2725	0.0935
0.6813	13.0	481	0.2694	0.0928
0.6618	14.0	518	0.2683	0.0925
0.6347	15.0	555	0.2661	0.0917
0.6194	16.0	592	0.2648	0.0913
0.6234	17.0	629	0.2643	0.0909
0.6042	18.0	666	0.2610	0.0904
0.6092	19.0	703	0.2608	0.0900
0.6032	20.0	740	0.2588	0.0896
0.5886	21.0	777	0.2578	0.0892
0.5548	22.0	814	0.2559	0.0888
0.5701	23.0	851	0.2557	0.0887
0.591	24.0	888	0.2556	0.0885
0.5481	25.0	925	0.2545	0.0882
0.5437	26.0	962	0.2541	0.0879
0.537	27.0	999	0.2531	0.0877
0.555	28.0	1036	0.2526	0.0875
0.5391	29.0	1073	0.2518	0.0873
0.531	30.0	1110	0.2523	0.0873
0.5248	31.0	1147	0.2517	0.0872
0.5125	32.0	1184	0.2514	0.0871
0.5133	33.0	1221	0.2513	0.0870
0.4936	34.0	1258	0.2519	0.0870
0.5095	35.0	1295	0.2514	0.0868
0.5121	36.0	1332	0.2509	0.0867
0.4983	37.0	1369	0.2505	0.0866

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0