mms-1b-all-sw-CV_Fleurs_AMMI_ALFFA-10hrs-v1

This model is a fine-tuned version of facebook/mms-1b-all on the None dataset. It achieves the following results on the evaluation set:

• Wer: 0.2298
• Cer: 0.0792

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 5e-05
• train_batch_size: 8
• eval_batch_size: 4
• seed: 42
• gradient_accumulation_steps: 2
• total_train_batch_size: 16
• optimizer: Use OptimizerNames.ADAMW_TORCH with betas=(0.9,0.999) and epsilon=1e-08 and optimizer_args=No additional optimizer arguments
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 100
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Val Wer	Val Cer
6.1301	1.0	367	0.3014	0.1016
0.706	2.0	734	0.2576	0.0893
0.6107	3.0	1101	0.2494	0.0863
0.5664	4.0	1468	0.2467	0.0848
0.5353	5.0	1835	0.2430	0.0836
0.5086	6.0	2202	0.2414	0.0830
0.4963	7.0	2569	0.2404	0.0822
0.4773	8.0	2936	0.2391	0.0817
0.4619	9.0	3303	0.2396	0.0814
0.4566	10.0	3670	0.2379	0.0810
0.4474	11.0	4037	0.2376	0.0807
0.4356	12.0	4404	0.2374	0.0804
0.4331	13.0	4771	0.2361	0.0801
0.4297	14.0	5138	0.2364	0.0800
0.4219	15.0	5505	0.2356	0.0798
0.4067	16.0	5872	0.2341	0.0793
0.4158	17.0	6239	0.2348	0.0792
0.4071	18.0	6606	0.2337	0.0788
0.4018	19.0	6973	0.2334	0.0787
0.3907	20.0	7340	0.2329	0.0787
0.3937	21.0	7707	0.2318	0.0784
0.3895	22.0	8074	0.2329	0.0787
0.3851	23.0	8441	0.2322	0.0784
0.3844	24.0	8808	0.2313	0.0780
0.3799	25.0	9175	0.2333	0.0791
0.3754	26.0	9542	0.2344	0.0794
0.3733	27.0	9909	0.2290	0.0779
0.3761	28.0	10276	0.2321	0.0793
0.3678	29.0	10643	0.2329	0.0789
0.3639	30.0	11010	0.2327	0.0795
0.361	31.0	11377	0.2313	0.0791
0.3559	32.0	11744	0.2303	0.0788
0.3596	33.0	12111	0.2310	0.0797
0.358	34.0	12478	0.2422	0.0874
0.3501	35.0	12845	0.2309	0.0802
0.3533	36.0	13212	0.2293	0.0793
0.349	37.0	13579	0.2298	0.0792

Framework versions

• Transformers 4.48.1
• Pytorch 2.5.1+cu124
• Datasets 3.2.0
• Tokenizers 0.21.0