ft_0124_korean_2

This model is a fine-tuned version of facebook/wav2vec2-xls-r-300m on the None dataset. It achieves the following results on the evaluation set:

• Loss: 0.5092
• Cer: 0.1001

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 500
• num_epochs: 20
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Cer
25.0263	0.25	500	5.1785	1.0
4.7178	0.5	1000	4.7781	1.0
4.4523	0.76	1500	4.1013	0.9077
3.2148	1.01	2000	2.3856	0.4777
2.4229	1.26	2500	1.8502	0.4047
2.0145	1.51	3000	1.5497	0.3475
1.7916	1.77	3500	1.3324	0.3076
1.5849	2.02	4000	1.1873	0.2773
1.3956	2.27	4500	1.0617	0.2578
1.3086	2.52	5000	0.9643	0.2368
1.2211	2.78	5500	0.8894	0.2246
1.1562	3.03	6000	0.8537	0.2189
1.0729	3.28	6500	0.7973	0.2101
1.0089	3.53	7000	0.7549	0.1959
1.0027	3.79	7500	0.7327	0.1945
0.9496	4.04	8000	0.7082	0.1849
0.887	4.29	8500	0.6909	0.1789
0.8607	4.54	9000	0.6617	0.1739
0.853	4.8	9500	0.6518	0.1730
0.8305	5.05	10000	0.6402	0.1657
0.774	5.3	10500	0.6365	0.1650
0.7621	5.55	11000	0.6206	0.1600
0.7553	5.81	11500	0.6080	0.1594
0.7186	6.06	12000	0.5951	0.1543
0.6772	6.31	12500	0.5814	0.1490
0.6752	6.56	13000	0.5815	0.1501
0.672	6.81	13500	0.5603	0.1440
0.6351	7.07	14000	0.5670	0.1439
0.6186	7.32	14500	0.5700	0.1431
0.6035	7.57	15000	0.5614	0.1417
0.5848	7.82	15500	0.5470	0.1396
0.5719	8.08	16000	0.5514	0.1386
0.556	8.33	16500	0.5515	0.1376
0.5596	8.58	17000	0.5407	0.1325
0.5472	8.83	17500	0.5405	0.1349
0.5309	9.09	18000	0.5279	0.1295
0.5072	9.34	18500	0.5275	0.1310
0.5072	9.59	19000	0.5330	0.1272
0.4905	9.84	19500	0.5238	0.1262
0.4842	10.1	20000	0.5234	0.1237
0.4513	10.35	20500	0.5210	0.1231
0.4513	10.6	21000	0.5165	0.1208
0.4541	10.85	21500	0.5189	0.1207
0.4417	11.11	22000	0.5209	0.1192
0.4337	11.36	22500	0.5246	0.1191
0.4339	11.61	23000	0.5210	0.1183
0.4357	11.86	23500	0.4990	0.1162
0.4066	12.12	24000	0.4956	0.1132
0.3932	12.37	24500	0.5064	0.1148
0.384	12.62	25000	0.5011	0.1134
0.3902	12.87	25500	0.5064	0.1130
0.3883	13.12	26000	0.5128	0.1121
0.3625	13.38	26500	0.5140	0.1119
0.3648	13.63	27000	0.5091	0.1108
0.365	13.88	27500	0.4923	0.1098
0.3604	14.13	28000	0.5062	0.1090
0.3517	14.39	28500	0.5007	0.1089
0.3485	14.64	29000	0.4956	0.1081
0.3407	14.89	29500	0.5090	0.1084
0.333	15.14	30000	0.5018	0.1067
0.3211	15.4	30500	0.5114	0.1063
0.3204	15.65	31000	0.4976	0.1053
0.3265	15.9	31500	0.4947	0.1046
0.3169	16.15	32000	0.4988	0.1043
0.304	16.41	32500	0.5115	0.1041
0.2944	16.66	33000	0.5144	0.1042
0.311	16.91	33500	0.5068	0.1025
0.2997	17.16	34000	0.5079	0.1030
0.288	17.42	34500	0.5065	0.1019
0.2897	17.67	35000	0.5077	0.1016
0.2939	17.92	35500	0.5003	0.1017
0.2766	18.17	36000	0.5116	0.1013
0.2841	18.43	36500	0.5019	0.1010
0.2882	18.68	37000	0.5046	0.1008
0.2678	18.93	37500	0.5086	0.1013
0.269	19.18	38000	0.5108	0.1001
0.2815	19.43	38500	0.5111	0.1001
0.2668	19.69	39000	0.5091	0.1000
0.2715	19.94	39500	0.5092	0.1001

Framework versions

• Transformers 4.36.2
• Pytorch 2.1.2+cu118
• Datasets 2.16.1
• Tokenizers 0.15.0