hubert-japanese-large-noise-0427

This model is a fine-tuned version of rinna/japanese-hubert-large on the audiofolder dataset. It achieves the following results on the evaluation set:

• Loss: 0.3383
• Cer: 0.0896
• Wer: 0.998

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 1e-05
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: cosine
• lr_scheduler_warmup_steps: 12500.0
• num_epochs: 25

Training results

Training Loss	Epoch	Step	Validation Loss	Cer	Wer
9.4222	1.0	2500	8.3462	0.9998	1.0
3.8795	2.0	5000	3.7791	0.7299	1.0
3.6295	3.0	7500	3.5969	0.7280	1.0
1.451	4.0	10000	1.0974	0.1931	1.0
0.7754	5.0	12500	0.5525	0.1595	1.0
0.636	6.0	15000	0.4586	0.1605	1.0
0.5528	7.0	17500	0.4240	0.1377	1.0
0.5064	8.0	20000	0.3931	0.1412	1.0
0.4767	9.0	22500	0.3593	0.1403	1.0
0.449	10.0	25000	0.3519	0.1112	1.0
0.4261	11.0	27500	0.3578	0.1048	1.0
0.4131	12.0	30000	0.3459	0.1142	1.0
0.3807	13.0	32500	0.3355	0.1072	1.0
0.3759	14.0	35000	0.3380	0.0967	1.0
0.3532	15.0	37500	0.3310	0.1198	1.0
0.3469	16.0	40000	0.3383	0.0927	1.0
0.3297	17.0	42500	0.3363	0.0911	1.0
0.3347	18.0	45000	0.3333	0.0895	0.998
0.3225	19.0	47500	0.3393	0.0944	0.998
0.3199	20.0	50000	0.3341	0.0873	0.998
0.3141	21.0	52500	0.3363	0.0863	0.998
0.2927	22.0	55000	0.3384	0.0889	0.998
0.3051	23.0	57500	0.3389	0.0902	0.998
0.3072	24.0	60000	0.3387	0.0895	0.998
0.3109	25.0	62500	0.3383	0.0896	0.998

Framework versions

• Transformers 4.39.3
• Pytorch 2.2.2
• Datasets 2.18.0
• Tokenizers 0.15.1