mascir_fr_hubert_version1000

This model is a fine-tuned version of facebook/hubert-large-ls960-ft on an unknown dataset. It achieves the following results on the evaluation set:

• Loss: 0.8026
• Wer: 0.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 8
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• lr_scheduler_warmup_steps: 1000
• num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.9386	2.0	500	2.9031	0.9856
2.0513	4.0	1000	1.0727	0.9144
1.0528	6.0	1500	0.7645	0.7567
0.7915	8.0	2000	0.6926	0.6744
0.6418	10.0	2500	0.6881	0.6633
0.5558	12.0	3000	0.6724	0.5978
0.4792	14.0	3500	0.6674	0.6011
0.4236	16.0	4000	0.6907	0.5778
0.3808	18.0	4500	0.7231	0.5444
0.3364	20.0	5000	0.7069	0.5456
0.3193	22.0	5500	0.7189	0.5456
0.2827	24.0	6000	0.7432	0.5322
0.2769	26.0	6500	0.7838	0.5656
0.2543	28.0	7000	0.8012	0.5333
0.2365	30.0	7500	0.8180	0.5178
0.2274	32.0	8000	0.7943	0.5233
0.2095	34.0	8500	0.7664	0.5222
0.2055	36.0	9000	0.7621	0.5122
0.2044	38.0	9500	0.7712	0.5056
0.1946	40.0	10000	0.7987	0.4989
0.1891	42.0	10500	0.7978	0.5044
0.1878	44.0	11000	0.7894	0.4967
0.1742	46.0	11500	0.7964	0.4944
0.1701	48.0	12000	0.7990	0.4956
0.163	50.0	12500	0.8026	0.5

Framework versions

• Transformers 4.31.0
• Pytorch 2.0.1+cu118
• Datasets 2.14.3
• Tokenizers 0.13.3