metadata

license: apache-2.0
base_model: facebook/hubert-large-ls960-ft
tags:
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: mascir_fr_hubert_version1000
    results: []

mascir_fr_hubert_version1000

This model is a fine-tuned version of facebook/hubert-large-ls960-ft on an unknown dataset. It achieves the following results on the evaluation set:

Loss: 0.8026
Wer: 0.5

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

learning_rate: 0.0001
train_batch_size: 8
eval_batch_size: 8
seed: 42
optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
lr_scheduler_type: linear
lr_scheduler_warmup_steps: 1000
num_epochs: 50

Training results

Training Loss	Epoch	Step	Validation Loss	Wer
5.9386	2.0	500	2.9031	0.9856
2.0513	4.0	1000	1.0727	0.9144
1.0528	6.0	1500	0.7645	0.7567
0.7915	8.0	2000	0.6926	0.6744
0.6418	10.0	2500	0.6881	0.6633
0.5558	12.0	3000	0.6724	0.5978
0.4792	14.0	3500	0.6674	0.6011
0.4236	16.0	4000	0.6907	0.5778
0.3808	18.0	4500	0.7231	0.5444
0.3364	20.0	5000	0.7069	0.5456
0.3193	22.0	5500	0.7189	0.5456
0.2827	24.0	6000	0.7432	0.5322
0.2769	26.0	6500	0.7838	0.5656
0.2543	28.0	7000	0.8012	0.5333
0.2365	30.0	7500	0.8180	0.5178
0.2274	32.0	8000	0.7943	0.5233
0.2095	34.0	8500	0.7664	0.5222
0.2055	36.0	9000	0.7621	0.5122
0.2044	38.0	9500	0.7712	0.5056
0.1946	40.0	10000	0.7987	0.4989
0.1891	42.0	10500	0.7978	0.5044
0.1878	44.0	11000	0.7894	0.4967
0.1742	46.0	11500	0.7964	0.4944
0.1701	48.0	12000	0.7990	0.4956
0.163	50.0	12500	0.8026	0.5

Framework versions

Transformers 4.31.0
Pytorch 2.0.1+cu118
Datasets 2.14.3
Tokenizers 0.13.3