README.md

metadata

library_name: transformers
language:
  - lg
license: mit
base_model: facebook/w2v-bert-2.0
tags:
  - generated_from_trainer
datasets:
  - Grain
metrics:
  - wer
model-index:
  - name: w
    results:
      - task:
          name: Automatic Speech Recognition
          type: automatic-speech-recognition
        dataset:
          name: Grain
          type: Grain
        metrics:
          - name: Wer
            type: wer
            value: 0.029878515924263983

w

This model is a fine-tuned version of facebook/w2v-bert-2.0 on the Grain dataset. It achieves the following results on the evaluation set:

• Loss: 0.0469
• Wer: 0.0299
• Cer: 0.0077

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

• learning_rate: 0.0001
• train_batch_size: 16
• eval_batch_size: 8
• seed: 42
• optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
• lr_scheduler_type: linear
• num_epochs: 100
• mixed_precision_training: Native AMP

Training results

Training Loss	Epoch	Step	Validation Loss	Wer	Cer
0.2995	1.0	1164	0.1521	0.1390	0.0283
0.1049	2.0	2328	0.0931	0.0946	0.0189
0.0719	3.0	3492	0.0861	0.0902	0.0183
0.0546	4.0	4656	0.0788	0.0704	0.0166
0.0447	5.0	5820	0.0609	0.0627	0.0135
0.0374	6.0	6984	0.0744	0.0618	0.0141
0.0338	7.0	8148	0.0673	0.0535	0.0137
0.029	8.0	9312	0.0770	0.0540	0.0128
0.0278	9.0	10476	0.0565	0.0482	0.0116
0.0227	10.0	11640	0.0516	0.0500	0.0115
0.0211	11.0	12804	0.0457	0.0392	0.0096
0.0207	12.0	13968	0.0527	0.0452	0.0098
0.0179	13.0	15132	0.0463	0.0370	0.0089
0.017	14.0	16296	0.0530	0.0452	0.0109
0.0167	15.0	17460	0.0447	0.0360	0.0091
0.0141	16.0	18624	0.0529	0.0434	0.0104
0.015	17.0	19788	0.0410	0.0387	0.0090
0.0141	18.0	20952	0.0480	0.0416	0.0102
0.0136	19.0	22116	0.0472	0.0368	0.0087
0.0125	20.0	23280	0.0428	0.0380	0.0091
0.0117	21.0	24444	0.0375	0.0328	0.0081
0.0113	22.0	25608	0.0392	0.0312	0.0083
0.0093	23.0	26772	0.0554	0.0394	0.0102
0.0111	24.0	27936	0.0624	0.0452	0.0108
0.0107	25.0	29100	0.0390	0.0346	0.0076
0.0082	26.0	30264	0.0505	0.0426	0.0101
0.0087	27.0	31428	0.0430	0.0320	0.0081
0.0086	28.0	32592	0.0541	0.0398	0.0101
0.0079	29.0	33756	0.0404	0.0304	0.0070
0.0084	30.0	34920	0.0416	0.0315	0.0075
0.0084	31.0	36084	0.0495	0.0366	0.0092
0.0075	32.0	37248	0.0469	0.0299	0.0077

Framework versions

• Transformers 4.45.2
• Pytorch 2.1.0+cu118
• Datasets 3.0.1
• Tokenizers 0.20.1