Hugging Face's logo

csikasote
/
mms-1b-all-bem-natbed-n-model

Automatic Speech Recognition
Transformers
Safetensors
wav2vec2
natbed
mms
Generated from Trainer
Inference Endpoints
Model card Files Community
mms-1b-all-bem-natbed-n-model / README.md
csikasote's picture
csikasote
End of training
4b1c2de verified
preview code
|
raw
history blame contribute delete
3.49 kB
metadata
library_name: transformers
license: cc-by-nc-4.0
base_model: facebook/mms-1b-all
tags:
  - automatic-speech-recognition
  - natbed
  - mms
  - generated_from_trainer
metrics:
  - wer
model-index:
  - name: mms-1b-all-bem-natbed-n-model
    results: []

mms-1b-all-bem-natbed-n-model

This model is a fine-tuned version of facebook/mms-1b-all on the NATBED - BEM dataset. It achieves the following results on the evaluation set:

  • Loss: 0.5309
  • Wer: 0.4631

Model description

More information needed

Intended uses & limitations

More information needed

Training and evaluation data

More information needed

Training procedure

Training hyperparameters

The following hyperparameters were used during training:

  • learning_rate: 0.0003
  • train_batch_size: 8
  • eval_batch_size: 8
  • seed: 42
  • optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
  • lr_scheduler_type: linear
  • lr_scheduler_warmup_steps: 100
  • num_epochs: 30.0
  • mixed_precision_training: Native AMP

Training results

Training Loss Epoch Step Validation Loss Wer
7.3623 0.2809 100 0.9287 0.7283
0.9213 0.5618 200 0.6511 0.5931
0.7224 0.8427 300 0.6387 0.5434
0.7132 1.1236 400 0.6140 0.5213
0.7195 1.4045 500 0.6097 0.5146
0.7054 1.6854 600 0.6145 0.5126
0.7417 1.9663 700 0.6062 0.5257
0.7029 2.2472 800 0.6022 0.4947
0.6845 2.5281 900 0.5886 0.5023
0.663 2.8090 1000 0.5915 0.4926
0.7129 3.0899 1100 0.5833 0.4920
0.6735 3.3708 1200 0.5877 0.4832
0.672 3.6517 1300 0.5863 0.5151
0.6494 3.9326 1400 0.5795 0.4844
0.7049 4.2135 1500 0.5724 0.4716
0.5898 4.4944 1600 0.5640 0.4762
0.6581 4.7753 1700 0.5582 0.4724
0.6262 5.0562 1800 0.5447 0.4751
0.6179 5.3371 1900 0.5497 0.4656
0.5896 5.6180 2000 0.5444 0.4779
0.6438 5.8989 2100 0.5399 0.4700
0.6086 6.1798 2200 0.5520 0.4598
0.6226 6.4607 2300 0.5386 0.4797
0.6148 6.7416 2400 0.5574 0.4680
0.5838 7.0225 2500 0.5497 0.4639
0.5407 7.3034 2600 0.5377 0.4631
0.6186 7.5843 2700 0.5404 0.4715
0.5922 7.8652 2800 0.5381 0.4609
0.5799 8.1461 2900 0.5312 0.4620
0.5914 8.4270 3000 0.5309 0.4631
0.6194 8.7079 3100 0.5317 0.4678
0.5851 8.9888 3200 0.5389 0.4575
0.5764 9.2697 3300 0.5579 0.4550

Framework versions

  • Transformers 4.46.0.dev0
  • Pytorch 2.4.1+cu121
  • Datasets 3.0.1
  • Tokenizers 0.20.0