---
license: mit
base_model: facebook/w2v-bert-2.0
tags:
- generated_from_trainer
metrics:
- wer
model-index:
- name: w2v-bert-2.0-br
  results: []
---

<!-- This model card has been generated automatically according to the information the Trainer had access to. You
should probably proofread and complete it, then remove this comment. -->

# w2v-bert-2.0-br

This model is a fine-tuned version of [facebook/w2v-bert-2.0](https://huggingface.co./facebook/w2v-bert-2.0) on the None dataset.
It achieves the following results on the evaluation set:
- Loss: 0.6660
- Wer: 42.4942
- Cer: 13.6525

## Model description

More information needed

## Intended uses & limitations

More information needed

## Training and evaluation data

More information needed

## Training procedure

### Training hyperparameters

The following hyperparameters were used during training:
- learning_rate: 6e-05
- train_batch_size: 4
- eval_batch_size: 8
- seed: 42
- gradient_accumulation_steps: 2
- total_train_batch_size: 8
- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
- lr_scheduler_type: linear
- lr_scheduler_warmup_ratio: 0.08
- lr_scheduler_warmup_steps: 500
- training_steps: 8001
- mixed_precision_training: Native AMP

### Training results

| Training Loss | Epoch | Step | Validation Loss | Wer     | Cer     |
|:-------------:|:-----:|:----:|:---------------:|:-------:|:-------:|
| 1.0369        | 0.58  | 500  | 1.2289          | 85.3288 | 32.8021 |
| 0.7211        | 1.16  | 1000 | 0.9727          | 70.1973 | 24.6147 |
| 0.5669        | 1.75  | 1500 | 0.8496          | 64.6176 | 21.7978 |
| 0.4229        | 2.33  | 2000 | 0.7448          | 57.2663 | 19.3988 |
| 0.4352        | 2.91  | 2500 | 0.6749          | 52.9790 | 17.4075 |
| 0.3392        | 3.49  | 3000 | 0.6703          | 50.9678 | 16.8375 |
| 0.2508        | 4.07  | 3500 | 0.6143          | 49.6249 | 16.2547 |
| 0.2303        | 4.65  | 4000 | 0.7121          | 48.4648 | 15.8534 |
| 0.1776        | 5.24  | 4500 | 0.6667          | 47.0777 | 15.2910 |
| 0.1645        | 5.82  | 5000 | 0.6715          | 46.1825 | 14.8910 |
| 0.1304        | 6.4   | 5500 | 0.7212          | 44.2784 | 14.5139 |
| 0.1157        | 6.98  | 6000 | 0.6678          | 44.2721 | 14.3043 |
| 0.0924        | 7.56  | 6500 | 0.6935          | 43.1310 | 13.9171 |
| 0.0517        | 8.14  | 7000 | 0.6746          | 42.8851 | 13.7599 |
| 0.0667        | 8.73  | 7500 | 0.6327          | 42.9733 | 13.8136 |
| 0.0483        | 9.31  | 8000 | 0.6660          | 42.4942 | 13.6525 |


### Framework versions

- Transformers 4.39.1
- Pytorch 2.0.1+cu117
- Datasets 2.18.0
- Tokenizers 0.15.2