End of training

README.md

@@ -0,0 +1,97 @@
+---
+license: apache-2.0
+base_model: joorock12/wav2vec2-large-xlsr-italian
+tags:
+- generated_from_trainer
+datasets:
+- common_voice_1_0
+metrics:
+- wer
+model-index:
+- name: DynamicWav2Vec_TEST_11
+  results: []
+---
+
+<!-- This model card has been generated automatically according to the information the Trainer had access to. You
+should probably proofread and complete it, then remove this comment. -->
+
+# DynamicWav2Vec_TEST_11
+
+This model is a fine-tuned version of [joorock12/wav2vec2-large-xlsr-italian](https://huggingface.co/joorock12/wav2vec2-large-xlsr-italian) on the common_voice_1_0 dataset.
+It achieves the following results on the evaluation set:
+- Loss: 0.1250
+- Wer: 0.2074
+
+## Model description
+
+More information needed
+
+## Intended uses & limitations
+
+More information needed
+
+## Training and evaluation data
+
+More information needed
+
+## Training procedure
+
+### Training hyperparameters
+
+The following hyperparameters were used during training:
+- learning_rate: 0.0001
+- train_batch_size: 4
+- eval_batch_size: 4
+- seed: 42
+- optimizer: Adam with betas=(0.9,0.999) and epsilon=1e-08
+- lr_scheduler_type: linear
+- lr_scheduler_warmup_steps: 500
+- num_epochs: 30
+
+### Training results
+
+| Training Loss | Epoch | Step  | Validation Loss | Wer    |
+|:-------------:|:-----:|:-----:|:---------------:|:------:|
+| 0.1961        | 0.84  | 500   | 0.0733          | 0.1918 |
+| 0.1958        | 1.67  | 1000  | 0.0808          | 0.1971 |
+| 0.1816        | 2.51  | 1500  | 0.0884          | 0.2061 |
+| 0.1615        | 3.34  | 2000  | 0.0978          | 0.2076 |
+| 0.1501        | 4.18  | 2500  | 0.0954          | 0.2083 |
+| 0.1449        | 5.02  | 3000  | 0.1094          | 0.2139 |
+| 0.143         | 5.85  | 3500  | 0.1036          | 0.2134 |
+| 0.1343        | 6.69  | 4000  | 0.1034          | 0.2171 |
+| 0.1285        | 7.53  | 4500  | 0.1150          | 0.2171 |
+| 0.1191        | 8.36  | 5000  | 0.1195          | 0.2171 |
+| 0.1163        | 9.2   | 5500  | 0.1212          | 0.2206 |
+| 0.1118        | 10.03 | 6000  | 0.1184          | 0.2217 |
+| 0.1042        | 10.87 | 6500  | 0.1211          | 0.2164 |
+| 0.1015        | 11.71 | 7000  | 0.1152          | 0.2172 |
+| 0.0999        | 12.54 | 7500  | 0.1273          | 0.2178 |
+| 0.1017        | 13.38 | 8000  | 0.1287          | 0.2180 |
+| 0.0988        | 14.21 | 8500  | 0.1299          | 0.2136 |
+| 0.0889        | 15.05 | 9000  | 0.1321          | 0.2166 |
+| 0.0913        | 15.89 | 9500  | 0.1292          | 0.2197 |
+| 0.093         | 16.72 | 10000 | 0.1354          | 0.2185 |
+| 0.0859        | 17.56 | 10500 | 0.1254          | 0.2186 |
+| 0.0788        | 18.39 | 11000 | 0.1326          | 0.2198 |
+| 0.0874        | 19.23 | 11500 | 0.1387          | 0.2175 |
+| 0.0889        | 20.07 | 12000 | 0.1314          | 0.2171 |
+| 0.0789        | 20.9  | 12500 | 0.1238          | 0.2126 |
+| 0.0724        | 21.74 | 13000 | 0.1278          | 0.2150 |
+| 0.0706        | 22.58 | 13500 | 0.1256          | 0.2126 |
+| 0.0739        | 23.41 | 14000 | 0.1316          | 0.2107 |
+| 0.0711        | 24.25 | 14500 | 0.1298          | 0.2120 |
+| 0.0626        | 25.08 | 15000 | 0.1290          | 0.2120 |
+| 0.0684        | 25.92 | 15500 | 0.1264          | 0.2088 |
+| 0.0726        | 26.76 | 16000 | 0.1242          | 0.2094 |
+| 0.0652        | 27.59 | 16500 | 0.1263          | 0.2083 |
+| 0.0597        | 28.43 | 17000 | 0.1265          | 0.2086 |
+| 0.0614        | 29.26 | 17500 | 0.1250          | 0.2074 |
+
+
+### Framework versions
+
+- Transformers 4.34.0
+- Pytorch 2.0.1+cu118
+- Datasets 2.14.5
+- Tokenizers 0.14.1

config.json

@@ -7,9 +7,9 @@
   "add_adapter": false,
   "apply_spec_augment": true,
   "architectures": [
-    "Wav2Vec2ForCTC"
+    "MyWav2Vec2ForCTC"
   ],
-  "attention_dropout": 0.0,
+  "attention_dropout": 0.1,
   "bos_token_id": 1,
   "classifier_proj_size": 256,
   "codevector_dim": 256,
@@ -43,7 +43,7 @@
     2
   ],
   "ctc_loss_reduction": "mean",
-  "ctc_zero_infinity": false,
+  "ctc_zero_infinity": true,
   "diversity_loss_weight": 0.1,
   "do_stable_layer_norm": true,
   "eos_token_id": 2,
@@ -59,7 +59,7 @@
   "initializer_range": 0.02,
   "intermediate_size": 4096,
   "layer_norm_eps": 1e-05,
-  "layerdrop": 0.0,
+  "layerdrop": 0.1,
   "mask_channel_length": 10,
   "mask_channel_min_space": 1,
   "mask_channel_other": 0.0,
@@ -109,7 +109,7 @@
     1
   ],
   "torch_dtype": "float32",
-  "transformers_version": "4.33.1",
+  "transformers_version": "4.34.0",
   "use_weighted_layer_sum": false,
   "vocab_size": 147,
   "xvector_output_dim": 512

pytorch_model.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e5f47c8a031f34ac114ef5883219a091858cd14b00ff0da0711c075a5fa4fc30
-size 1262504557
+oid sha256:f8919521bce1087478c2541d5032aa4766c0f9d368b3c4d8c48280255919e25c
+size 1283014437

training_args.bin

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:291b8f5bd3fc4c11185a8d7c6ea3ce26e8652b2d95eda2259df8aedb64c24088
+oid sha256:44ffae33484139899096f14895b7335d0c8f7cd936aef39cebce9601bea535eb
 size 4027