File size: 3,670 Bytes
e1891d9 dd57f71 e1891d9 3f171d3 e1891d9 3f171d3 e1891d9 dd57f71 e1891d9 5f1bd4c e1891d9 3f171d3 e1891d9 3f171d3 5973beb 3f171d3 e1891d9 dd57f71 e1891d9 5d8720c e1891d9 dd57f71 e1891d9 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 |
---
license: apache-2.0
language:
- ru
library_name: transformers
pipeline_tag: automatic-speech-recognition
base_model: waveletdeboshir/whisper-base-ru-pruned
tags:
- asr
- Pytorch
- pruned
- finetune
- audio
- automatic-speech-recognition
model-index:
- name: Whisper Base Pruned and Finetuned for Russian
results:
- task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 15.0 (Russian part, test)
type: mozilla-foundation/common_voice_15_0
args: ru
metrics:
- name: WER
type: wer
value: 26.52
- task:
name: Speech Recognition
type: automatic-speech-recognition
dataset:
name: Common Voice 15.0 (Russian part, test)
type: mozilla-foundation/common_voice_15_0
args: ru
metrics:
- name: WER (without punctuation)
type: wer
value: 21.35
datasets:
- mozilla-foundation/common_voice_15_0
---
# Whisper-base-ru-pruned-ft
## Model info
This is a finetuned version of pruned whisper-base model ([waveletdeboshir/whisper-base-ru-pruned](https://huggingface.co./waveletdeboshir/whisper-base-ru-pruned)) for Russian language.
Model was finetuned on russian part of [mozilla-foundation/common_voice_15_0](https://huggingface.co./datasets/mozilla-foundation/common_voice_15_0) with Specaugment, Colored Noise augmentation and Noise from file augmentation.
## Metrics
| metric | dataset | waveletdeboshir/whisper-base-ru-pruned | waveletdeboshir/whisper-base-ru-pruned-ft |
| :------ | :------ | :------ | :------ |
| WER (without punctuation) | common_voice_15_0_test | 0.3352 | **0.2135** |
| WER | common_voice_15_0_test | 0.4050 | **0.2652** |
## Limitations
Because texts in Common Voice don't contain digits and other characters except letters and punctuation signs, model lost an ability to predict numbers and special characters.
## Size
Only 10% tokens was left including special whisper tokens (no language tokens except \<|ru|\> and \<|en|\>, no timestamp tokens), 200 most popular tokens from tokenizer and 4000 most popular Russian tokens computed by tokenization of russian text corpus.
Model size is 30% less then original whisper-base:
| | openai/whisper-base | waveletdeboshir/whisper-base-ru-pruned-ft |
| :------ | :------ | :------ |
| n of parameters | 74 M | 48 M |
| n of parameters (with proj_out layer) | 99 M | 50 M |
| model file size | 290 Mb | 193 Mb |
| vocab_size | 51865 | 4207 |
## Usage
Model can be used as an original whisper:
```python
>>> from transformers import WhisperProcessor, WhisperForConditionalGeneration
>>> import torchaudio
>>> # load audio
>>> wav, sr = torchaudio.load("audio.wav")
>>> # load model and processor
>>> processor = WhisperProcessor.from_pretrained("waveletdeboshir/whisper-base-ru-pruned-ft")
>>> model = WhisperForConditionalGeneration.from_pretrained("waveletdeboshir/whisper-base-ru-pruned-ft")
>>> input_features = processor(wav[0], sampling_rate=sr, return_tensors="pt").input_features
>>> # generate token ids
>>> predicted_ids = model.generate(input_features)
>>> # decode token ids to text
>>> transcription = processor.batch_decode(predicted_ids, skip_special_tokens=False)
['<|startoftranscript|><|ru|><|transcribe|><|notimestamps|> Начинаем работу.<|endoftext|>']
```
The context tokens can be removed from the start of the transcription by setting `skip_special_tokens=True`.
## Other pruned whisper models
* [waveletdeboshir/whisper-tiny-ru-pruned](https://huggingface.co./waveletdeboshir/whisper-tiny-ru-pruned)
* [waveletdeboshir/whisper-small-ru-pruned](https://huggingface.co./waveletdeboshir/whisper-small-ru-pruned) |