Update README.md

README.md

@@ -1,126 +1,124 @@
----
-license: mit
----
----
-language:
-- th
-license: mit
-library_name: transformers
-tags:
-- whisper-event
-- generated_from_trainer
-datasets:
-- CMKL/Porjai-Thai-voice-dataset-central
-metrics:
-- wer
-base_model: biodatlab/whisper-th-medium-combined
-model-index:
-- name: Whisper Medium Thai Timestamp - biodatlab
-  results:
-  - task:
-      type: automatic-speech-recognition
-      name: Automatic Speech Recognition
-    dataset:
-      name: mozilla-foundation/common_voice_13_0 th
-      type: mozilla-foundation/common_voice_13_0
-      config: th
-      split: test
-      args: th
-    metrics:
-    - type: wer
-      value: 15.57
-      name: Wer
----
-
-# Whisper Medium (Thai) Timestamp
-
-This model is a fine-tuned version of [biodatlab/whisper-th-medium-combined](biodatlab/whisper-th-medium-combined) on a custom-created longform dataset derived from the CMKL/Porjai-Thai-voice-dataset-central. It achieves the following results on the common-voice-13 test set:
-- WER: 15.57 (with Deepcut Tokenizer)
-
-## Model description
-
-This model is designed to perform automatic speech recognition (ASR) for the Thai language, with the added capability of generating timestamps for the transcribed text. It's based on the Whisper medium architecture and has been fine-tuned on a specially crafted dataset to enable timestamp generation.
-
-Use the model with Hugging Face's `transformers` as follows:
-
-```py
-from transformers import pipeline
-import torch
-
-MODEL_NAME = "biodatlab/whisper-th-medium-timestamp"  # specify the model name
-lang = "th"  # Thai language
-
-device = 0 if torch.cuda.is_available() else "cpu"
-
-pipe = pipeline(
-    task="automatic-speech-recognition",
-    model=MODEL_NAME,
-    chunk_length_s=30,
-    device=device,
-    return_timestamps=True,
-)
-pipe.model.config.forced_decoder_ids = pipe.tokenizer.get_decoder_prompt_ids(
-  language=lang,
-  task="transcribe"
-)
-result = pipe("audio.mp3", return_timestamps=True)
-text = result["text"]
-timestamps = result["chunks"]
-```
-
-## Intended uses & limitations
-This model is intended for Thai automatic speech recognition tasks, particularly where timestamp information is required. It can be used for transcribing Thai audio content, creating subtitles, or any application that needs to align text with specific time points in audio.
-The model's performance on speech recognition may be lower compared to non-timestamped versions due to the additional complexity of the task and the pseudo-timestamp generation method used in training.
-## Training and evaluation data
-The model was trained on a custom-created longform dataset derived from the CMKL/Porjai-Thai-voice-dataset-central. The dataset creation process involved the following steps:
-
-- Combining multiple short audio clips from the original dataset into longer audio segments (up to 30 seconds).
-- Adding environmental noises and silences between clips to simulate more realistic speech scenarios.
-- Generating pseudo-timestamps for the combined audio using a Voice Activity Detection (VAD) model (Silero VAD).
-
-This approach allowed us to create a dataset with longer, more diverse audio samples and approximate timestamp information, which is crucial for training a model capable of generating timestamps.
-## Training procedure
-The model was fine-tuned using a custom training script that incorporates the following:
-
-- Mixed precision training (FP16)
-- Gradient accumulation
-- SpecAugment for data augmentation during training
-
-## Training hyperparameters
-The following hyperparameters were used during training:
-
-learning_rate: 1e-05
-train_batch_size: 8
-eval_batch_size: 8
-gradient_accumulation_steps: 1
-num_train_iters: 50000
-warmup_steps: 50
-fp16: True
-optimizer: AdamW
-lr_scheduler_type: linear
-
-## Framework versions
-
-Transformers 4.44.2
-Pytorch 2.4.1
-Datasets 3.0.0
-Tokenizers 0.20.0
-
-## Performance and Limitations
-The WER (Word Error Rate) of 15.57 on the Common Voice 13 test set indicates good performance for Thai ASR. However, it's important to note that the timestamp generation model has a lower accuracy compared to the non-timestamped version of the model. This is due to several factors:
-
-- The use of pseudo-timestamps in training data, which are approximations based on VAD rather than precise human annotations.
-- The additional complexity of the timestamp prediction task, which requires the model to learn both transcription and temporal alignment.
-- Potential discrepancies between the VAD-generated timestamps and actual word boundaries in continuous speech.
-
-Users should be aware that while the timestamps provide a general indication of when words or phrases occur in the audio, they may not be as precise as manually annotated timestamps. The model's performance may also vary depending on the acoustic conditions, speaker variability, and the presence of background noise in the input audio.
-## Citation
-If you use this model in your research or applications, please cite it as follows:
-
-@misc{biodatlab_whisper_th_medium_timestamp,
-    author       = {Atirut Boribalburephan, Zaw Htet Aung, Knot Pipatsrisawat, Titipat Achakulvisut},
-    title        = {Whisper Medium Thai Timestamp: A fine-tuned Whisper model for Thai automatic speech recognition with timestamp generation},
-    year         = 2024,
-    publisher    = {Hugging Face},
-    howpublished = {\url{https://huggingface.co/biodatlab/whisper-th-medium-timestamp}}
-}
+---
+language:
+- th
+license: mit
+library_name: transformers
+tags:
+- whisper-event
+- generated_from_trainer
+datasets:
+- CMKL/Porjai-Thai-voice-dataset-central
+metrics:
+- wer
+base_model: biodatlab/whisper-th-medium-combined
+model-index:
+- name: Whisper Medium Thai Timestamp - biodatlab
+  results:
+  - task:
+      type: automatic-speech-recognition
+      name: Automatic Speech Recognition
+    dataset:
+      name: mozilla-foundation/common_voice_13_0 th
+      type: mozilla-foundation/common_voice_13_0
+      config: th
+      split: test
+      args: th
+    metrics:
+    - type: wer
+      value: 15.57
+      name: Wer
+---
+
+# Whisper Medium (Thai) Timestamp
+
+This model is a fine-tuned version of [biodatlab/whisper-th-medium-combined](biodatlab/whisper-th-medium-combined) on a custom-created longform dataset derived from the CMKL/Porjai-Thai-voice-dataset-central. It achieves the following results on the common-voice-13 test set:
+- WER: 15.57 (with Deepcut Tokenizer)
+
+## Model description
+
+This model is designed to perform automatic speech recognition (ASR) for the Thai language, with the added capability of generating timestamps for the transcribed text. It's based on the Whisper medium architecture and has been fine-tuned on a specially crafted dataset to enable timestamp generation.
+
+Use the model with Hugging Face's `transformers` as follows:
+
+```py
+from transformers import pipeline
+import torch
+
+MODEL_NAME = "biodatlab/whisper-th-medium-timestamp"  # specify the model name
+lang = "th"  # Thai language
+
+device = 0 if torch.cuda.is_available() else "cpu"
+
+pipe = pipeline(
+    task="automatic-speech-recognition",
+    model=MODEL_NAME,
+    chunk_length_s=30,
+    device=device,
+    return_timestamps=True,
+)
+pipe.model.config.forced_decoder_ids = pipe.tokenizer.get_decoder_prompt_ids(
+  language=lang,
+  task="transcribe"
+)
+result = pipe("audio.mp3", return_timestamps=True)
+text = result["text"]
+timestamps = result["chunks"]
+```
+
+## Intended uses & limitations
+This model is intended for Thai automatic speech recognition tasks, particularly where timestamp information is required. It can be used for transcribing Thai audio content, creating subtitles, or any application that needs to align text with specific time points in audio.
+The model's performance on speech recognition may be lower compared to non-timestamped versions due to the additional complexity of the task and the pseudo-timestamp generation method used in training.
+## Training and evaluation data
+The model was trained on a custom-created longform dataset derived from the CMKL/Porjai-Thai-voice-dataset-central. The dataset creation process involved the following steps:
+
+- Combining multiple short audio clips from the original dataset into longer audio segments (up to 30 seconds).
+- Adding environmental noises and silences between clips to simulate more realistic speech scenarios.
+- Generating pseudo-timestamps for the combined audio using a Voice Activity Detection (VAD) model (Silero VAD).
+
+This approach allowed us to create a dataset with longer, more diverse audio samples and approximate timestamp information, which is crucial for training a model capable of generating timestamps.
+## Training procedure
+The model was fine-tuned using a custom training script that incorporates the following:
+
+- Mixed precision training (FP16)
+- Gradient accumulation
+- SpecAugment for data augmentation during training
+
+## Training hyperparameters
+The following hyperparameters were used during training:
+
+- learning_rate: 1e-05
+- train_batch_size: 8
+- eval_batch_size: 8
+- gradient_accumulation_steps: 1
+- num_train_iters: ~50000
+- warmup_steps: 500
+- fp16: True
+- optimizer: AdamW
+- lr_scheduler_type: linear
+
+## Framework versions
+
+- Transformers 4.44.2
+- Pytorch 2.4.1
+- Datasets 3.0.0
+- Tokenizers 0.20.0
+
+## Performance and Limitations
+The WER (Word Error Rate) of 15.57 on the Common Voice 13 test set indicates good performance for Thai ASR. However, it's important to note that the timestamp generation model has a lower accuracy compared to the non-timestamped version of the model. This is due to several factors:
+
+- The use of pseudo-timestamps in training data, which are approximations based on VAD rather than precise human annotations.
+- The additional complexity of the timestamp prediction task, which requires the model to learn both transcription and temporal alignment.
+- Potential discrepancies between the VAD-generated timestamps and actual word boundaries in continuous speech.
+
+Users should be aware that while the timestamps provide a general indication of when words or phrases occur in the audio, they may not be as precise as manually annotated timestamps. The model's performance may also vary depending on the acoustic conditions, speaker variability, and the presence of background noise in the input audio.
+## Citation
+If you use this model in your research or applications, please cite it as follows:
+```
+@misc{biodatlab_whisper_th_medium_timestamp,
+    author       = {Atirut Boribalburephan, Zaw Htet Aung, Knot Pipatsrisawat, Titipat Achakulvisut},
+    title        = {Whisper Medium Thai Timestamp: A fine-tuned Whisper model for Thai automatic speech recognition with timestamp generation},
+    year         = 2024,
+    publisher    = {Hugging Face},
+    howpublished = {\url{https://huggingface.co/biodatlab/whisper-th-medium-timestamp}}
+}
+```