Update README.md

README.md

@@ -31,7 +31,6 @@ datasets:
 
 <img src="https://huggingface.co/datasets/parler-tts/images/resolve/main/thumbnail.png" alt="Parler Logo" width="800" style="margin-left:'auto' margin-right:'auto' display:'block'"/>
 
-
 # Indic Parler-TTS
 
 <a target="_blank" href="https://huggingface.co/spaces/PHBJT/multi_parler_tts">
@@ -51,17 +50,15 @@ Thanks to its **better prompt tokenizer**, it can easily be extended to other la
 
 ## 📖 Quick Index
 * [👨‍💻 Installation](#👨‍💻-installation)
+* [🛠️ Key capabilities](#🛠️-key-capabilities)
 * [🎲 Using a random voice](#🎲-random-voice)
 * [🌍 Switching languages](#[🌍-switching-languages)
 * [🎯 Using a specific speaker](#🎯-using-a-specific-speaker)
+* [Some Description Examples](#some-description-examples)
 * [📐Evaluation](#📐-evaluation)
 * [Motivation](#motivation)
 * [Optimizing inference](https://github.com/huggingface/parler-tts/blob/main/INFERENCE.md)
 
-## 🛠️ Usage
-
-🚨 Unlike previous versions of Parler-TTS, here we use two tokenizers - one for the prompt and one for the description. 🚨
-
 ### 👨‍💻 Installation
 
 Using Parler-TTS is as simple as "bonjour". Simply install the library once:
@@ -70,9 +67,45 @@ Using Parler-TTS is as simple as "bonjour". Simply install the library once:
 pip install git+https://github.com/huggingface/parler-tts.git
 ```
 
+## 🛠️ Key capabilities
+
+The model accepts two primary inputs:  
+1. **Transcript** - The text to be converted to speech.  
+2. **Caption** - A detailed description of how the speech should sound, e.g., "Leela speaks in a high-pitched, fast-paced, and cheerful tone, full of energy and happiness. The recording is very high quality with no background noise."  
+
+### Key Features  
+
+1. **Language Support**  
+   - **Officially supported languages**: Assamese, Bengali, Bodo, Dogri, Kannada, Malayalam, Marathi, Sanskrit, Nepali, English, Telugu, Hindi, Gujarati, Konkani, Maithili, Manipuri, Odia, Santali, Sindhi, Tamil, and Urdu.  
+   - **Unofficial support**: Chhattisgarhi, Kashmiri, Punjabi.  
+
+2. **Speaker Diversity**  
+   - **69 unique voices** across the supported languages.  
+   - Supported languages have a set of **recommended voices** optimized for naturalness and intelligibility.  
+   
+3. **Emotion Rendering**  
+   - **10 languages** officially support emotion-specific prompts: Assamese, Bengali, Bodo, Dogri, Kannada, Malayalam, Marathi, Sanskrit, Nepali, and Tamil.  
+   - Emotion support for other languages exists but has not been extensively tested.  
+   - **Available emotions** include: Command, Anger, Narration, Conversation, Disgust, Fear, Happy, Neutral, Proper Noun, News, Sad, and Surprise.  
 
-### 🎲 Random voice
+4. **Accent Flexibility**  
+   - The model **officially supports Indian English accents** through its English voices, providing clear and natural speech.  
+   - For other accents, the model allows customization by specifying accent details, such as "A male British speaker" or "A female American speaker," using style transfer for more dynamic and personalized outputs.  
 
+5. **Customizable Output**  
+   IndicParlerTTS offers precise control over various speech characteristics using the **caption** input:  
+
+   - **Background Noise**: Adjust the noise level in the audio, from clear to slightly noisy environments.  
+   - **Reverberation**: Control the perceived distance of the voice, from close-sounding to distant-sounding speech.  
+   - **Expressivity**: Specify how dynamic or monotone the speech should be, ranging from expressive to slightly expressive or monotone.  
+   - **Pitch**: Modify the pitch of the speech, including high, low, or balanced tones.  
+   - **Speaking Rate**: Change the speaking rate, from slow to fast.  
+   - **Voice Quality**: Control the overall clarity and naturalness of the speech, adjusting from basic to refined voice quality.
+
+
+## 🎲 Random voice
+
+🚨 Unlike previous versions of Parler-TTS, here we use two tokenizers - one for the prompt and one for the description. 🚨
 
 **Parler-TTS** has been trained to generate speech with features that can be controlled with a simple text prompt, for example:
 
@@ -99,7 +132,19 @@ audio_arr = generation.cpu().numpy().squeeze()
 sf.write("indic_tts_out.wav", audio_arr, model.config.sampling_rate)
 ```
 
-### 🌍 Switching languages
+IndicParlerTTS provides highly effective control over key aspects of speech synthesis using descriptive captions. Below is a summary of what each control parameter can achieve:
+
+| **Control Type**        | **Capabilities**                                                                 |
+|--------------------------|----------------------------------------------------------------------------------|
+| **Background Noise**     | Adjusts the level of background noise, supporting clear and slightly noisy environments. |
+| **Reverberation**        | Controls the perceived distance of the speaker’s voice, allowing close or distant sounds. |
+| **Expressivity**         | Modulates the emotional intensity of speech, from monotone to highly expressive. |
+| **Pitch**                | Varies the pitch to achieve high, low, or moderate tonal output.                |
+| **Speaking Rate**        | Changes the speed of speech delivery, ranging from slow to fast-paced.          |
+| **Speech Quality**       | Improves or degrades the overall audio clarity, supporting basic to refined outputs. |
+
+
+## 🌍 Switching languages
 
 The template automatically adapts to the language it detects in the prompt. You don't need to specify the language you want to use. For example, to switch to Hindi, simply use an Hindi prompt:
 
@@ -126,7 +171,7 @@ audio_arr = generation.cpu().numpy().squeeze()
 sf.write("indic_tts_out.wav", audio_arr, model.config.sampling_rate)
 ```
 
-### 🎯 Using a specific speaker
+## 🎯 Using a specific speaker
 
 To ensure speaker consistency across generations, this checkpoint was also trained on pre-determined speakers, characterized by name (e.g. Rohit, Karan, Leela, Maya, Sita, ...).
 To take advantage of this, simply adapt your text description to specify which speaker to use: `Divya's voice is monotone yet slightly fast in delivery, with a very close recording that almost has no background noise.`
@@ -153,34 +198,30 @@ audio_arr = generation.cpu().numpy().squeeze()
 sf.write("parler_tts_out.wav", audio_arr, model.config.sampling_rate)
 ```
 
-Here is the list of speakers:
-
-| Language      | Male     | Female   |
-|---------------|----------|----------|
-| Assamese      | Amit     | Aditi    |
-| Bengali       | Arjun    | Sita     |
-| Bodo          | Bikram   | Maya     |
-| Chhattisgarhi | Bhanu    | Champa   |
-| Dogri         | Karan    | Leela    |
-| English       | Thoma    | Mary     |
-| Gujarati      | Yash     | Neha     |
-| Hindi         | Rohit    | Divya    |
-| Kannada       | Suresh   | Anu      |
-| Kashmiri      | Rohit    | Shabnam  |
-| Konkani       | Deepak   | Sunita   |
-| Maithili      | Anil     | Divya    |
-| Malayalam     | Harish   | Anjali   |
-| Manipuri      | Ranjit   | Laishram |
-| Marathi       | Sanjay   | Sunita   |
-| Nepali        | Ram      | Amrita   |
-| Odia          | Manas    | Debjani  |
-| Punjabi       | Gurpreet | Divjot   |
-| Sanskrit      | Aryan    | Vasudha  |
-| Santali       | Arjun    | Pushpa   |
-| Sindhi        | Rajesh   | Rekha    |
-| Tamil         | NA       | Jaya     |
-| Telugu        | Prakash  | Lalitha  |
-| Urdu          | Rohit    | Zainab   |
+The model includes **69 speakers** across 18 officially supported languages, with each language having a set of recommended voices for optimal performance. Below is a table summarizing the available speakers for each language, along with the recommended ones.
+
+Here is the table based on the provided data:
+
+| **Language**      | **Available Speakers**                                       | **Recommended Speakers**       |
+|--------------------|-------------------------------------------------------------|---------------------------------|
+| Assamese          | Amit, Sita, Poonam, Rakesh                                  | Amit, Sita                     |
+| Bengali           | Arjun, Aditi, Tapan, Rashmi, Arnav, Riya                    | Arjun, Aditi                   |
+| Bodo              | Bikram, Maya, Kalpana                                      | Bikram, Maya                   |
+| Chhattisgarhi     | Bhanu, Champa                                              | Bhanu, Champa                  |
+| Dogri             | Karan                                                      | Karan                          |
+| English           | Thoma, Mary, Swapna, Dinesh, Meera, Jatin, Aakash, Sneha, Kabir, Tisha, Chingkhei, Thoiba, Priya, Tarun, Gauri, Nisha, Raghav, Kavya, Ravi, Vikas, Riya | Thoma, Mary                     |
+| Gujarati          | Yash, Neha                                                 | Yash, Neha                     |
+| Hindi             | Rohit, Divya, Aman, Rani                                   | Rohit, Divya                   |
+| Kannada           | Suresh, Anu, Chetan, Vidya                                 | Suresh, Anu                    |
+| Malayalam         | Anjali, Anju, Harish                                       | Anjali, Harish                   |
+| Manipuri          | Laishram, Ranjit                                           | Laishram, Ranjit               |
+| Marathi           | Sanjay, Sunita, Nikhil, Radha, Varun, Isha                  | Sanjay, Sunita                 |
+| Nepali            | Amrita                                                     | Amrita                         |
+| Odia              | Manas, Debjani                                             | Manas, Debjani                 |
+| Punjabi           | Divjot, Gurpreet                                           | Divjot, Gurpreet               |
+| Sanskrit          | Aryan                                                      | Aryan                          |
+| Tamil             | Kavitha, Jaya                                        | Jaya                 |
+| Telugu            | Prakash, Lalitha, Kiran                                    | Prakash, Lalitha               |
 
 
 **Tips**:
@@ -190,8 +231,71 @@ Here is the list of speakers:
 * The remaining speech features (gender, speaking rate, pitch and reverberation) can be controlled directly through the prompt
 
 
+## Some Description Examples
+
+1. **Aditi - Slightly High-Pitched, Expressive Tone**:  
+   *"Aditi speaks with a slightly higher pitch in a close-sounding environment. Her voice is clear, with subtle emotional depth and a normal pace, all captured in high-quality recording."*  
+
+2. **Sita - Rapid, Slightly Monotone**:  
+   *"Sita speaks at a fast pace with a slightly low-pitched voice, captured clearly in a close-sounding environment with excellent recording quality."*  
+
+3. **Tapan - Male, Moderate Pace, Slightly Monotone**:  
+   *"Tapan speaks at a moderate pace with a slightly monotone tone. The recording is clear, with a close sound and only minimal ambient noise."*  
+
+4. **Sunita - High-Pitched, Happy Tone**:  
+   *"Sunita speaks with a high pitch in a close environment. Her voice is clear, with slight dynamic changes, and the recording is of excellent quality."*  
+
+5. **Karan - High-Pitched, Positive Tone**:  
+   *"Karan’s high-pitched, engaging voice is captured in a clear, close-sounding recording. His slightly slower delivery conveys a positive tone."*  
+
+6. **Amrita - High-Pitched, Flat Tone**:  
+   *"Amrita speaks with a high pitch at a slow pace. Her voice is clear, with excellent recording quality and only moderate background noise."*  
+
+7. **Aditi - Slow, Slightly Expressive**:  
+   *"Aditi speaks slowly with a high pitch and expressive tone. The recording is clear, showcasing her energetic and emotive voice."*  
+
+8. **Young Male Speaker, American Accent**:  
+   *"A young male speaker with a high-pitched American accent delivers speech at a slightly fast pace in a clear, close-sounding recording."*  
+
+9. **Bikram - High-Pitched, Urgent Tone**:  
+   *"Bikram speaks with a higher pitch and fast pace, conveying urgency. The recording is clear and intimate, with great emotional depth."*  
+
+10. **Anjali - High-Pitched, Neutral Tone**:  
+   *"Anjali speaks with a high pitch at a normal pace in a clear, close-sounding environment. Her neutral tone is captured with excellent audio quality."*  
+
+
 ## 📐 Evaluation
 
+IndicParlerTTS has been evaluated using a MOS-like framework by native and non-native speakers. The results highlight its exceptional performance in generating natural and intelligible speech, especially for native speakers of Indian languages.  
+
+| **Language** | **Native Speaker Score (%)** | **Highlights**                                                                                     |
+|--------------|-------------------------------|--------------------------------------------------------------------------------------------------|
+| Assamese     | 87.36 ± 1.81                 | Clear, natural synthesis with excellent expressiveness.                                          |
+| Bengali      | 86.16 ± 1.85                 | High-quality outputs with smooth intonation.                                                    |
+| Bodo         | 94.47 ± 4.12                 | Near-perfect accuracy for a lesser-resourced language.                                          |
+| Dogri        | 88.80 ± 3.57                 | Robust and consistent synthesis for Dogri.                                                     |
+| Gujarati     | 75.36 ± 1.78                 | Strong clarity and naturalness even for smaller languages.                                      |
+| Hindi        | 84.79 ± 2.09                 | Reliable and expressive outputs for India's most widely spoken language.                       |
+| Kannada      | 88.17 ± 2.81                 | Highly natural and accurate voices for Kannada.                                                |
+| Konkani      | 76.60 ± 4.14                 | Produces clear and natural outputs for diverse speakers.                                       |
+| Maithili     | 95.36 ± 2.52                 | Exceptionally accurate, showcasing fine-tuning success.                                         |
+| Malayalam    | 86.54 ± 1.67                 | Smooth, high-quality synthesis with expressive outputs.                                        |
+| Manipuri     | 85.63 ± 2.60                 | Natural intonation with minimal errors.                                                        |
+| Marathi      | 76.96 ± 1.45                 | Maintains clarity and naturalness across speakers.                                             |
+| Nepali       | 80.02 ± 5.75                 | Strong synthesis for native and proximal Nepali speakers.                                      |
+| Odia         | 88.94 ± 3.26                 | High expressiveness and quality for Odia speakers.                                             |
+| Sanskrit     | 99.79 ± 0.34                 | Near-perfect synthesis, ideal for classical use cases.                                         |
+| Sindhi       | 76.46 ± 1.29                 | Clear and natural voices for underrepresented languages.                                       |
+| Tamil        | 75.48 ± 2.18                 | Delivers intelligible and expressive speech.                                                   |
+| Telugu       | 88.54 ± 1.86                 | Smooth and natural tonal quality for Telugu.                                                   |
+| Urdu         | 77.75 ± 3.82                 | Produces high-quality speech despite resource constraints.                                     |
+
+**Key Strengths**:  
+- Exceptional performance for native speakers, with top scores for **Maithili (95.36)**, **Sanskrit (99.79)**, and **Bodo (94.47)**.  
+- Competitive results for lesser-resourced and unofficially supported languages like **Kashmiri (55.30)** and **Sindhi (76.46)**.  
+- Adaptability to non-native and anonymous speaker scenarios with consistently high clarity.  
+
+
 ## Motivation
 
 Parler-TTS is a reproduction of work from the paper [Natural language guidance of high-fidelity text-to-speech with synthetic annotations](https://www.text-description-to-speech.com) by Dan Lyth and Simon King, from Stability AI and Edinburgh University respectively.