Update README.md

README.md

@@ -1,320 +1,8 @@
-<h1 align="center">
-  <a href="https://applio.org" target="_blank"><img src="https://github.com/IAHispano/Applio/assets/133521603/78e975d8-b07f-47ba-ab23-5a31592f322a" alt="Applio"></a>
-</h1>
-
-<p align="center">
-    <img alt="Contributors" src="https://img.shields.io/github/contributors/iahispano/applio?style=for-the-badge&color=FFFFFF" />
-    <img alt="Release" src="https://img.shields.io/github/release/iahispano/applio?style=for-the-badge&color=FFFFFF" />
-    <img alt="Stars" src="https://img.shields.io/github/stars/iahispano/applio?style=for-the-badge&color=FFFFFF" />
-    <img alt="Fork" src="https://img.shields.io/github/forks/iahispano/applio?style=for-the-badge&color=FFFFFF" />
-    <img alt="Issues" src="https://img.shields.io/github/issues/iahispano/applio?style=for-the-badge&color=FFFFFF" />
-</p>
-  
-<p align="center">VITS-based Voice Conversion focused on simplicity, quality, and performance.</p>
-
-<p align="center">
-  <a href="https://applio.org" target="_blank">🌐 Website</a>
-  •
-  <a href="https://docs.applio.org" target="_blank">📚 Documentation</a>
-  •
-  <a href="https://discord.gg/iahispano" target="_blank">☎️ Discord</a>
-</p>
-
-<p align="center">
-  <a href="https://github.com/IAHispano/Applio-Plugins" target="_blank">🛒 Plugins</a>
-  •
-  <a href="https://huggingface.co/IAHispano/Applio/tree/main/Compiled" target="_blank">📦 Compiled</a>
-  •
-  <a href="https://applio.org/playground" target="_blank">🎮 Playground</a>
-  •
-  <a href="https://colab.research.google.com/github/iahispano/applio/blob/master/assets/Applio.ipynb" target="_blank">🔎 Google Colab (UI)</a>
-  •
-  <a href="https://colab.research.google.com/github/iahispano/applio/blob/master/assets/Applio_NoUI.ipynb" target="_blank">🔎 Google Colab (No UI)</a>
-</p>
-
-## Table of Contents
-
-- [Installation](#installation)
-  - [Windows](#windows)
-  - [macOS](#macos)
-  - [Linux](#linux)
-  - [Makefile](#makefile)
-- [Usage](#usage)
-  - [Windows](#windows-1)
-  - [macOS](#macos-1)
-  - [Linux](#linux-1)
-  - [Makefile](#makefile-1)
-- [Technical Information](#technical-information)
-- [Repository Enhancements](#repository-enhancements)
-- [Commercial Usage](#commercial-usage)
-- [References](#references)
-  - [Contributors](#contributors)
-
-## Installation
-
-Download the latest version from [GitHub Releases](https://github.com/IAHispano/Applio-RVC-Fork/releases) or use the [Compiled Versions](https://huggingface.co/IAHispano/Applio/tree/main/Compiled).
-
-### Windows
-
-```bash
-./run-install.bat
-```
-
-### macOS
-
-For macOS, you need to install the requirements in a Python environment version 3.9 to 3.11. Here are the steps:
-
-```bash
-python3 -m venv .venv
-source .venv/bin/activate
-chmod +x run-install.sh
-./run-install.sh
-```
-
-### Linux
-
-Certain Linux-based operating systems may encounter complications with the installer. In such instances, we suggest installing the `requirements.txt` within a Python environment version 3.9 to 3.11.
-
-```bash
-chmod +x run-install.sh
-./run-install.sh
-```
-
-### Makefile
-
-For platforms such as [Paperspace](https://www.paperspace.com/):
-
-```bash
-make run-install
-```
-
-## Usage
-
-Visit [Applio Documentation](https://docs.applio.org/) for a detailed UI usage explanation.
-
-### Windows
-
-```bash
-./run-applio.bat
-```
-
-### macOS
-
-```bash
-chmod +x run-applio.sh
-./run-applio.sh
-```
-
-### Linux
-
-```bash
-chmod +x run-applio.sh
-./run-applio.sh
-```
-
-### Makefile
-
-For platforms such as [Paperspace](https://www.paperspace.com/):
-
-```bash
-make run-applio
-```
-
-## Technical Information
-
-Applio uses an enhanced version of the Retrieval-based Voice Conversion (RVC) model, a powerful technique for transforming the voice of an audio signal to sound like another person. This advanced implementation of RVC in Applio enables high-quality voice conversion while maintaining simplicity and performance.
-
-### 0. Pre-Learning: Key Concepts in Speech Processing and Voice Conversion
-
-This section introduces fundamental concepts in speech processing and voice conversion, paving the way for a deeper understanding of the RVC pipeline:
-
-#### 1. Speech Representation
-
-- **Phoneme:** The smallest unit of sound in a language that distinguishes one word from another. Examples: /k/, /æ/, /t/.
-- **Spectrogram:** A visual representation of the frequency content of a sound over time, showing how the intensity of different frequencies changes over the duration of the audio.
-- **Mel-Spectrogram:** A type of spectrogram that mimics human auditory perception, emphasizing frequencies that are more important to human hearing.
-- **Speaker Embedding:** A vector representation that captures the unique acoustic characteristics of a speaker's voice, encoding information about pitch, tone, timbre, and other vocal qualities.
-
-#### 2. Text-to-Speech (TTS)
-
-- **TTS Model:** A machine learning model that generates artificial speech from written text.
-- **Encoder-Decoder Architecture:** A common architecture in TTS models, where an encoder processes the text and pitch information to create a latent representation, and a decoder uses this representation to synthesize the audio signal.
-- **Transformer Architecture:** A powerful neural network architecture particularly well-suited for sequence modeling, allowing the model to handle long sequences of text or audio and capture relationships between elements.
-
-#### 3. Voice Conversion
-
-- **Voice Conversion (VC):** The process of transforming the voice of a speaker in an audio signal to sound like another speaker.
-- **Speaker Adaptation:** The process of adapting a TTS model to a specific speaker, often by training on a small dataset of the speaker's voice.
-- **Retrieval-Based VC (RVC):** A voice conversion approach where speaker embeddings are retrieved from a database and used to guide the TTS model in synthesizing audio with the target speaker's voice.
-
-#### 4. Additional Concepts
-
-- **ContentVec:** A powerful self-supervised learning model for speech representation, excelling at capturing speaker-specific information.
-- **FAISS:** A library for efficient similarity search, used to retrieve speaker embeddings that are similar to the extracted ContentVec embedding.
-- **Neural Source Filter (NSF):** A module that models audio generation as a filtering process, allowing the model to produce high-quality and realistic audio signals by learning complex relationships between the source signal and the output waveform.
-
-#### 5. Why are these concepts important?
-
-Understanding these concepts is essential for appreciating the mechanics and capabilities of the RVC pipeline:
-
-- **Speech Representation:** Different representations capture different aspects of speech, allowing for effective analysis and manipulation.
-- **TTS Models:** The TTS model forms the foundation of RVC, providing the ability to synthesize audio from text and pitch.
-- **Voice Conversion:** Voice conversion aims to transfer a speaker's identity to a different audio signal.
-- **ContentVec and Speaker Embeddings:** ContentVec provides a powerful way to extract speaker-specific information, which is crucial for accurate voice conversion.
-- **FAISS:** This library enables efficient speaker embedding retrieval, facilitating the selection of appropriate target voices.
-- **NSF:** The NSF is a critical component of the TTS model, contributing to the generation of realistic and high-quality audio.
-
-### 1. Model Architecture
-
-The RVC model comprises two main components:
-
-#### A. Encoder-Decoder Network
-
-This network synthesizes audio based on text and pitch information while incorporating speaker characteristics from the ContentVec embedding.
-
-**Encoder:**
-
-- **Input:** Phoneme sequences (text representation) and pitch information (optional).
-- **Embeddings:**
-  - Phonemes are represented as vectors using linear layers, creating a dense representation of the text input.
-  - Pitch is usually converted to a one-hot encoding or a continuous value and embedded similarly.
-- **Transformer Encoder:** Processes the embedded features in a highly parallel manner.
-
-  It employs:
-
-  - **Self-Attention:** Allows the encoder to attend to different parts of the input sequence to understand the relationships between words and their context.
-  - **Feedforward Networks (FFN):** Apply non-linear transformations to further refine the features captured by self-attention.
-  - **Layer Normalization:** Stabilizes training and improves performance by normalizing the outputs of each layer.
-  - **Dropout:** A regularization technique to prevent overfitting.
-  - **Output:** Produces a latent representation of the input text and pitch, capturing their relationships and serving as the input for the decoder.
-
-**Decoder:**
-
-- **Input:** The latent representation from the encoder.
-- **Transformer Decoder:** Receives the encoder output and utilizes:
-  - **Self-Attention:** Allows the decoder to attend to different parts of the generated sequence to maintain consistency and coherence in the output audio.
-  - **Encoder-Decoder Attention:** Enables the decoder to incorporate information from the input text and pitch into the audio generation process.
-- **Neural Source Filter (NSF):** A powerful component for generating audio, modeling the generation process as a filter applied to a source signal. It uses:
-  - **Upsampling:** Increases the resolution of the latent representation to match the desired length of the audio signal.
-  - **Residual Blocks:** Learn complex and non-linear relationships between input features and the output audio, contributing to realistic and detailed waveforms.
-  - **Source Module:** Generates the excitation signal (often harmonic) that drives the NSF. It combines sine waves (for voiced sounds) and noise (for unvoiced sounds) to create a natural source signal.
-  - **Noise Convolution:** Convolves noise with the harmonic signal to introduce additional variation and realism.
-  - **Final Convolutional Layer:** Converts the filtered output to a single-channel audio waveform.
-- **Output:** Synthesized audio signal.
-
-#### B. ContentVec Speaker Embedding Extractor
-
-Extracts speaker-specific information from the input audio.
-
-- **Input:** The preprocessed audio signal.
-- **Processing:** The ContentVec model, trained on a massive dataset of speech data, processes the input audio and extracts a speaker embedding vector, capturing the unique acoustic properties of the speaker's voice.
-- **Output:** A speaker embedding vector representing the voice of the speaker.
-
-### 2. Training Stage
-
-The RVC model is trained using a combination of two key losses:
-
-- **Generative Loss:**
-  - **Mel-Spectrogram:** The Mel-spectrogram is computed for both the target audio and the generated audio.
-  - **L1 Loss:** Measures the absolute difference between the Mel-spectrograms of the target and generated audio, encouraging the decoder to produce audio with a similar spectral profile.
-- **Discriminative Loss:**
-  - **Multi-Period Discriminator:** Tries to distinguish between real and generated audio at different time scales, using convolution layers to capture long-term dependencies in the audio.
-  - **Adversarial Training:** The generator tries to fool the discriminator by producing audio that sounds real, while the discriminator is trained to correctly identify generated audio.
-- **Optional KL Divergence Loss:** Measures the difference between the distributions of latent variables generated by the encoder and a posterior encoder (which infers the latent representation from the target audio). Encourages the model to learn a more efficient and stable latent representation.
-
-### 3. Inference Stage
-
-The inference stage utilizes the trained model to convert the voice of an audio input to sound like a target speaker. Here's a breakdown:
-
-**Input:**
-
-- Phoneme sequences (text representation).
-- Pitch information (optional).
-- Target speaker ID (identifies the desired voice).
-
-**Steps:**
-
-- **ContentVec Embedding Extraction:**
-  - The ContentVec model processes the input audio and extracts a speaker embedding vector, capturing the voice characteristics of the speaker.
-- **Optional Embedding Retrieval:**
-  - **FAISS Index:** Used to efficiently search for speaker embeddings similar to the extracted ContentVec embedding. It helps guide the voice conversion process toward a specific speaker when multiple speakers are available.
-  - **Embedding Retrieval:** The FAISS index is queried using the extracted ContentVec embedding, and similar embeddings are retrieved.
-- **Embedding Manipulation:**
-  - **Blending:** The extracted ContentVec embedding can be blended with retrieved embeddings using the index_rate parameter, allowing control over how much the target speaker's voice influences the conversion.
-- **Encoder-Decoder Processing:**
-  - **Encoder:** Encodes the phoneme sequences and pitch into a latent representation, capturing the relationships between them.
-  - **Decoder:** Synthesizes the audio signal, incorporating the speaker characteristics from the ContentVec embedding (potentially blended with retrieved embeddings).
-- **Post-Processing:**
-  - **Resampling:** Adjusts the sampling rate of the generated audio if needed.
-  - **RMS Adjustment:** Adjusts the volume (RMS) of the output audio to match the input audio.
-
-### 4. Key Techniques
-
-- **Transformer Architecture:** The Transformer architecture is a powerful tool for sequence modeling, enabling the encoder and decoder to efficiently process long sequences and capture complex relationships within the data.
-- **Neural Source Filter (NSF):** Models audio generation as a filtering process, allowing the model to produce high-quality and realistic audio signals by learning complex relationships between the source signal and the output waveform.
-- **Flow-Based Generative Model:** Enables the model to learn complex probability distributions for the audio signal, leading to more realistic and diverse generated speech.
-- **Multi-period Discriminator:** Helps improve the quality and realism of the generated audio by evaluating the audio at
-
-different temporal scales and providing feedback to the generator.
-
-- **Relative Positional Encoding:** Helps the model understand the relative positions of elements within the input sequences, improving the model's ability to handle long sequences and maintain context.
-
-### 5. Future Challenges
-
-Despite the advancements in Retrieval-Based Voice Conversion, several challenges and areas for future research remain:
-
-- **Speaker Generalization:** Improving the ability of models to generalize to unseen speakers with minimal data.
-- **Real-time Processing:** Enhancing the efficiency of models to support real-time voice conversion applications.
-- **Emotional Expression:** Better capturing and transferring emotional nuances in voice conversion.
-- **Noise Robustness:** Improving the robustness of voice conversion models to handle noisy and low-quality input audio.
-
-## Repository Enhancements
-
-This repository has undergone significant enhancements to improve its functionality and maintainability:
-
-- **Modular Codebase:** Restructured codebase for better organization, readability, and maintenance.
-- **Hop Length Implementation:** Improved efficiency and performance, especially on Crepe (formerly Mangio-Crepe), thanks to [@Mangio621](https://github.com/Mangio621/Mangio-RVC-Fork).
-- **Translations in 30+ Languages:** Added support for over 30 languages.
-- **Cross-Platform Compatibility:** Ensured seamless operation across various platforms.
-- **Optimized Requirements:** Fine-tuned project requirements for enhanced performance.
-- **Streamlined Installation:** Simplified installation process for a user-friendly setup.
-- **Hybrid F0 Estimation:** Introduced a personalized 'hybrid' F0 estimation method utilizing nanmedian.
-- **Easy-to-Use UI:** Implemented an intuitive user interface.
-- **Plugin System:** Introduced a plugin system for extending functionality.
-- **Overtraining Detector:** Implemented a detector to prevent excessive training.
-- **Model Search:** Integrated model search feature for easy discovery.
-- **Pretrained Models:** Added support for custom pretrained models.
-- **Voice Blender:** Developed a feature to combine two trained models to create a new one.
-- **Accessibility Improvements:** Enhanced with descriptive tooltips for UI elements.
-- **New F0 Extraction Methods:** Introduced methods like FCPE or Hybrid for pitch extraction.
-- **Output Format Selection:** Added feature to choose audio file formats.
-- **Hashing System:** Assigned unique IDs to models to prevent unauthorized duplication.
-- **Model Download System:** Supported downloads from various platforms.
-- **TTS Enhancements:** Improved Text-to-Speech functionality.
-- **Split Audio:** Implemented audio splitting for faster processing.
-- **Discord Presence:** Displayed usage status on Discord.
-- **Flask Integration:** Enabled automatic model downloads via Flask.
-- **Support Tab:** Added a tab for screen recording to report issues.
-
-These enhancements contribute to a more robust and scalable codebase, making the repository more accessible for contributors and users alike.
-
-## Commercial Usage
-
-For commercial purposes, please adhere to the guidelines outlined in the [MIT license](./LICENSE) governing this project. Prior to integrating Applio into your application, we kindly request that you contact us at [email protected] to ensure ethical use.
-
-Please note, the use of Applio-generated audio files falls under your own responsibility and must always respect applicable copyrights. We encourage you to consider supporting the continuous development and maintenance of Applio through a donation. 
-
-Your cooperation and support are greatly appreciated. Thank you!
-
-## References
-
-Applio is possible to these projects and those cited in their references.
-
-- [gradio-screen-recorder](https://huggingface.co/spaces/gstaff/gradio-screen-recorder) by gstaff
-- [rvc-cli](https://github.com/blaise-tk/rvc-cli) by blaisewf
-
-### Contributors
-
-<a href="https://github.com/IAHispano/Applio/graphs/contributors" target="_blank">
-  <img src="https://contrib.rocks/image?repo=IAHispano/Applio" />
-</a>
+---
+license: mit
+title: Applio Full GPU
+sdk: gradio
+emoji: 🗣️
+colorFrom: blue
+colorTo: blue
+---