Update with HF Implementation (#2)

- Update README.md with HF implementation (5ba3c8ae692e3d8153d42883111a252c55d34bdc)

README.md

@@ -6,9 +6,9 @@ license: cc-by-nc-4.0
 
 # MusicGen - Medium - 1.5B
 
-Audiocraft provides the code and models for MusicGen, a simple and controllable model for music generation. 
-MusicGen is a single stage auto-regressive Transformer model trained over a 32kHz EnCodec tokenizer with 4 codebooks sampled at 50 Hz. 
-Unlike existing methods like MusicLM, MusicGen doesn't not require a self-supervised semantic representation, and it generates all 4 codebooks in one pass. 
+MusicGen is a text-to-music model capable of genreating high-quality music samples conditioned on text descriptions or audio prompts.
+It is a single stage auto-regressive Transformer model trained over a 32kHz EnCodec tokenizer with 4 codebooks sampled at 50 Hz. 
+Unlike existing methods, like MusicLM, MusicGen doesn't require a self-supervised semantic representation, and it generates all 4 codebooks in one pass. 
 By introducing a small delay between the codebooks, we show we can predict them in parallel, thus having only 50 auto-regressive steps per second of audio.
 
 MusicGen was published in [Simple and Controllable Music Generation](https://arxiv.org/abs/2306.05284) by *Jade Copet, Felix Kreuk, Itai Gat, Tal Remez, David Kant, Gabriel Synnaeve, Yossi Adi, Alexandre Défossez*.
@@ -23,15 +23,75 @@ Four checkpoints are released:
 
 Try out MusicGen yourself!
 
-- <a target="_blank" href="https://colab.research.google.com/drive/1fxGqfg96RBUvGxZ1XXN07s3DthrKUl4-?usp=sharing">
+* Audiocraft Colab:
+
+<a target="_blank" href="https://colab.research.google.com/drive/1fxGqfg96RBUvGxZ1XXN07s3DthrKUl4-?usp=sharing">
   <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/>
 </a>
 
-- <a target="_blank" href="https://huggingface.co/spaces/facebook/MusicGen">
-  <img src="https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm.svg" alt="Open in HugginFace"/>
+* Hugging Face Colab:
+
+<a target="_blank" href="https://colab.research.google.com/github/sanchit-gandhi/notebooks/blob/main/MusicGen.ipynb"> 
+  <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open In Colab"/> 
+</a>
+
+* Hugging Face Demo:
+
+<a target="_blank" href="https://huggingface.co/spaces/facebook/MusicGen">
+  <img src="https://huggingface.co/datasets/huggingface/badges/raw/main/open-in-hf-spaces-sm.svg" alt="Open in HuggingFace"/>
 </a>
 
-- You can run MusicGen locally as well:
+## 🤗 Transformers Usage
+
+You can run MusicGen locally with the 🤗 Transformers library from version 4.31.0 onwards.
+
+1. First install the 🤗 [Transformers library](https://github.com/huggingface/transformers) from main:
+
+```
+pip install git+https://github.com/huggingface/transformers.git
+```
+
+2. Run the following Python code to generate text-conditional audio samples:
+
+```py
+from transformers import AutoProcessor, MusicgenForConditionalGeneration
+
+
+processor = AutoProcessor.from_pretrained("facebook/musicgen-medium")
+model = MusicgenForConditionalGeneration.from_pretrained("facebook/musicgen-medium")
+
+inputs = processor(
+    text=["80s pop track with bassy drums and synth", "90s rock song with loud guitars and heavy drums"],
+    padding=True,
+    return_tensors="pt",
+)
+
+audio_values = model.generate(**inputs, do_sample=True, guidance_scale=3, max_new_tokens=256)
+```
+
+3. Listen to the audio samples either in an ipynb notebook:
+
+```py
+from IPython.display import Audio
+
+sampling_rate = model.config.audio_encoder.sampling_rate
+Audio(audio_values[0].numpy(), rate=sampling_rate)
+```
+
+Or save them as a `.wav` file using a third-party library, e.g. `scipy`:
+
+```py
+import scipy
+
+sampling_rate = model.config.audio_encoder.sampling_rate
+scipy.io.wavfile.write("musicgen_out.wav", rate=sampling_rate, data=audio_values[0, 0].numpy())
+```
+
+For more details on using the MusicGen model for inference using the 🤗 Transformers library, refer to the [MusicGen docs](https://huggingface.co/docs/transformers/model_doc/musicgen).
+
+## Audiocraft Usage
+
+You can also run MusicGen locally through the original [Audiocraft library]((https://github.com/facebookresearch/audiocraft):
 
 1. First install the [`audiocraft` library](https://github.com/facebookresearch/audiocraft)
 ```
@@ -46,16 +106,15 @@ apt get install ffmpeg
 3. Run the following Python code:
 
 ```py
-import torchaudio
 from audiocraft.models import MusicGen
 from audiocraft.data.audio import audio_write
 
-model = MusicGen.get_pretrained('medium')
+model = MusicGen.get_pretrained("medium")
 model.set_generation_params(duration=8)  # generate 8 seconds.
 
-descriptions = ['happy rock', 'energetic EDM', 'sad jazz']
+descriptions = ["happy rock", "energetic EDM"]
 
-wav = model.generate(descriptions)  # generates 3 samples.
+wav = model.generate(descriptions)  # generates 2 samples.
 
 for idx, one_wav in enumerate(wav):
     # Will save under {idx}.wav, with loudness normalization at -14 db LUFS.