full code snippets

README.md

@@ -44,6 +44,8 @@ Gengembre, N., Le Blouch, O., Gendrot, C. (2024) Disentangling prosody and timbr
 ```
 
 # Usage
+This first code snippet is for the model creation and download:
+
 ```
 import torch
 import torch.nn as nn
@@ -77,10 +79,46 @@ class EmbeddingsModel(WavLMPreTrainedModel):
         x_stats = torch.cat((base_out.mean(dim=1),v.pow(0.5)),dim=1).unsqueeze(dim=2)
         return self.top_layers(x_stats)
 
-nt_extractor = EmbeddingsModel("ggmbr/wnt")
+nt_extractor = EmbeddingsModel.from_pretrained("ggmbr/wnt")
+nt_extractor.eval()
+```
+
+You may have noticed that the model produces normalized vectors as embeddings.
+Next, we define a function that extracts the non-timbral embedding from an audio signal. In this tutorial version, the audio file is expected to be sampled at 16kHz.
+
+```
+import torchaudio
+
+MAX_SIZE = 320000 # max number of audio samples
+
+def compute_embedding(fnm, model):
+    sig, sr = torchaudio.load(fnm)
+    assert sr == 16000, "please convert your audio file to a sampling rate of 16 kHz"
+    sig = sig.mean(dim=0).to(device)
+    if sig.shape[0] > MAX_SIZE:
+        print(f"truncating long signal {fnm}")
+        sig = sig[:MAX_SIZE]
+    embd = model(sig.unsqueeze(dim=0))
+    return embd.clone().detach()
+```
+
+And finally, we can compute two embeddings from two different files and compare them with a cosine similarity:
+
+```
+wav1 = "/data/AUDIO/speakerid/corpus/voxceleb1_2019/test/wav/id10270/x6uYqmx31kE/00001.wav"
+wav2 = "/data/AUDIO/speakerid/corpus/voxceleb1_2019/test/wav/id10270/8jEAjG6SegY/00008.wav"
+
+e1 = compute_embedding(wav1, nt_extractor)
+e2 = compute_embedding(wav2, nt_extractor)
+sim = float(torch.matmul(e1,e2.t()))
 ```
 
 # Evaluations
+Although it is not directly designed for this use case, evaluation on a standard ASV task can be performed with this model. Applied to 
+the [VoxCeleb1-clean test set](https://www.robots.ox.ac.uk/~vgg/data/voxceleb/meta/veri_test2.txt), it leads to an equal error rate (EER) of **10.681%** 
+(with a decision threshold of **0.467**). This value can be interpreted as the ability to identify speakers only with non-timbral cues. Discussion about this interpretation can be
+found in the paper mentioned hereabove, as well as other experiments showing correlations between these embeddings and non-timbral voice attributes.
+
 
 # Limitations
 The fine tuning data used to produce this model (VoxCeleb, VCTK) are mostly in english, which may affect the performance on other languages.