riffusion

/

riffusion-model-v1

like 576

Follow

Riffusion 33

(I asked this question over email, but I'll also subscribe to answers here)

This file seems to contain list of tokens: https://huggingface.co./riffusion/riffusion-model-v1/raw/main/tokenizer/vocab.json

Tokenizers usually give lower number to tokens that are more common in dataset. So, "latin" has id 9888, and Mozart has id 22132, which means that latin should be more likely to produce good results.

+1 Subscribing for when they respond.

i guarantee its copyrighted music lol

I'm more interested in the amount of data that they used in order to do this fine-tuning.

@Phoeo I don't think they fine-tuned the text encoder - so the vocab.json is the original CLIP ViT-L/14 vocabulary.

i guarantee its copyrighted music lol

I don't think so

i guarantee its copyrighted music lol

I don't think so

I would like to believe it's not. But the total lack of transparency or even a mention of what data was used to train this model suggests that it was done in an unethical way.

i guarantee its copyrighted music lol

I don't think so

I would like to believe it's not. But the total lack of transparency or even a mention of what data was used to train this model suggests that it was done in an unethical way.

That's like saying if something isn't open source, they are selling your data for profit.

Was my commented removed? Understandable anyways. Thanks for the work on this.

Additional fine tuning and data information has been added to the model-card. This was trained using approaches similar to hugging face examples, but fine-tuning can be achieved with very small datasets using a dreambooth approach.

I'm still confused. The model card still doesn't say what it was fine tuned on. Was it trained on LAION-AI/audio-dataset or are you just mentioning it to mention it?