Where can I get the tokenizer.json file?

#2
by badrabbitt - opened

I don't see you releasing the tokenizer file with the repo, so please show me how to get this file.

NVIDIA org

The tokenizer is uploaded in each repository under the filename mt_nlg_plus_multilingual_ja_zh_the_stack_frac_015_256k.model.

rwaleffe changed discussion status to closed

Sign up or log in to comment