Novel-Kaguya / added_tokens.json
kichan05's picture
Upload tokenizer
a148882
raw
history blame
95 Bytes
{
"<|endoftext|>": 50256,
"<||bos||>": 50257,
"<||pad||>": 50258,
"<||unk||>": 50259
}