transformer-lm-japanese-0.1b / added_tokens.json
fukugawa's picture
Upload tokenizer
01799ff verified
raw
history blame
76 Bytes
{
"<pad>": 30002,
"[CLS]": 30000,
"[MASK]": 30003,
"[SEP]": 30001
}