semeru
/

completeformer_tokenizer

Model card Files Files and versions Community

completeformer_tokenizer / added_tokens.json

ncoop57

Add tokenizer

25f241b over 1 year ago

94 Bytes

	{
	"<BOS>": 32770,
	"<EOS>": 32771,
	"<MASK>": 32768,
	"<PAD>": 32769,
	"<TAB>": 32772
	}