S288C_WordPiece_Tokenizer_k6 / tokenizer_config.json
zluvolyote's picture
add tokenizer
4d0dfdd
raw
history blame contribute delete
51 Bytes
{
"tokenizer_class": "PreTrainedTokenizerFast"
}