ModernBERT-base / tokenizer_config.json

Commit History

Set tokenizer "model_max_length" property to 8192
9e1cd05
verified

NohTow commited on

Update tokenizer: Set lstrip=True for [MASK]
51c9b61

Tom Aarsen commited on

Upload ModernBERT-base model
cb3183f

Tom Aarsen commited on