Update tokenizer.json

Fixes a bug with offset tracking: https://github.com/huggingface/transformers/issues/9637

Files changed (1) hide show

tokenizer.json ADDED Viewed

The diff for this file is too large to render. See raw diff