fix-tokenizer
#12
by
dbarker9
- opened
No description provided.
Description
Currently, loading the tokenizer displays a warning when loading:
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
This PR updates tokenizer.json using the workflow here to fix the issue.
dbarker9
changed pull request status to
open