gbenson
/

dom-tokenizer-10k

gbenson commited on May 17

Commit

85a0b70

•

1 Parent(s): 10f6ed4

Switch to cased base model

Files changed (3) hide show

tokenizer.json CHANGED Viewed

The diff for this file is too large to render. See raw diff

tokenizer_config.json CHANGED Viewed

@@ -109,7 +109,7 @@
   "bos_token": "[BOS]",
   "clean_up_tokenization_spaces": true,
   "cls_token": "[CLS]",
-  "do_lower_case": true,
   "eos_token": "[EOS]",
   "mask_token": "[MASK]",
   "model_max_length": 512,

   "bos_token": "[BOS]",
   "clean_up_tokenization_spaces": true,
   "cls_token": "[CLS]",
+  "do_lower_case": false,
   "eos_token": "[EOS]",
   "mask_token": "[MASK]",
   "model_max_length": 512,

vocab.txt CHANGED Viewed

The diff for this file is too large to render. See raw diff