Case sensitivity of the tokenizer
#3
by
abhinav-kashyap-asus
- opened
I found that the tokenizer that is provided as part of this model is not case sensitive. That is the lower case and uppercase text, get tokenized in the same way. Is this the case?? Why was this decision taken, since T5 itself seems to be case sensitive.
Thank you
Best,
Abhinav
It should only be the base model. Unfortunately, the reason is not very exciting: bug. Whoops!
:D aaah!! I wish it wasn't. But thanks for the quick response :)
xyla
changed discussion status to
closed