tokenizer
#5
by
nazgut
- opened
Why do you have flan t5 as tokenizer?
tokenizer = T5Tokenizer.from_pretrained("google/flan-t5-small")
That is the tokenizer used for the base model, which is flan-t5-small.
roborovski
changed discussion status to
closed
But it doesn't make any sense, your model is based on flan-t5 so it should/could be used as tokenizer, right?