Update model and tokenizer
#1
by
waveletdeboshir
- opened
Now there are no timestamp and language tokens (except ru and en) in tokenizer, merges.txt is modified.
Instead of timestamps tokens I added more Russian tokens.
waveletdeboshir
changed pull request status to
merged