Add missing merge to tokenizer

#50
sanchit-gandhi changed pull request status to merged

Sign up or log in to comment