The result is a model with over 100M parameters. The use of WordPiece (one of the large family of subword tokenization algorithms that includes the BPE algorithm we saw in Chapter 2) means that BERT and its descendants are based on subword tokens rather than words. Every input sentence first has to be tokenized, and then all further processing takes place on subword tokens rather than words. This will require, as we'll see, that for some NLP tasks that require notions of words (like named entity tagging, or parsing) we will occasionally need to map subwords back to words.