There are many kinds of static embeddings. An extension of word2vec, fasttext fasttext (Bojanowski et al., 2017) , addresses a problem with word2vec as we have presented it so far: it has no good way to deal with unknown words -words that appear in a test corpus but were unseen in the training corpus. A related problem is word sparsity, such as in languages with rich morphology, where some of the many forms for each noun and verb may only occur rarely. Fasttext deals with these problems by using subword models, representing each word as itself plus a bag of constituent n-grams, with special boundary symbols < and > added to each word. For example, 6.9 • VISUALIZING EMBEDDINGS 119 with n = 3 the word where would be represented by the sequence <where> plus the character n-grams: <wh, whe, her, ere, re>