In this chapter, we expand on this idea in two large ways. First, we'll introduce the idea of contextual embeddings: representations for words in context. The methods of Chapter 6 like word2vec or GloVe learned a single vector embedding for each unique word w in the vocabulary. By contrast, with contextual embeddings, such as those learned by popular methods like BERT (Devlin et al., 2019) or GPT (Radford et al., 2019) or their descendants, each word w will be represented by a different vector each time it appears in a different context.