Given a pretrained language model and a novel input sentence, we can think of the output of the model as constituting contextual embeddings for each token in the contextual embeddings input. These contextual embeddings can be used as a contextual representation of the meaning of the input token for any task requiring the meaning of word.