As an example, let's return to the problem of sentiment classification. A simple approach to fine-tuning a classifier for this application involves learning a set of weights, W C , to map the output vector for the [CLS] token, y CLS to a set of scores over the possible sentiment classes. Assuming a three-way sentiment classification task (positive, negative, neutral) and dimensionality d h for the size of the language model hidden layers gives W C ∈ R 3×d h . Classification of unseen documents proceeds by passing the input text through the pretrained language model to generate y CLS , multiplying it by W C , and finally passing the resulting vector through a softmax.