During training, the output vector from the final layer associated with the [CLS] token represents the next sentence prediction. As with the MLM objective, a learned set of classification weights W NSP ∈ R 2×d h is used to produce a two-class prediction from the raw [CLS] vector.