Generally training proceeds by taking as input a very long text, concatenating all the sentences, starting with random weights, and then iteratively moving through the text predicting each word w t . At each word w t , we use the cross-entropy (negative log likelihood) loss. Recall that the general form for this (repeated from Eq. 7.26 is: