Total loss is the average cross-entropy loss per target word: Figure 10 .7 Training the basic RNN encoder-decoder approach to machine translation. Note that in the decoder we usually don't propagate the model's softmax outputsŷ t , but use teacher forcing to force each input to the correct gold value for training. We compute the softmax output distribution overŷ in the decoder in order to compute the loss at each token, which can then be averaged to compute a loss for the sentence.