Note the differences between training (Fig. 10.7 ) and inference ( Fig. 10.4 ) with respect to the outputs at each time step. The decoder during inference uses its own estimated outputŷ t as the input for the next time step x t+1 . Thus the decoder will tend to deviate more and more from the gold target sentence as it keeps generating more tokens. In training, therefore, it is more common to use teacher forcing in the teacher forcing decoder. Teacher forcing means that we force the system to use the gold target token from training as the next input x t+1 , rather than allowing it to rely on the (possibly erroneous) decoder outputŷ t . This speeds up training.