That is, the final output of the encoder H enc = h 1 , ..., h t is multiplied by the cross-attention layer's key weights W K and value weights W V , but the output from the prior decoder layer H dec [i−1] is multiplied by the cross-attention layer's query weights W Q : Figure 10 .16 The transformer block for the encoder and the decoder. Each decoder block has an extra cross-attention layer, which uses the output of the final encoder layer H enc = h 1 , ..., h t to produce its key and value vectors.