, and these get multiplied by the inputs packed into X to produce Q ∈ R N×d k , K ∈ R N×d k , and V ∈ R N×d v . The output of each of the h heads is of shape N × d v , and so the output of the multi-head layer with h heads consists of h vectors of shape N × d v . To make use of these vectors in further processing, they are combined and then reduced down to the original input dimension d. This is accomplished by concatenating the outputs from each head and then using yet another linear projection, W O ∈ R hd v ×d , to reduce it to the original output dimension for each token, or a total N × d output. Fig. 9 .19 illustrates this approach with 4 self-attention heads. This multihead layer replaces the single self-attention layer in the transformer block shown earlier in Fig. 9 .18, the rest of the transformer block with its feedforward layer, residual connections, and layer norms remains the same.