Unfortunately, this process goes a bit too far since the calculation of the comparisons in QK results in a score for each query value to every key value, including those that follow the query. This is inappropriate in the setting of language modeling since guessing the next word is pretty simple if you already know it. To fix this, the elements in the upper-triangular portion of the matrix are zeroed out (set to −∞), thus eliminating any knowledge of words that follow in the sequence. Fig. 9 .17 depicts the QK matrix. (we'll see in Chapter 11 how to make use of words in the future for tasks that need it).