PMI has the problem of being biased toward infrequent events; very rare words tend to have very high PMI values. One way to reduce this bias toward low frequency Figure 6 .12 The PPMI matrix showing the association between words and context words, computed from the counts in Fig. 6 .11. Note that most of the 0 PPMI values are ones that had a negative PMI; for example PMI(cherry,computer) = -6.7, meaning that cherry and computer co-occur on Wikipedia less often than we would expect by chance, and with PPMI we replace negative values by zero. events is to slightly change the computation for P(c), using a different function P α (c) that raises the probability of the context word to the power of α: