Pointwise mutual information (Fano, 1961) 4 is one of the most important con-pointwise mutual information cepts in NLP. It is a measure of how often two events x and y occur, compared with what we would expect if they were independent: