More formally, let's assume we have a co-occurrence matrix F with W rows (words) and C columns (contexts), where f i j gives the number of times word w i occurs in 4 PMI is based on the mutual information between two random variables X and Y , defined as: