PMI values range from negative to positive infinity. But negative PMI values (which imply things are co-occurring less often than we would expect by chance) tend to be unreliable unless our corpora are enormous. To distinguish whether two words whose individual probability is each 10 −6 occur together less often than chance, we would need to be certain that the probability of the two occurring together is significantly different than 10 −12 , and this kind of granularity would require an enormous corpus. Furthermore it's not clear whether it's even possible to evaluate such scores of 'unrelatedness' with human judgments. For this reason it is more common to use Positive PMI (called PPMI) which replaces all negative PPMI PMI values with zero (Church and Hanks 1989 , Dagan et al. 1993 , Niwa and Nitta 1994 5 :