But suppose our test set has phrases like: denied the offer denied the loan Our model will incorrectly estimate that the P(offer|denied the) is 0! These zerosthings that don't ever occur in the training set but do occur in zeros the test set-are a problem for two reasons. First, their presence means we are underestimating the probability of all sorts of words that might occur, which will hurt the performance of any application we want to run on this data. Second, if the probability of any word in the test set is 0, the entire probability of the test set is 0. By definition, perplexity is based on the inverse probability of the test set. Thus if some words have zero probability, we can't compute perplexity at all, since we can't divide by 0!