A feedforward neural LM is a feedforward network that takes as input at time t a representation of some number of previous words (w t−1 , w t−2 , etc.) and outputs a probability distribution over possible next words. Thus-like the n-gram LM-the feedforward neural LM approximates the probability of a word given the entire prior context P(w t |w 1:t−1 ) by approximating based on the N previous words: