We first represent each of the N previous words as a one-hot vector of length |V |, i.e., with one dimension for each word in the vocabulary. A one-hot vector is one-hot vector a vector that has one element equal to 1-in the dimension corresponding to that word's index in the vocabulary-while all the other elements are set to zero. Thus in a one-hot representation for the word "toothpaste", supposing it is V 5 , i.e., index 5 in the vocabulary, x 5 = 1, and x i = 0 ∀i = 5, as shown here: