multiplied by the embedding matrix E, to give the first part of the first hidden layer, the embedding layer. Since each column of the input matrix E is an embedding layer embedding for a word, and the input is a one-hot column vector x i for word V i , the embedding layer for input w will be Ex i = e i , the embedding for word i. We now concatenate the three embeddings for the three context words to produce the embedding layer e.