Recall that the skip-gram model learns two separate embeddings for each word i: the target embedding w i and the context embedding c i , stored in two matrices, the target embedding context embedding target matrix W and the context matrix C. It's common to just add them together, representing word i with the vector w i + c i . Alternatively we can throw away the C matrix and just represent each word i by the vector w i .