We multiply by W (and add b) and pass through the ReLU (or other) activation function to get the hidden layer h. 3. Multiply by U: h is now multiplied by U 4. Apply softmax: After the softmax, each node i in the output layer estimates the probability P(w t = i|w t−1 , w t−2 , w t−3 )