But these derivatives only give correct updates for one weight layer: the last one! For deep networks, computing the gradients for each weight is much more complex, since we are computing the derivative with respect to weight parameters that appear all the way back in the very early layers of the network, even though the loss is computed only at the very end of the network. The solution to computing this gradient is an algorithm called error backpropagation or backprop (Rumelhart et al., 1986) . While backprop was invented spe-error backpropagation cially for neural networks, it turns out to be the same as a more general procedure called backward differentiation, which depends on the notion of computation graphs. Let's see how that works in the next subsection.