The gradient descent algorithm answers this question by finding the gradient gradient of the loss function at the current point and moving in the opposite direction. The gradient of a function of many variables is a vector pointing in the direction of the greatest increase in a function. The gradient is a multi-variable generalization of the slope, so for a function of one variable like the one in Fig. 5 .3, we can informally think of the gradient as the slope. The dotted line in Fig. 5 .3 shows the slope of this hypothetical loss function at point w = w 1 . You can see that the slope of this dotted line is negative. Thus to find the minimum, gradient descent tells us to go in the opposite direction: moving w in a positive direction. The magnitude of the amount to move in gradient descent is the value of the slope d dw L( f (x; w), y) weighted by a learning rate η. A higher (faster) learning learning rate rate means that we should move w more on each step. The change we make in our parameter is the learning rate times the gradient (or the slope, in our single-variable example):