Stochastic gradient descent is an online algorithm that minimizes the loss function by computing its gradient after each training example, and nudging θ in the right direction (the opposite direction of the gradient). (an "online algorithm" is one that processes its input example by example, rather than waiting until it sees the entire input). x is the set of training inputs