Step 1 (computing the loss) is used to report how well we are doing on the current tuple. The algorithm can terminate when it converges (or when the gradient norm < ), or when progress halts (for example when the loss starts going up on a held-out set).