a [i] = g [i] (z [i] ) y = a [n] The activation functions g(•) are generally different at the final layer. Thus g [2] might be softmax for multinomial classification or sigmoid for binary classification, while ReLU or tanh might be the activation function g(•) at the internal layers.