We'll discuss three popular non-linear functions f () below (the sigmoid, the tanh, and the rectified linear ReLU) but it's pedagogically convenient to start with the sigmoid function since we saw it in Chapter 5: