The simplest activation function, and perhaps the most commonly used, is the rectified linear unit, also called the ReLU, shown in Fig. 7 .3b. It's just the same as z ReLU when z is positive, and 0 otherwise: