The input layer x is a vector of simple scalar values just as we saw in Fig. 7 .2. The core of the neural network is the hidden layer h formed of hidden units h i , hidden layer each of which is a neural unit as described in Section 7.1, taking a weighted sum of its inputs and then applying a non-linearity. In the standard architecture, each layer is fully-connected, meaning that each unit in each layer takes as input the outputs fully-connected from all the units in the previous layer, and there is a link between every pair of units from two adjacent layers. Thus each hidden unit sums over all the input units.