Let's introduce some constants to represent the dimensionalities of these vectors and matrices. We'll refer to the input layer as layer 0 of the network, and have n 0 represent the number of inputs, so x is a vector of real numbers of dimension n 0 , or more formally x ∈ R n 0 , a column vector of dimensionality [n 0 , 1]. Let's call the hidden layer layer 1 and the output layer layer 2. The hidden layer has dimensionality n 1 , so h ∈ R n 1 and also b ∈ R n 1 (since each hidden unit can take a different bias value). And the weight matrix W has dimensionality W ∈ R n 1 ×n 0 , i.e. [n 1 , n 0 ].