The fact that the computation at time t requires the value of the hidden layer from time t − 1 mandates an incremental inference algorithm that proceeds from the start of the sequence to the end as illustrated in Fig. 9 .4. The sequential nature of simple recurrent networks can also be seen by unrolling the network in time as is shown in Fig. 9 .5. In this figure, the various layers of units are copied for each time step to illustrate that they will have differing values over time. However, the various weight matrices are shared across time.