The origins of neural networks lie in the 1940s McCulloch-Pitts neuron (McCulloch and Pitts, 1943 ), a simplified model of the human neuron as a kind of computing element that could be described in terms of propositional logic. By the late 1950s and early 1960s, a number of labs (including Frank Rosenblatt at Cornell and Bernard Widrow at Stanford) developed research into neural networks; this phase saw the development of the perceptron (Rosenblatt, 1958) , and the transformation of the threshold into a bias, a notation we still use (Widrow and Hoff, 1960) . The field of neural networks declined after it was shown that a single perceptron unit was unable to model functions as simple as XOR (Minsky and Papert, 1969) . While some small amount of work continued during the next two decades, a major revival for the field didn't come until the 1980s, when practical tools for building deeper networks like error backpropagation became widespread (Rumelhart et al., 1986) . During the 1980s a wide variety of neural network and related architectures were developed, particularly for applications in psychology and cognitive science (Rumelhart and McClelland 1986b , McClelland and Elman 1986 , Rumelhart and McClelland 1986a , Elman 1990 , for which the term connectionist or paralconnectionist lel distributed processing was often used (Feldman and Ballard 1982, Smolensky 1988) . Many of the principles and techniques developed in this period are foundational to modern work, including the ideas of distributed representations (Hinton, 1986) , recurrent networks (Elman, 1990) , and the use of tensors for compositionality (Smolensky, 1990) .