A brief history of the Hebbian Learning Rule

Canadian Psychology, Feb 2003 by Peter Milner

Abstract

Hebb included a version of his neural postulate of learning in his MA thesis (1932). It seems to be a translation of Pavlovian conditioning into neural terms. The version that appears in his book, The Organization of Behavior (1949), speaks of synapses rather than routes, but the idea of simultaneous firing of afferent and efferent elements is common to both versions. The postulate was adopted by groups interested in programming computers to learn and think. It was also one of the hypotheses tested by neurophysiologists in their search for the synaptic mechanism of learning. So far the neurophysiologists have had more success than the logicians; the NMDA synapse appears to explain the main features of Hebb's postulate.

In his monograph, The Organization of Behavior, Hebb (1949) presented a theory concerning the way stimuli might be represented in the brain. It was the first such attempt to be widely accepted by psychologists and it has had a strong influence on subsequent theories. Commonsense decrees that brain representations must be learned, at least in the case of representations of the shapes of human artefacts such as the letters of the alphabet, tools, and buildings. The radical empiricism of early 20th century psychology led Hebb to go the whole hog and base his model on the assumption that the neurons of the newborn visual cortex are randomly interconnected. In order to proceed, Hebb then had to specify the conditions under which visual input might organize these connections.

In 1932, when Hebb was a part-time graduate student at McGill, he submitted an MA thesis entitled Conditioned and Unconditioned Reflexes and Inhibition. The gist of the thesis is that spinal reflexes are the result of prenatal Pavlovian conditioning; in his more mature years Hebb referred to it as nonsense (Hebb, 1980). The McGill library had, in the meantime, lost its copy of the thesis, so Hebb's evaluation could not readily be verified. Recently, however, Professor Richard Brown of Dalhousie University, in connection with a presentation he made to the Society of Neuroscience, tracked down another copy of the thesis. The part of it dealing with the ontogeny of spinal reflexes may well be nonsense, as Hebb maintained, but Brown (2001, 2002) made the interesting discovery that the thesis includes an analysis of the neural learning mechanism underlying Pavlovian conditioning that foreshadows the one he later presented to explain the learning of visual representations. Based on the observations of Pavlov and others that a stimulus occurring repeatedly at about the same time as a response, acquires a connection to that response, Hebb concluded that:

An excited neuron tends to decrease its discharge to inactive neurons, and to increase this discharge to any active neuron, and therefore to form a route to it, whether there are intervening neurons between the two or not. With repetition this tendency is prepotent in the formation of neural routes. (Hebb, 1932, p. 8)

An accompanying diagram makes it clear that this postulate was indeed a transposition of Pavlov's conditioning paradigm to the neural level.

When, some dozen or so years later, Hebb needed to specify the neural learning mechanism responsible for the acquisition of shape representations in the brain, it seems that he consciously or unconsciously returned to the formulation in his thesis. The neurological postulate was presented in The Organization of Behavior as follows:

Let us assume then that the persistence or repetition of a reverberatory activity (or "trace") tends to induce lasting cellular changes that add to its stability. The assumption can be precisely stated as follows: When an axon of cell A is near enough to excite a cell B and repeatedly or persistently takes a part in in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency, as one of the cells firing B, is increased. (Hebb, 1949, p. 62)

The idea that a connection between two neurons takes place only if both neurons are firing at about the same time is common to both of Hebb's formulations. This idea was not entirely new. From the earliest times philosophers and psychologists have recognized that learning often involves the establishment of an association between two or more concurrent events. Hartley (1746/1959) made the ingenious suggestion that stimuli produce neural vibrations, and that ideas consist of smaller after-vibrations (vibratiuncles). When two events occur in close succession, Hartley speculated that the frequencies of their vibrations draw together so that if one occurs it excites vibratiuncles of the other.

With the improved knowledge about the nervous system that came in the latter part of the 19th century, the psychologist Alexander Bain made a more modern attempt to explain associative learning in neural terms. In his book, Mind and Body (1874), written when Pavlov was still a medical student and even before the neuron doctrine had been enunciated (Waldeyer, 1891), Bain wrote: