Detailed Course Descriptions

Course Description

An Introduction to Connectionist Networks:

How They Learn and, especially, How They Forget

Robert M. French

University of Liege, Belgium

This course will provide a simple introduction of connectionist modeling and will assume no prior knowledge of the subject. We will begin by motivating the connectionist approach to cognitive modeling. Thereafter, we will consider a number of basic connectionist models, each time discussing the psychological, developmental or neurobiological plausibility of the various models. We will look at a number of problems with these models and how these problems have (or have not) been overcome.

1. Why connectionism? The origins and motivations of connectionist modeling

The earliest mathematical models of the neuron, McCulloch-Pitts neurons
Hebb’s contribution, the Hebbian learning rule
The beginning of neural net modeling in the 1950’s
The Perceptron
Problems with the perceptron and the collapse of perceptron modeling
The re-emergence of neural net models, especially multi-layered perceptrons
Justifications for using connectionist models.

Required Readings:

Rumelhart, D. E., Hinton, G. E., & McClelland, J. L. (1986). The Appeal of Parallel Distributed Processing. In D. E. Rumelhart, J. L. McClelland and the PDP Research Group, Parallel distributed processing: Explorations in the microstructure of cognition. Vol. I. Cambridge, MA: MIT Press, (Ch. 1), pp. 3-44

Ibid. A General Framework for Parallel Distributed Processing (Ch. 2). pp. 45-54.

Ibid. PDP models and General Issues in Cognitive Science. (Ch. 4). 110-146.

Optional readings:

Finish Chapter 2, above.

2. Learning in connectionist models

Characteristics of connectionist memory systems
Supervised, unsupervised and reinforcement learning
Sparse coding, coarse coding in the brain, in connectionist networks
Sequence learning
Forgetting in humans, in connectionist networks
Complementary learning systems

Required readings:

Robert M. French. Connectionist Models: The Briefest Course.

Optional readings:

McClelland, J. L., McNaughton, B. L., & O'Reilly, R. C. (1995). Why there are complementary learning systems in hippocampus and neocortex: Insights from the successes and failures of connectionist models of learning and memory. Psychological Review, 102, 419-457.

Elman, J. (1990) Finding structure in time. Cognitive Science, 14, 179-211.

3. Forgetting in connectionist models

Human forgetting vs. “catastrophic” forgetting in connectionist networks
Analysis of the problem
Various solutions
Dual-network solutions to the problem
Pseudopatterns and the use of noise in connectionist systems

Required Readings:

French, R. M. (1999). Catastrophic Forgetting in Connectionist Networks. Trends in Cognitive Sciences, 3(4), 128-135.

Ans, B., Rousset, S., French, R. M., & Musca, S. (2002). Preventing Catastrophic Interference in Multiple-Sequence Learning Using Coupled Reverberating Elman Networks. Proceedings of the 24^th Annual Conference of the Cognitive Science Society. NJ:LEA. 71-76.

4. Connectionist models and categorization by young infants

A surprising categorization asymmetry in young infants
A connectionist (bottom-up) explanation of this asymmetry
Predictions of the connectionist model and empirical data
The necessity of a dual-network model for categorization in older infants
Implications for other areas, notably category-specific deficits in

Required readings:

Mareschal, D., French, R. M., & Quinn, P. (2000). A Connectionist Account of Asymmetric Category Learning in Early Infancy. Developmental Psychology, 36, 635-645.

Optional readings:

French, R. M. & Mareschal, D. (1998). Could Category-Specific Semantic Deficits Reflect Differences in the Distributions of Features Within a Unified Semantic Memory? In Proceedings of the Twentieth Annual Cognitive Science Society Conference. NJ:LEA. 374-379.

5. Other

Details of three other important types of neural networks:

Kohonen networks and unsupervised learning
Radial Basis Function (RBF) networks
Hopfield Networks and their relation to Sparse Distributed Memory (SDM)

Required Readings:

Course notes.

Small groups/Labs:

Participants will get hands-on experience with various connectionist models in different contexts.

Assessment:

An examination.

Robert M. French

Robert M. French was originally trained as a mathematician, receiving a Bachelor’s and Master’s Degree in mathematics. He then worked for many years as a translator in Paris, translating along with a colleague, Douglas Hofstadter’s Gödel, Escher, Bach into French. He then returned to the U.S. to do his Ph.D in artificial intelligence, jointly directed by Douglas Hofstadter and John Holland, at the University of Michigan. He did a post-doc in experimental psychology at the University of Wisconsin and then moved back to Europe in 1995, where he is currently Professor of Psychology, directing the Quantitative Psychology and Cognitive Science unit at the University of Liege in Belgium. For more detailed information see: http://www.ulg.ac.be/cogsci/rfrench.html