An Introduction to Connectionist Networks of Human Cognitive Capabilities

 

Robert M. French

CNRS, University of Burgundy, Dijon, France

(homepage)

 

 

This course will provide a simple introduction of connectionist modeling and will assume no prior knowledge of the subject. We will begin by motivating the connectionist approach to cognitive modeling.  Thereafter, we will consider a number of basic connectionist models, each time discussing the psychological, developmental or neurobiological plausibility of the various models.  We will look at a number of problems with these models and how these problems have (or have not) been overcome. 

 

>>>Modeling in psychology: What's the point?  [download presentation]

 

1. Why connectionism?  The origins and motivations of connectionist modeling  [download presentation]

• The earliest mathematical models of the neuron, McCulloch-Pitts neurons
• Hebb’s contribution, the Hebbian learning rule
• The beginning of neural net modeling in the 1950’s
• The Perceptron
• Problems with the perceptron and the collapse of perceptron modeling
• The re-emergence of neural net models, especially multi-layered perceptrons
• Justifications for using connectionist models.

 

Required Readings:

Rumelhart, D. E., Hinton, G. E., & McClelland, J. L. (1986).    The Appeal of Parallel Distributed Processing.  In D. E. Rumelhart, J. L. McClelland and the PDP Research  Group,  Parallel distributed processing: Explorations   in the microstructure of cognition. Vol. I. Cambridge, MA:   MIT Press,  (Ch. 1), pp. 3-44

Ibid.  A General Framework for Parallel Distributed Processing (Ch. 2). pp. 45-54.

Ibid.  PDP models and General Issues in Cognitive Science. (Ch. 4).  110-146.

 

Optional readings:

Finish Chapter 2, above.

 

 

2. Learning in connectionist models [download presentation]

• Characteristics of connectionist memory systems
• Supervised, unsupervised and reinforcement learning
• Sparse coding, coarse coding in the brain, in connectionist networks
• Sequence learning
• Forgetting in humans, in connectionist networks
• Complementary learning systems

 

Required readings:

Robert M. French. Connectionist Models: The Briefest Course.

 

           

Optional readings:

McClelland, J. L., McNaughton, B. L., & O'Reilly,  R. C. (1995). Why there are complementary learning systems in   hippocampus and neocortex: Insights from the successes and  failures of connectionist models of learning and memory.   Psychological Review, 102, 419-457.

Elman, J. (1990) Finding structure in time. Cognitive Science, 14, 179-211.

 

 

3. Forgetting in connectionist models [download presentation]

• Human forgetting vs. “catastrophic” forgetting in connectionist networks
• Analysis of the problem
• Various solutions
• Dual-network solutions to the problem
• Pseudopatterns and the use of noise in connectionist systems

 

Required Readings:

French, R. M. (1999). Catastrophic Forgetting in Connectionist Networks. Trends in Cognitive Sciences, 3(4), 128-135.

Ans, B., Rousset, S., French, R. M., & Musca, S. (2002). Preventing Catastrophic Interference in Multiple-Sequence Learning Using Coupled Reverberating Elman Networks. Proceedings of the 24^th Annual Conference of the Cognitive Science Society. NJ:LEA. 71-76.

 

 

4. Connectionist models and categorization by young infants

• A surprising categorization asymmetry in young infants
• A connectionist (bottom-up) explanation of this asymmetry
• Predictions of the connectionist model and empirical data
• The necessity of a dual-network model for categorization in older infants
• Implications for other areas, notably category-specific deficits in

 

Required readings:

Mareschal, D., French, R. M., & Quinn, P. (2000). A Connectionist Account of Asymmetric Category Learning in Early Infancy. Developmental Psychology, 36, 635-645.

 

Optional readings:

French, R. M. & Mareschal, D. (1998). Could Category-Specific Semantic Deficits Reflect Differences in the Distributions of Features Within a Unified Semantic Memory?  In Proceedings of the Twentieth Annual Cognitive Science Society Conference. NJ:LEA. 374-379.

 

 

5.  Rule learning and connectionist networks  [download presentation]

1. The debate:  Are rules necessary?  What do we mean by a rule?  Where might they come from?
   
2. Kohonen networks and unsupervised learning
3. How one might plausibly extract rules from unsupervised learning systems?

           

Required Readings:

Marcus, G. F., Vijayan, S., Bandi Rao, S., and Vishton, P. M. (1999). Rule-learning in seven-month-old infants. Science, 283, 77-80.

Cowell, R. A. & French, R. M. (2007). An unsupervised, Dual-Network Connectionist Model of Rule Emergence in Category Learning. In Proceedings of the European Cognitive Science Society Confernece, 2007.

 

 

Extra:  How to Give a Decent Scientific Presentation  [download presentation]  [download paper]

Robert M. French

 

Robert M. French was originally trained as a mathematician, receiving a Bachelor’s and Master’s Degree in mathematics.  He then worked for many years as a translator in Paris, translating along with a colleague, Douglas Hofstadter’s Gödel, Escher, Bach into French.  He then returned to the U.S. to do his Ph.D in artificial intelligence at the University of Michigan.  His dissertation was jointly directed by Douglas Hofstadter and John Holland.  He did a post-doc in experimental psychology at the University of Wisconsin and then moved back to Europe in 1995.  He is currently a Research Director at the French National Scientific Research Center (CNRS) and is based at the LEAD-CNRS research lab at the University of Burgundy in Dijon, France. For more detailed information see: http://www.u-bourgogne.fr/lead/people/rfrench.html