|
2002 > |
Course Description |
Connectionist
approaches to psycholinguistics, language acquisition, and linguistic theory
Jeff Elman (homepage)
University of California, San Diego
Language has been one of the most fruitful domains for studying human cognition. Not only does it play a central role in human activity, making possible cultural, social, and intellectual activity that is unparalleled among the animal kingdom; but it is also a domain that is richly complex and attractive as an arena for formal modeling. Not surprisingly, the history of modern computation, cognitive theory, and language reveal tight and important interconnections.
The major focus of this course will be on language, from a connectionist perspective. We will begin with a brief historical review of the intellectual roots of modern cognitive science (from the mid-1800s to present). The bulk of the course will then a set of phenomena in psycholingustics and language acquisition, viewed from a connectionist perspective, and concluding with a discussion of implications for linguistic theory.
(NEW) The power-point version of lectures with slides may be accessed here
Day 1: Historical roots of modern cognitive science
§ psychology in the 19th century
§ Behaviorism
§ cybernetics, computation, AI
§ the cognitivist revolution
§ cognition revised: connectionism, artificial life, situated cognition,
§ & dynamics
Required Readings
A short history of AI
Von Neumann, J. (1948/1963). The general and logical theory of automata. In A.H. Taub, Ed., John von Neumann, Collected Works, Vol. 5. Oxford: Pergamon Press.
Turing, A.M. (1950). Computing Machinery and Intelligence. In Readings in Cognitive Science: A Perspective from Psychology and Artificial Intelligence, (1988). A. Collins and E. E. Smith (Eds). Kaufmann, San Mateo, CA.
Optional Readings
Chomsky, N. (1959). On certain formal properties of grammars. Information and Control, 2, 137-167.
Chomsky, N. (1959). A review of B. F. Skinner's Verbal Behavior. Language, 35, 26-58.
Day 2: Early connectionist language models
§ The word superiority effect model
§ The TRACE model of speech perception
§ The problem of learning, and a solution
§ Rules or networks: The past tense debate
Required Readings
Rumelhart, D.E., & McClelland, J.L. (1986). On learning the past tenses of English verbs. In J.L. McClelland and D.E. Rumelhart (Eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 2. Cambridge, MA: MIT Press. Ch. 18.
Rumelhart, D.E., Hinton, G.E., & Williams, R. (1986). Learning internal representations by error propagation. In D.E. Rumelhart and J.L. McClelland (Eds.) Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1. Cambridge, MA: MIT Press. Ch. 9.
Annotated reading list of past tense, infant learning, and generalization debates (PDF format)
Optional Readings
McClelland, J.L. & Rumelhart, D.E. (1991). An interactive activation model of context effects in letter perception: Part 1. Psychological Review, 5, 375-407.
McClelland, J.L. & Elman, J.L. (1986). Interactive processes in speech perception: The TRACE Model. In D.E. Rumelhart & J.L. McClelland (Eds.) Parallel Distributed Processing, Vol. II. Cambridge, MA: MIT Press.
Hare, M., & Elman, J.L. (1995). Learning and morphological change. Cognition, 56, 61-98.
Day 3: Language acquisition
§ The poverty of the stimulus; lack of negative feedback
§ Generalization vs. conservatism: Empirical findings
§ What do infants and children know, and when do they know it?
§ speech perception; language identification; word segmentation; grammatical categories
§ Infants and grammar learning
§ (The past tense)
Required Readings
Bates, E., & Goodman, J. C. (1997). On the inseparability of grammar and the lexicon: Evidence from acquisition, aphasia, and real-time processing. Language and Cognitive Processes, 12, 507-584.
Marcus, G. F., Vijayan, S., Rao, S. B., & Vishton, P. M. (1999). Rule learning by seven-month-old infants. Science, 283(5398), 77-80.
Seidenberg, M. S., & Elman, J. L. (1999a). Do infants learn grammar with algebra or statistics. Science, 284, 434-435.
Seidenberg, M. S., & Elman, J. L. (1999b). Networks are not 'hidden rules'. Trends in Cognitive Sciences, 3(8), 288-289.
Lewis, J., & Elman, J. (2001). A connectionist investigation of linguistic arguments from the poverty of the stimulus: Learning the unlearnable.
Optional Readings
Gomez, R. L., & Gerken, L. (1999). Artificial grammar learning by 1-year-olds leads to specific and abstract knowledge. Cognition, 70(2), 109-135.
Plunkett, K., & Marchman, V. (1993). From Rote Learning to System Building - Acquiring Verb Morphology in Children and Connectionist Nets. Cognition, 48(1), 21-69.
Plunkett, K., & Juola, P. (1999). A connectionist model of english past tense and plural morphology, Cognitive Science, 23, 463-490.
Day 4: Sentence processing: Experimental results and modeling
§ Basic issues and phenomena in sentence processing
§ The sausage machine and two-stage processor
§ Constraint satisfaction / probabalistic / expectation generation models
Required Readings
Tanenhaus, M.K., & Trueswell, J.C., (1995). Sentence comprehension. In J.L. Miller and P.D. Eimas (Eds.) Handbook of Perception and Cognition, 2nd edition. Vol. 11: Speech, Language, and Communication. NY: Academic Press. Pp. 217-262.
McRae, K., Spivey-Knowlton, M.J., & Tanenhaus, M.K. (1998). Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38, 283-312.
Seidenberg, M.S., & MacDonald, M.C. (1999). A probabalistic constraints approach to language acquisition and processing. Cognitive Science, 23, 569-588.
Optional Readings
Christiansen, M.H., & Chater, N. (1999). Connectionist natural language processing: the state of the art, Cognitive Science, 23, 417-437.
Pickering, M.J., & Traxler, M.J. (1998). Plausibility and recovery from garden paths: An eye-tracking study. Journal of ExperimentPsychology: Learning, Memory, and Cognition, 24, 940-961.
Day 5: Dynamical approaches & linguistic theory
§ Sequential processing
§ Simple (and other) recurrent networks
§ The lexicon and grammar, rethought
§ Dynamical analyses and usage-based grammars
Required Readings
Elman, J.L. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
Elman, J.L. (1995). Language as a dynamical system. In R. Port and T. van Gelder (Eds.), Mind as Motion: Explorations in the Dynamics of Cognition. Cambridge, MA: MIT Press. Pp. 195-223.
Optional Readings
Rodriguez, P., Wiles, J., & Elman, J.L. (1999). A recurrent neural network that learns to count. Connection Science, 11, 5-40.
Afternoon section meetings
Afternoon sessions will be used to explore specific topics in depth and to present additional material.
Assessment
Students who take the course for credit will be asked to write a brief (5-7 page) paper that critical reviews one or more of the articles read in class, or to comment on other work that is related to the issues discussed in the class.
Jeff Elman (homepage)
Jeff Elman received his Ph.D. in 1977 from the University of Texas at Austin. He joined the Department of Linguistics at the University of California, San Diego in that year. In 1986, he participated in the founding of the Department of Cognitive Science at UC San Diego, serving as Chair of that department from 1990 to 1994. He was Chair of the Governing Board of the Cognitive Science Society in 1999-2000. His research interests include language processing, connectionist models, human development, and the evolution of cognition.
Last updated 27/08/2002