|
2006> |
Course
Description |
Connectionist
approaches to psycholinguistics, language acquisition, and linguistic theory
Jeff Elman
University of California, San Diego
Language has been one of the most fruitful domains for studying human cognition. Not only does it play a central role in human
activity, making possible cultural, social, and intellectual activity that is
unparalleled among the animal kingdom; but it is also a domain that is richly
complex and attractive as an arena for formal modeling. Not surprisingly, the
history of modern computation, cognitive theory, and language reveal tight and
important interconnections.
The major focus of this course will be on language,
from a connectionist perspective. We will begin with a brief historical review
of the intellectual roots of modern cognitive science (from the mid-1800s to
present). The bulk of the course will then a set of phenomena in
psycholingustics and language acquisition, viewed from a connectionist perspective,
and concluding with a discussion of implications for linguistic theory.
(NEW) The power-point version of lectures with slides may be accessed here
Day 1: Historical roots of modern cognitive science
·
psychology in the 19th century
·
Behaviorism
·
cybernetics, computation, AI
·
the cognitivist revolution
·
cognition revised: connectionism, artificial life, situated
cognition,
·
& dynamics
Required Readings
A short
history of AI
Von Neumann, J. (1948/1963). The
general and logical theory of automata. In A.H. Taub, Ed., John von Neumann,
Collected Works, Vol. 5. Oxford: Pergamon Press.
Turing, A.M.
(1950). Computing Machinery
and Intelligence. In Readings in Cognitive Science: A Perspective from
Psychology and Artificial Intelligence, (1988). A. Collins and E. E. Smith
(Eds). Kaufmann, San Mateo, CA.
Optional Readings
Chomsky,
N. (1959). On certain formal properties of grammars.
Information and Control, 2, 137-167.
Chomsky,
N. (1959). A review
of B. F. Skinner's Verbal Behavior. Language, 35, 26-58.
Day 2: Early connectionist language models
·
The word superiority effect model
·
The TRACE model of speech perception
·
The problem of learning, and a solution
·
Rules or networks: The past tense debate
Required Readings
Rumelhart,
D.E., & McClelland, J.L. (1986). On
learning the past tenses of English verbs. In J.L. McClelland and D.E.
Rumelhart (Eds.) Parallel Distributed Processing: Explorations in the
Microstructure of Cognition, Vol. 2. Cambridge, MA: MIT Press. Ch. 18.
Rumelhart, D.E., Hinton, G.E., & Williams, R. (1986). Learning internal representations by
error propagation. In D.E. Rumelhart and J.L. McClelland (Eds.) Parallel
Distributed Processing: Explorations in the Microstructure of Cognition, Vol.
1. Cambridge, MA: MIT Press. Ch. 9.
Annotated reading list of past tense,
infant learning, and generalization debates
Optional Readings
McClelland,
J.L. & Rumelhart, D.E. (1991). An
interactive activation model of context effects in letter perception: Part 1.
Psychological Review, 5, 375-407.
McClelland,
J.L. & Elman, J.L. (1986). Interactive processes in speech perception:
The TRACE Model. In D.E. Rumelhart & J.L. McClelland (Eds.) Parallel
Distributed Processing, Vol. II. Cambridge, MA: MIT Press.
Hare, M.,
& Elman, J.L. (1995). Learning and
morphological change. Cognition, 56, 61-98.
Day 3: Language acquisition
·
The poverty of the stimulus; lack of negative feedback
·
Generalization vs. conservatism: Empirical findings
·
What do infants and children know, and when do they know it?
·
speech perception; language identification; word
segmentation; grammatical categories
·
Infants and grammar learning
·
(The past tense)
Required Readings
Bates, E.,
& Goodman, J. C. (1997). On the inseparability of
grammar and the lexicon: Evidence from acquisition, aphasia, and real-time
processing. Language and Cognitive Processes, 12, 507-584.
Marcus, G.
F., Vijayan, S., Rao, S. B., & Vishton, P. M. (1999). Rule learning by seven-month-old infants.
Science, 283(5398), 77-80.
Seidenberg,
M. S., & Elman, J. L. (1999a). Do
infants learn grammar with algebra or statistics. Science, 284, 434-435.
Seidenberg,
M. S., & Elman, J. L. (1999b). Networks
are not 'hidden rules'. Trends in Cognitive Sciences, 3(8), 288-289.
Lewis, J., & Elman, J. (2001). A connectionist investigation of linguistic arguments from the poverty of the stimulus: Learning the unlearnable.
Optional Readings
Gomez, R.
L., & Gerken, L. (1999). Artificial
grammar learning by 1-year-olds leads to specific and abstract knowledge.
Cognition, 70(2), 109-135.
Plunkett,
K., & Marchman, V. (1993). From
Rote Learning to System Building - Acquiring Verb Morphology in Children and
Connectionist Nets. Cognition, 48(1), 21-69.
Plunkett,
K., & Juola, P.(1999). A
connectionist model of english past tense and plural morphology, Cognitive
Science, 23, 463-490.
Day 4: Sentence processing:
Experimental results and modeling
·
Basic issues and phenomena in sentence processing
·
The sausage machine and two-stage processor
·
Constraint satisfaction / probabalistic / expectation
generation models
Required Readings
Tanenhaus,
M.K., & Trueswell, J.C., (1995). Sentence comprehension.
In J.L. Miller and P.D. Eimas (Eds.) Handbook of Perception and Cognition, 2nd
edition. Vol. 11: Speech, Language, and Communication. NY: Academic Press. Pp.
217-262.
McRae, K., Spivey-Knowlton, M.J., & Tanenhaus, M.K. (1998). Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38, 283-312.
Seidenberg,
M.S., & MacDonald, M.C. (1999). A
probabalistic constraints approach to language acquisition and processing.
Cognitive Science, 23, 569-588.
Optional Readings
Christiansen,
M.H., & Chater, N. (1999). Connectionist
natural language processing: the state of the art, Cognitive
Science, 23, 417-437.
Pickering,
M.J., & Traxler, M.J. (1998). Plausibility and recovery from garden paths:
An eye-tracking study. Journal of ExperimentPsychology: Learning, Memory, and
Cognition, 24, 940-961.
Day 5: Dynamical approaches &
linguistic theory
·
Sequential processing
·
Simple (and other) recurrent networks
·
The lexicon and grammar, rethought
·
Dynamical analyses and usage-based grammars
Required Readings
Elman,
J.L. (1990). Finding structure in time. Cognitive Science, 14,
179-211.
Elman,
J.L. (1995). Language as a dynamical system. In R. Port and T.
van Gelder (Eds.), Mind as Motion: Explorations in the Dynamics of Cognition.
Cambridge, MA: MIT Press. Pp. 195-223.
Optional Readings
Rodriguez,P., Wiles, J., & Elman, J.L. (1999). A recurrent neural
network that learns to count. Connection Science, 11, 5-40.
Afternoonsection meetings
Afternoon
sessions will be used to explore specific topics in depth and to present
additional material.
Assessment
Students who take the course for credit will be asked to write a brief (5-7 page) paper
that critical reviews one or more of the articles read in class, or to comment
on other work that is related to the issues discussed in the class.
Jeff Elman
Jeff Elman received his Ph.D. in 1977 from the University of Texas at Austin. He joined
the Department of Linguistics at the University of California, San Diego in
that year. In 1986, he participated in the founding of the Department of
Cognitive Science at UC San Diego, serving as Chair of that department from
1990 to 1994. He was Chair of the Governing Board of the Cognitive Science
Society in 1999-2000. His research interests include language processing,
connectionist models, human development, and the evolution of cognition.