Simplicity as a Fundamental Cognitive Principle

 

Nick Chater

Warwick University

 

 

This course considers the hypothesis that the cognitive system seeks to find the simplest explanation of perceptual and linguistic input, where simplicity is measured in terms of code length. This idea has a century-long history in perceptual psychology, and philosophy of science. Moreover, in the last 35 years, a rich mathematical theory, Kolmogorov complexity, has developed, with both makes the approach precise, and shows why simplicity makes sense as an effective inferential method. Moreover, simplicity underpins a wide range of practical methods in statistics and AI. This course describes some of these developments, but focuses on simplicity as a key explanatory construct for understanding human cognition.

            The ‘key readings’ below deal with some of the issues in the lectures; other issues are new, and have not yet been discussed in the literature. The discussion in the lectures and especially small group discussion will go beyond these readings and consider the broader literature in this area. The course will also discuss new unpublished results, and work in progress in this field.

 

 

1. Cognition and simplicity: An overview

           

This lecture gives a general overview of the approach, with examples.

 

Key Reading

 

Chater, N. (1999). The search for simplicity: A fundamental cognitive principle? Quarterly Journal of Experimental Psychology, 52A, 273-302.

 

 

2. Simplicity in perception

 

Outlines, and argues for the usefulness of, a simplicity-based account of perception, based on elementary perceptual phenomena.

 

Key Reading

Chater, N. (1996). Reconciling simplicity and likelihood principles in perceptual organization. Psychological Review, 103, 566-581.

 

See also readings for Peter van der Helm’s course at this Summer School.

 

 

3. Simplicity in language

 

Outlines some new formal results, developed with Paul Vitányi, that show that language learning is possible, in principle, purely from ‘positive’ evidence. Considers the viability of this approach as a psychological account.

 

Key Reading

Chater, N. & Vitányi, P. (under review). A simplicity principle for language acquisition: Re-evaluating what can be learned from positive evidence.

 

Background

Probabilistic Models of Grammar: Phonology as Information Minimization, Goldsmith, J.  Available at: http://humanities.uchicago.edu/faculty/goldsmith/

 

4. Representing and learning about causal structure

 

Considers how causality can be formally represented, using recent breakthroughs in graphical modelling. Relates this approach to simplicity principle.

 

Key Reading

 

Judea Pearl: The art and science of cause and effect. Epilogue in Pearl J. (2000). Causality. Cambridge: Cambridge University Press. Available at: http://singapore.cs.ucla.edu/LECTURE/lecture_sec1.htm.

 

5. Simplicity as a fundamental principle of scientific inference

 

Considers general problem of scientific inference, and proposes that existing formal results indicate that preferring the simplest theory is, in a specific sense, optimal.

 

Key Reading

 

M. Li and P.M.B. Vitanyi, Inductive reasoning and Kolmogorov complexity, J. Comp. System Sciences, 44:2(1992), 343-384.

Available in postscript form:http://www.cwi.nl/~paulv/kolmcompl.html

 

(note a draft of a new paper by Chater and Vitanyi may be available at the conference)

 

 

Small group discussions

 

Small group discussion will be responsive to issues emerging from the lectures, but may include:

 

1.      Limitations/restrictions of simplicity-based approaches

2.      Levels of explanation of cognitive phenomena

3.      Relation to statistical models, rational analysis, connectionist models

4.      Philosophical implications of the approach

5.      Technical discussion of relevant mathematical issues

6.      Further discussion of specific topics in cognition, in relation to simplicity and related methods.

 

Assessment

 

Students who desire credit can write a 10 page paper describing the viability (or otherwise) of the simplicity principle, in relation to a research area of their choice, from any aspect of the cognitive sciences.

 

 

Nick Chater

 

Nick Chater received an M.A. in Experimental Psychology at Cambridge University, and a PhD in Cognitive Science at Edinburgh University. He has worked as a lecturer at University College London, Edinburgh University and Oxford University. He is currently Professor of Psychology, and Director of the Institute for Applied Cognitive Science, at Warwick University. He has won three national prizes for research. His research focuses on mathematical and computational models of cognition, including reasoning, language processing and perception. He also works on applying basic cognitive science research to practical problems, particularly in education and business.