Download Psychology 8020 : Seminar in Conditioning and Learning for FALL

Psychology 8020 : Seminar in Conditioning and Learning for FALL 2006
Topic: “Associative and motivational determinants of action and
choice”
When: Wednesdays 10 am-12 noon
(times may be adjusted somewhat to accommodate your schedules)*
Where: 225 Elliott Hall
Who: Professor Bruce Overmier ([email protected])
Minimum enrollment = 5.
Precis:
Learning psychologists have long been interested in how animals and humans integrate
their associative, reward, and motivational experiences into a final choice. This seminar will
review both historical and contemporary attempts to resolving the issue from Thorndike (1898)
to Dickinson & Balleine (in press). In doing so, we shall contrast a variety of meta-theoretical
and theoretical approaches seeking empirically assessable contrasts where possible. The
concepts and experiments will range from basic animal learning paradigms to drug abuse to
neuroscience mechanisms of behavior.
Seminar meetings will focus on a major reading each week and participants will take
turns as seminar leaders. Leaders will read an additional paper or two as part of the preparation.
[ 11 meetings only. Not: Sept 6, Oct 4, Nov 8, Dec 13]
General References:
Hilgard, Ernest R. (Ed). (1978). American psychology in historical perspective.. American
Psychological Association. viii, 558 pp. (available on line at APA/PsycArticles)
Benjamin, Ludy T. Jr. (Ed). (1988). A history of psychology: Original sources and contemporary
research. Mcgraw-Hill Book Company. xiii, 560 pp.
Reference suggestions for weekly topics:
1. The Past….
Warren, Howard C. (1916). Mental association from Plato to Hume.
Psychological Review. 23(3), May 1916, 208-230.
Hume, David (1711-1766): A Treatise of Human Nature. In Benj. Rand, (Ed). (1912). The
classical psychologists: Selections illustrating psychology from Anaxagoras to Wundt. (pp. 279312). Houghton, Mifflin and Company. xxii, 734 pp.
http://content.apa.org/books/10885-020.pdf?sid=60e8dd04-a5bb-4450-a549-ec9497666e9f
Bricke, John. (1974). Hume's associationist psychology. Journal of the History of the
Behavioral Sciences. 10(4), Oct 1974, 397-409.
2. The Eastern Europeans: Pavlov, Bekterev, and Konorski: Brain models of associations
Pavlov. I.P. (1927/1960). Lecture II. Conditioned Reflexes. (Pp. 16-32). Dover
Konorski, J. (1948). An outline ofPavlov’s theory of activity of the cerebral cortex. In J
Konorski, Conditioned Reflexes and Neuron Organization. (pp 7-33). Cambridge
University Press.
Bekhterev, V.M. (1932). General Principles of Human Reflexology, trans. 4th Russian ed. New
York: International Publishers. ( Also 1932 in London; originally published as Objective
Psychology in 1907).
Doty, RW (2006). Konorski and conditional reflexes: A historical summary and an addendum.
Acta Neurobiologiae Experimentalis , 66, 91-97.
Miller, S.; Konorski, J. (1969/1928). On a particular form of conditioned reflex.
Journal of the Experimental Analysis of Behavior. 12(1), 187-18
Konorski, J.; Miller, S. (1937). On two types of conditioned reflex.
Journal of General Psychology. 16, 264-272.
.
Konorski, J. (1964). Some problems concerning the mechanism of instrumental conditioning
Acta Biologiae Experimentalis. 24(2), 59-72.
3. Thorndike Reinforced Associations
Thorndike, E. L. (1898) "Animal Intelligence: an Experimental Study of the Associative
Processes in Animals," Psychological Review Monograph Supplement, 2 (1898): No. 4.
Whole No. 8 (1-109)
[An exerpt:
http://content.apa.org/journals/amp/53/10/1125.pdf )
Thorndike, E.L. (1913) The Psychology of Learning (Educational Psychology, II). New York:
Teacher’s College. Pp. 1-31
Hull, Clark L. (1935). Thorndike's Fundamentals of Learning.
Psychological Bulletin. 32(10), Dec 1935, 807-823.
………………..
Hull, Clark L. (1937). Mind, mechanism, and adaptive behavior. Psychological Review, 44, 132. [Also in Hilgard, Ernest R. (Ed). (1978). American psychology in historical
perspective. (pp. 309-336). American Psychological Association. viii,
http://content.apa.org/books/10049-017.pdf?sid=6ffd62e3-55d3-40a3-bc94-ab072657e70a ]
.
Hull, Clark L.(1950). Behavior postulates and corollaries--1949.
Psychological Review. 57(3), 173-180.
4. Wagner-Rescorla Instantiation
Rescorla, R.A., & Wagner. A.R. (1972). A theory of Pavlovian conditioning: Variations in
effectiveness of reinforcement and non-reinforcement. In A.H. Black & W.F. Prokasy
(Eds.), Classical Conditioning II: Current Research and Theory. (Pp 64-98). Appleton
Century Crofts.
Wagner, A.R. ( 1971). Elementary associations. In HH Kendler & JT Spence (eds), Essays in
Neo-Behaviorism. (Pp 187-213). Appleton Century Crofts.
ALSO:
http://www2.psy.uq.edu.au/~landcp/PY269/r-wmodel/r-wmodel.html
Miller, R R.; Barnet, R C.; Grahame, N J. (1995). Assessment of the Rescorla-Wagner model.
Psychological Bulletin. 117(3), May 1995, 363-386.
5 . Tolman and re-introduction of motivations and goals as objects as learned
Tolman, E.C. (1932/ 1949/1967). Purposive Behavior in Animals and Men. Appleton Century
Crofts. [While a key reference and worth reading, Tolman is likely better summarized
by others than presented here by himself. Sample…]
Tolman, Edward Chace (1938). The determiners of behavior at a choice point. Psychological
Review, 45, 1-41. [Also in Hilgard, Ernest R. (Ed). (1978). American psychology in
historical perspective. (pp. 337-370). American Psychological Association. viii, 558 pp.
http://content.apa.org/books/10049-018.pdf?sid=3c84e18e-c58b-4e04-b3ca-e8684ff221fd]
Bolles R.C. (1972) Reinforcement, expectancy, and learning. Psychological Review, 79, 394409. [Bolles was a Tolman student.]
6. Mowrer: Motivations as learned mediating ACTIVATORS in Two-Process theory.
Mowrer, O.H. (1947). On the dual nature of learning—A reinterpretation of “conditioning” and
“problem solving”. Harvard Educational Review, 17, 102-148.
Rescorla R.A. & Solomon, R.L. (1967). Two-process learning theory: Relationships between
Pavlovian conditioning and instrumental learning. Psychological Review, 74, 151-182.
Konorski, J. (1964). Some problems concerning the mechanism of instrumental conditioning
Acta Biologiae. Experimentalis, 24, 59-72
Zielinski, K. (1985). Jerzy Konorski’s thory of conditioned reflexes. Acta Neurobiologiae
Experimentalis, 45, 173-186.
7. Stimulus  Outcome Expectations as CUES for responses
Hull, C.L. (1931). Goal attraction and directing ideas conceived of as habit phenomena
Psychological Review. 38(6), Nov 1931, 487-506.
Trapold, MA, & Overmier, JB. (1972) The second learning process in instrumental learning. In
AH Black & WF Prokasy (eds.), Classical Conditioning II: Current Theory and
Research. (Pp 427-452). Appleton Century Crofts.
Overmier, JB, & Lawry, JA. (1979). Pavlovian conditioning and the mediation of behavior. In G.
Bower (ed.), The Psychology of Learning and Motivation, 13, 1-55.
Overmier, J.B., Bull, J.A., III, & Trapold, M.A. (1971). Discriminative cue properties of
different fears and their role in response selection in dogs. Journal of Comparative &
Physiological Psychology, 76, 478-482.
Other…
Kruse, John M.; Overmier, J. Bruce; Konz, Wilbert A.; Rokke, Eric (1983). Pavlovian
conditioned stimulus effects upon instrumental choice behavior are reinforcer specific.
Learning and Motivation. 14(2), 165-181.
Peterson, Gail B.; Trapold, Milton A. (1980). Effects of altering outcome expectancies on
pigeons' delayed conditional discrimination performance. Learning and Motivation.
11(3), 267-288.
Colwill, Ruth M.; Rescorla, Robert A. (1988). Associations between the discriminative stimulus
and the reinforcer in instrumental learning. Journal of Experimental Psychology: Animal
Behavior Processes. 14(2), Apr 1988, 155-164.
BUT, the challenge
Rescorla, Robert A. (1994) . Control of instrumental performance by Pavlovian and instrumental
stimuli. Journal of Experimental Psychology: Animal Behavior Processes. 20(1), Jan
1994, 44-50.
Therefore:
8. Response  Outcome associations in Two-Process response control.
Colwill, Ruth M. (1994). Associative representations of instrumental contingencies. In
Medin, Douglas L. (Ed), The psychology of learning and motivation: Advances in
research and theory, Vol. 31. (pp. 1-72). Academic Press.
Colwill, Ruth M.; Rescorla, Robert A (1990). Effect of reinforcer devaluation on discriminative
control of instrumental behavior. Journal of Experimental Psychology: Animal Behavior
Processes. 16(1), Jan 1990, 40-47.
Colwill, Ruth M.; Rescorla, Robert A. (1990). Evidence for the hierarchical structure of
instrumental learning. Animal Learning & Behavior. 18(1), Feb 1990, 71-82.
Rescorla, Robert A.; Colwill, Ruth M. (1989). Associations with anticipated and obtained
outcomes in instrumental learning. Animal Learning & Behavior. 17(3), Aug 1989, 291303.
Rescorla, Robert A. (1990). Instrumental responses become associated with reinforcers that
differ in one feature. Animal Learning & Behavior. 18(2), 206-211.
Rescorla, Robert A. (1990). The role of information about the response^outcome relation in
instrumental discrimination learning. Journal of Experimental Psychology: Animal
Behavior Processes. 16(3), Jul 1990, 262-270.
Rescorla, Robert A. (1990). Evidence for an association between the discriminative stimulus and
the response^outcome association in instrumental learning. Journal of Experimental
Psychology: Animal Behavior Processes. 16(4), Oct 1990, 326-334.
9. Motivational Control of Associations with Outcomes
Dickinson, Anthony; Nicholas, D. J. (1983). Irrelevant incentive learning during instrumental
conditioning: The role of the drive-reinforcer and response-reinforcer relationships. The
Quarterly Journal of Experimental Psychology B: Comparative and Physiological
Psychology. 35B(3), 249-263.
Dickinson, Anthony; Dawson, G. R. (1988). Motivational control of instrumental performance:
The role of prior experience of the reinforcer. The Quarterly Journal of Experimental
Psychology B: Comparative and Physiological Psychology. 40(2-B), 113-134.
Balleine, Bernard; Dickinson, Anthony. (1991). Instrumental performance following reinforcer
devaluation depends upon incentive learning. The Quarterly Journal of Experimental
Psychology B: Comparative and Physiological Psychology. 43B(3), 279-296.
Dickinson, Anthony; Balleine, Bernard . (1995). Motivational control of instrumental action.
Current Directions in Psychological Science. 4(5), , 162-167.
OR
Dickinson, Anthony; Balleine, Bernard (1994). Motivational control of goal-directed action.
Animal Learning & Behavior. 22(1), 1-18.
Dickinson, Anthony; Balleine, Bernard (2002). The role of learning in the operation of
motivational systems. In H. Pashler & C.R. Gallistel (Ed). Steven's handbook of
experimental psychology (3rd ed.), Vol. 3: Learning, motivation, and emotion. (pp. 497533). John Wiley & Sons, Inc. x, 900 pp.
Lopez, Matias; Balleine, Bernard; Dickinson, Anthony. (1992). Incentive learning following
reinforcer devaluation is not conditional upon the motivational state during re-exposure.
The Quarterly Journal of Experimental Psychology B: Comparative and Physiological
Psychology. 45B(4), 265-284.
Dickinson, Anthony; de Wit, Sanne. (2003). The interaction between discriminative stimuli and
outcomes during instrumental learning. The Quarterly Journal of Experimental
Psychology B: Comparative and Physiological Psychology. 56B(1), 127-139.
Balleine, Bernard W.; Paredes-Olay, Concepción; Dickinson, Anthony (2005). Effects of
Outcome Devaluation on the Performance of a Heterogeneous Instrumental Chain.
International Journal of Comparative Psychology. 18(4), 2005, 257-272.
And just how critical details can be (trying to reconcile discrepant results):
Balleine, Bernard; Dickinson, Anthony (1992). Signalling and incentive processes in
instrumental reinforcer devaluation. The Quarterly Journal of Experimental Psychology
B: Comparative and Physiological Psychology. 45B(4), Nov 1992, 285-301.
10. Computational Causal Learning
Shanks, David R.; Dickinson, Anthony. (1987). Associative accounts of causality judgment.
Bower, Gordon H. (Ed). The psychology of learning and motivation: Advances in
research and theory, Vol. 21. (pp. 229-261). Academic Press. vii, 319 pp.
Dickinson, Anthony (2001). Causal learning: Association versus computation.
Current Directions in Psychological Science. 10(4), 127-132.
OR
Dickinson, Anthony (2001). Causal learning: An associative analysis.
The Quarterly Journal of Experimental Psychology B: Comparative and Physiological
Psychology. 54B(1), 3-25.
Dickinson, Anthony; Balleine, Bernard W. (2000). Causal cognition and goal-directed action.
In C. Heyes & L. Huber, Ludwig (Ed). The evolution of cognition., Vienna series in
theoretical biology. (pp. 185-204). The MIT Press. viii, 386 pp.
[Are they arguing the same as Rescorla or something more than an association??]
Shanks, David R.; Lopez, Francisco J.; Darby, Richard J.; Dickinson, Anthony . (1996).
Distinguishing associative and probabilistic contrast theories of human contingency
judgment. In Shanks, David R. (Ed); Holyoak, Keith (Ed); Medin, Douglas L. (Ed).
Causal learning. (pp. 265-311). Academic Press. xii, 435 pp.
OTHER TOPICS WE COULD EXPLORE:
Human Causal learning of contingency:
Wasserman, Edward A.; Chatlosh, D. L.; Neunaber, D. J. (1983). Perception of causal relations
in humans: Factors affecting judgments of response-outcome contingencies under freeoperant procedures. Learning and Motivation. 14(4), Nov 1983, 406-432.
Or
Wasserman, E. A.; Elek, S. M.; Chatlosh, D. L.; Baker, A. G. (1993). Rating causal relations:
Role of probability in judgments of response^outcome contingency. Journal of
Experimental Psychology: Learning, Memory, and Cognition. 19(1), 174-188
Shanks, David R.; Dickinson, Anthony. (1987). Associative accounts of causality judgment.
Bower, Gordon H. (Ed). The psychology of learning and motivation: Advances in
research and theory, Vol. 21. (pp. 229-261). Academic Press.
Cheng, PW. 1993). Separating causal laws from causal facts. Pressing the limits of statistical
relevance. In D. Medin (ed.), Psychology of Learning and Motivation, v 30. (Pp
)
Academic Press.
Cheng, PW. (1997). From covariation to causation: A causal Power theory. Psychological
Review, 104, 367-405.
Gallistel’s new(er) theory:
Gallistel, C. R.; Gibbon, John (2000). Time, rate, and conditioning.
Psychological Review. 107(2), 289-344.
Gallistel, C. R.; Gibbon, John (2001). Computational versus associative models of simple
conditioning. Current Directions in Psychological Science. 10(4), 146-150.
Gallistel, C. R. (2002). Conception, perception and the control of action.
Trends in Cognitive Sciences. 6(12), Dec 2002, 504.
Balsam, Peter D.; Fairhurst, Stephen; Gallistel, Charles R. (2006). Pavlovian Contingencies and
Temporal Information. Journal of Experimental Psychology: Animal Behavior
Processes. 32(3), Jul 2006, 284-294.