Download Empirical Background for Skinner`s Basic Arguments Regarding

Empirical Background for Skinner’s
Basic Arguments Regarding Selection
by Consequences
Iver Iversen
University of North Florida, Jacksonville
Presentation at NAFO, April 2016
Gol, Norway
Skinner was Controvercial
1971
1948. A novel about a utopian
society based on positive
reinforcement. Fun to read!
Three Levels of Variation and Selection
• Evolution of life
• Conditioning, Pavlovian and Operant
• Evolution of Cultures
Second Level, Operant Conditioning
• “Operant conditioning occurs at a speed at
which it can be observed from moment to
moment”
• “Operant conditioning is selection in progress.
It resembles a hundred million years of natural
selection or a thousand years of the evolution
of a culture compressed into a very short
period of time” (p. 502)
Today’s Purpose
• Try to show what Skinner had discovered in
the laboratory that led him to these
arguments about selection by consequences.
• What does he mean by “moment to moment”
• And by “a very short time”
• These expressions refer to concrete findings in
his early research with rats (1930’s)
Ivan Pavlov
• First to seriously study intact animals one at a
time in great detail under controlled
conditions
• Pavlov was influenced by physiologists and
received the Nobel Price in Physiology (1904)
• His later work is what he is famous for in
Psychology
10
Example of Pavlov’s Research, One Dog
From Pavlov (1927)
11
Single Trial from One Dog
Detailed Event Chart of Salivation
Top: Salivation
Bottom: Onset of Conditioned Stimulus (left)
and Unconditioned stimulus (right)
From Pavlov (1927)
12
Thorndike
Thorndike at around the time
The older Thorndike
he came to Harvard University
From: Joncich (1968)
From:http://www.s9.com/Biography/ThorndikeEdward-Lee
Puzzle Box
From: Boakes (1984)
Examples of learning curves from
Thorndike’s book Animal Intelligence
(1911).
Each curve is from one animal in one
box.
Trials on the X-axis.
Latency on the Y-axis (no scale).
Logical Diagram
Law of Effect : Trial and Error
• Unnecessary behavior was stamped out (the
errors)
• Successful behavior was stamped in (occurred
sooner)
• A Learning Curve represented the gradually
shortening latencies as the undesirable responses
dropped out
• Early work on Selection by Consequences
• The Effect mattered for behavior
Skinner
1904-1990
PhD from Harvard
1931
From:
Skinner, 1979
Scientific Methods
• Skinner followed the mantra from Pavlov:
“Control your conditions and you will see
order”
• From Crozier (Skinner’s mentor) and Pavlov,
he learned to look for details in behavior and
to set up experiments where each subject (rat)
served as its own control
21
Initial Research
• Skinner first studied eating reflexes and had
rats open a door to a food tray to make a
pellet drop down and recorded door openings
– the eating reflex
• But this response was both learned and part
of the food retrieval process
• To separate the two, he therefore made a
lever that the rat had to press to deliver the
food
22
First Lever Pressing Experiment
• Four food deprived rats
• Habituation to the box
• Magazine training (lever present but
fixed so it couldn’t move)
• Lever was released and each press produced a
food pellet (no shaping)
• All 4 rats learned to press the lever
23
From: Skinner, . B. F. The behavior of Organisms, 1938.
Famous First Data, 1932
25
Skinner: Even one single reinforcer
changes behavior
Time Scale
• These results are what led Skinner to later argue
for “moment to moment” changes in behavior
• Plus even a single reinforcer can affect behavior
• These conditioning effects occur in very short
time scales, even less than one minute
• Hence: “Operant conditioning occurs at a speed
at which it can be observed from moment to
moment”
• The effect was instantaneous, there was no
behavior to stamp out, as for Thorndike, and
there was no “Learning curve”
Acquisition
• Such data changed psychology
• Skinner said many times that one important
aspect of reinforcement is that it may change
behavior instantaneously
• But it is often forgotten that he also showed
how a single reinforcer is sufficient for
immediate change of behavior
• These findings are easy to observe and to
replicate
Video link
Extinction
• Skinner removed the
reinforcer for already
established behavior
and saw dramatic
extinction curves
• This was “Pure
Behavior” resulting
from previous
conditioning history
Skinner (1979) in Iversen (1992)
Role of Extinction
• Extinction was the turning point for Skinner:
• “Free operant” without any kind of eliciting
stimulus
• Extinction plays a role in Shaping – creation of
novel behavior
• Extinction plays a role in acquisition of
complex, sequential behavior
Free Operant
• It meant behavior not prompted or elicited or
goaded by a current stimulus
• In Thorndike’s experiment the behavior was not
free to occur at any time, animal had to get in
and out of the box for each response
• In Skinner’s box the rat could press any time
• In extinction, there is nothing happening before
or after the response, yet it occurs
• Previous conditioning history is what controls the
behavior
Skinner’s Conclusions
•
•
•
•
•
•
Acquisition can be very fast
Can be seen in individual rats
Extinction shows the influence of past history
“Voluntary behavior” can be controlled
Behavior is not “goal directed”
The “future” does not affect behavior, only
past consequences influence the behavior
Schedules of Reinforcement
Schedules
• Were discovered as combinations of
extinction curves
Stimulus Control
• Later Skinner added a stimulus when a
response could be reinforced and all other
responses extinguished.
• The response would occur only when the
stimulus was presented
• This was a method to control when (and
where) a voluntary response will occur
• Skinner focused on the Function of a stimulus
not on discrimination
Stimulus Control Development
• Skinner’s methods were developed over many steps, too long
and complicated to describe here
• Ending in a clear formulation: Three-Term Contingency
• Shows instantaneous selection of behavior, when the stimulus
is presented. Voluntary behavior is controlled.
• Momentary stimulus control results of previous history
Summary
• Skinner revolutionized psychology by getting rid of the
stimulus as a “goad” for behavior as in Reflexology
• The operant had no identifiable stimulus that made it occur, it
was “free” of control at the moment it occurred, hence “free
operant” = voluntary behavior
• Incredibly, the next revolution was to put the stimulus back
again, but this time serving a discriminative function of
voluntary behavior
• The Three-Term Contingency is a way for the environment to
select behavior immediately: Stimulus On  Behavior occurs
• This had a profound influence on Skinner’s development of
Verbal Behavior
38
Time Scales
• The time scale of behavior change was very
short, minutes or even less
• This was a drastic change to Pavlov and
Thorndike
• As Skinner said, conditioning can occur with
one single reinforcement – and nothing is
faster than that
• Much modern research focuses on collecting
data over large time scales
JEAB, 1976, 25, 218
These "molecular“ changes in probability of responding are
most immediately relevant to our own daily lives.
Thank You
41
References
► Boakes, R. (1984). From Darwin to behaviorism: Psychology and the
minds of animals. New York: Cambridge University Press.
► Iversen, I. H. (1992). Skinner’s early research: From reflexology to
operant conditioning. American Psychologist, 47, 1318-1328
► Joncich, G. (1968). The sane positivist: A biography of Edward L.
Thorndike. Middletown, CN: Wesleyan University Press.
► Pavlov, I. P. (1927). Conditioned reflexes. (G. V. Anrep, Trans).
London: Oxford University Press.
► Skinner, B. F. (1938). The behavior of organisms: An experimental
analysis of behavior. Appleton-Century-Crofts, New York.
► Skinner, B. F. (1948). Walden two. Macmillan, New York.
► Skinner, B. F. (1971). Beyond freedom and dignity. Knopf, New York.
► Skinner, B. F. (1976), Farewell my lovely! Journal of the Experimental
Analysis of Behavior, 25, 218.
► Skinner, B. F. (1979). The shaping of a behaviorist. Knopf, New York.
► Skinner, B. F. (1981). Selection by consequences. Science, 213, 501504
► Thorndike, E. L. (1911). Animal intelligence. New York: Macmillan
Iversen data
Iversen data
return