Download Behaviorist approach

1
Introduction
Learning and Memory are usually considered separately.
Definitions
Learning = “…a relatively permanent change in behavior
due to experience.” (Note: By definition, maturation and
injury would not be learning.)
Memory = “…the representation of experience that is stored
in the mind.”
The definition of learning is quite “behavioristic” and that of
memory is quite “cognitive.”
That makes sense historically:
The two subject matters were studied under different paradigms.
Learning was the focus for Behaviorism. Memory is an important
area within Cognition.
Paradigmatic assumptions and implications:
Behaviorist approach
Organism is “empty” (merely a switchboard)
There is no “mind”
Environmentalism
No “free will”
Strict adherence to British Empiricism principles
Experimental approach
The central problem of psychology is learning
Learning follows laws that apply across all species
2
Cognitive approach
Organism is not empty
Mind is the “author” of the majority of behavior
Anti-environmentalism
There is free will
More moderate empiricism approach
Also takes experimental approach
Central problem of psychology is explaining the
“higher mental processes”
Every species has different cognitive system
Forms of learning
Habituation = a decrease in responding (i.e., orienting or
alerting) that occurs when a stimulus is presented repeatedly
and the stimulus is not related to any biologically significant
event
Sensitization = an increase in responding to a mild stimulus
after an intense stimulus
These two processes are related reciprocally and probably
help to maintain a biological imperative, homeostasis.
Classical conditioning = A previously neutral stimulus comes
to elicit a reflexive or involuntary response.
Instrumental (and also Operant) conditioning = The rate of a
freely emitted response is changed based on the
consequences of that response.
3
Forms of memory
Short-term memory = Sometimes called “working memory”
and is roughly equivalent to “consciousness” – limited
capacity – maintained by rehearsal
Long-term memory = More or less permanent store – large
capacity – no maintenance seems to be required
Important memory terms
Encoding = Forming an internal code (representation) that
reflects one’s experience
Storage = Maintaining the internal code within the memory
system over time
Representation = deals with what a stored memory “looks
like” (i.e., its nature)
Retrieval = locating and activating a stored memory so that it
can be used. (Note: Memory may be quite “constructive”
and, consequently, retrieval may be a very creative process.)
Relationship Between Learning and Memory
An interactive relationship: Learning is guided by past
experience (stored as memory); and, of course, learning
becomes stored as memory.
4
Learning and memory are potentially biologically adaptive
because they cope with a changing environment. (If the
environment didn’t change, hardwired “instincts” would be
more efficient.)
But, to be adaptive, learned behaviors must be retrieved at
the appropriate time.
But, even though learning and memory are related, we
usually keep them conceptually separate.
One reason is that unique variables can apply to each: For
example, learning efficiency is influenced by the practice
schedule during acquisition; but once acquisition has
occurred, memory performance is influenced by length of the
retention interval.
Most important perhaps: Historically, learning and memory
have been studied separately, usually under separate
paradigms, by different psychologists. Separate literatures.
Biological Basis of Learning and Memory
In higher organisms, the brain seems to be involved.
But the picture is less than clear.
Hippocampus seems to be involved in memory registration.
The hippocampus is part of the limbic system (a subcortical
part of the forebrain). Note that the limbic system is also
involved with emotion and motivation.
“H.M.” is a famous case (reported by Milner in 1950s):
5
H.M.’s hippocampus was removed bilaterally (both sides) to
relieve epilepsy. (This would not be done today.)
Result: Some retrograde amnesia (going back in time from
present) with worst effects for more recent past; and almost
complete anterograde amnesia (going forward in time from
the present) with nothing new being remembered after a
distraction.
Principle of Mass Action (Lashley, 1950s): Rat’s
performance on a complex task (e.g., a maze problem)
depended more on the amount of cortex remaining in the
brain, not the location of the remaining cortex. As more
cortex is removed, errors go up but the basic task remains
intact.
Other possibilities for representation: changes in firing rates
of neurons; changes in brain chemistry; altered pathways
(e.g., changes in patterns of excitation and inhibition across
large numbers of neurons)
We know that some areas of the brain become “active”
during certain cognitive tasks, but we are not sure what that
means.
6
Learning might even have a basis outside of what would
normally be thought of as “brain.”
Consider the classical conditioning of the planarian flatworm
(Dugesia species). The planarian is capable of “regeneration”
after sectioning.
A cartoon from McConnell, Jacobson, and Kimble (1950s)
illustrates the classical conditioning procedure:
Note: The posterior portion of the sectioned planarian
retained just as much of the response as the anterior portion.
Finally note structure of planarian nervous system below
(ganglia are only in the anterior part of the body):
(Ant.)
(Post.)
7
Learning can be biologically constrained—this is an
important exception to the Behaviorist notion that learning
principles apply across species uniformly.
Wilcoxon study (1970s): Rats and quail drank water that was
both blue and sour. Later a drug injected to make them sick.
Subsequently, rats avoided sour (but not blue) water; but
quail avoided blue (but not sour) water. Note: In rats,
olfactory and gustatory senses dominate; in quail, visual
sense dominates.
Species-Specific Defense Reactions (Bolles, 1970s): Rats
easily learn to escape from a shock grid by jumping over a
barrier to a non-shock area (called a “shuttle” response), but
rats have difficulty learning to press a bar to end shock.
The Role of Learning in the Higher Mental Processes
This is a tough subject. Wolfgang Köhler (early 1900s), a
Gestalt psychologist, studied problem solving in chimps. Do
chimps learn to do things incrementally (i.e., trial and error
via conditioning -- Behaviorism) or all at once (i.e., suddenly
via insight -- Cognitivism)?
In one study, he placed a chimp in a cage along with a stick;
outside the cage was a bunch of bananas out of direct reach.
A given chimp would grasp the stick, play with it for a while,
then suddenly rush to the bars using the stick to claim his
prize.
But Pavlov objected: Maybe prior experience with sticks
biased the animal toward using the stick to get the bananas
(i.e., the use of the stick was already high – so a random
encounter with the bananas using the stick is likely). Who is
right? Hard to tell.
8
Classical Conditioning
Ivan Pavlov (early 1900s) did the basic experiments. He was a
physiologist interested in digestion.
Basic idea: A previously neutral stimulus comes to signal that a
reflexive stimulus-response sequence is about to occur. In effect,
the organism learns to anticipate the reflexive sequence.
9
Definitions of terms:
CS = Conditioned Stimulus = The previously neutral stimulus =
The clicking sound (or bell, etc.)
US = UCS = Unconditioned Stimulus = The stimulus that
naturally elicits the reflexive response = The meat powder
UR = UCR = The reflexive response that is naturally elicited =
Salivation
CR = The conditioned response that comes to be elicited by the
Conditioned Stimulus = Salivation (similar to UR)
Classical Conditioning is Ubiquitous
Whenever we have a reflexive or emotional response in the
presence of a stimulus that does not naturally elicit it, we
have evidence for classical conditioning.
Classical conditioning and drug addiction
To understand this, we need an important theory:
Opponent-Process Theory of Motivation (Solomon & Corbit,
1974):
A pleasurable stimulus (e.g., drug) produces an “aprocess” which is fast-acting and decays quickly when
the stimulus is removed.
The body reacts with a “b-process” (to counteract the aprocess). The b-process is sluggish in its rise and decay.
10
The purpose of this opponent process is to maintain
homeostasis.
With repeated presentations, the a-process remains the
same, but the b-process grows (starts sooner, becomes
more intense, and lasts longer).
So with repeated presentations of the drug, we see a
decrease in the primary emotional response and an
increase in the after-reaction. This is the foundation of
the “drug-tolerance effect.”
Figure from Solomon and Corbit (1974):
So the body’s need for homeostasis produces the drugtolerance effect.
Question: Suppose you run out of regular coffee (which
you usually drink in the morning) and substitute de-caf,
what will happen?
11
Siegel et al. (1980’s) – an “animal model” for heroin addiction
Two groups of rats were injected with heroin every other day
for 30 days – on non-heroin days, they received dextrose
injections.
Both groups received injections in two distinct rooms: the
rat’s home room or an alternate room.
For one group of rats, heroin only in home room and dextrose
in alternate room; for the other group of rats, dextrose only in
home room and heroin in alternate room.
Amount of heroin was increased across days to adjust for
tolerance effects.
A third (control) group of rats received no heroin, but
received dextrose in both home and alternate rooms.
After 30 days, all rats were given a double dose of heroin:
The experimental rats either received the overdose in their
normal heroin room or their normal dextrose room; the
control rats received their first (and only) experience with
heroin as a double dose.
What should happen with regard to overdose fatalities?
Consider the three groups: controls, OD in “heroin” room,
and OD in “dextrose” room
12
Results:
Note: Different room = dextrose room; Same room =
heroin room.
The customary heroin room served as a CS for a
compensatory CR (an opponent process, or b-process)
that helped to counteract the effects of the overdose.
Summary: controls (no opponent process); different
room (opponent process, but not well elicited); same
room (opponent process well elicited).
Contingency – the dependency between CS and US
A contingency means there is a correlation between CS and
US occurrence
13
A positive contingency means that the CS reliably predicts
the delivery of the US; a negative contingency means that the
CS reliably predicts the absence of the US.
Experiments using positive contingency are called classical
excitatory conditioning experiments; experiments using
negative contingency are called classical inhibitory
conditioning experiments.
If there is no contingency between CS and US, conditioning
cannot occur (i.e., there is nothing to learn).
Common Classical Conditioning Paradigms
Eye-Blink Reflex (Human)
US = air puff
UR = eye blink reflex
CS = tone
CR = eye blink
Nictitating Membrane Reflex (Rabbit)
Same as above except inner eye membrane blinks
Conditioned Emotional Response (CER)
A really important paradigm for developing animal
models of pathologies such as phobias
US = shock
UR = fear (operationalized as startle or freezing)
CS = tone
CR = fear (operationalized as startle or freezing)
14
To measure freezing, we present the CS while the
subject is freely emitting a high-rate operant behavior
(such as bar pressing for food) and measure the amount
of suppression of the operant behavior. (more later)
Variables that affect excitatory conditioning
Temporal relationship between CS and US:
Best: Forward Delayed
Some: Forward Trace
None: Simultaneous, Backward
CS-US Interval
Stimulus onset asynchrony (SOA) = the time between
onsets of two stimulus events (a cognitive term really)
Here we are interested in the SOA between CS and US
15
Smith et al. (1969) studied conditioning as a function of
SOA (using 50-msec tone CS and 50-msec aversive
US)
Note that 50 msec is only 1/20th of a second, so these
are very brief stimuli
Results:
The backward (-50 msec SOA), simultaneous (0 msec
SOA), and slightly forward (+50 msec SOA) CS-US
arrangements do not result in conditioning.
The best arrangement here seems to be an SOA of +200
msec (which is really a trace conditioning procedure
due to short stimulus durations involved).
Spooner and Kellogg (1947) used somewhat longer
stimulus durations (200 msec) in human study. CS =
tone, US = finger shock, UR = finger withdrawl, CR =
finger withdrawl on CS-alone trials.
16
Results:
Here we see, once again, that the backward and
simultaneous procedures don’t work.
But now the maximum conditioning occurs at an SOA
of 500 msec (1/2 sec). This is the more usual finding.
There is a sharp roll off after 500 msec.
Note: As CS-US SOAs become very long, we would
start to experience a negative contingency between the
CS and US, and have a classical inhibitory conditioning
procedure (i.e., the CS would signal a US-free period).
17
Learned Taste Aversion Studies (e.g., Garcia and his
colleagues in the 1960s and 1970s)
At first glance, it seems to be the exception to the
fairly short positive SOA rule.
In LTA studies, the animal eats or drinks a novel
substance with distinctive taste or odor.
If the animal is made sick by a drug hours later,
they will avoid the novel ingested substance in the
future.
This is one-trial learning of a very special kind.
LTA probably does not represent normal
conditioning – therefore, it should not be listed as
an exception to the usual short positive SOA rule.
18
The Role of Contingency (i.e., the correlation of CS and US)
Background: Traditional conditioning theory
emphasized psychology’s traditional philosophical
viewpoint, associationism: By this view, the more
pairings of CS-US, the better the conditioning. Today,
we call this a contiguity view. In other words, what is
learned is an association based on the appearance of
CS-US together in time (i.e., contiguously).
Robert Rescorla challenged that view as a graduate
student at Pennsylvania (late 1960s). Because he
emphasized the CS-US correlation, his was a
contingency view. In other words, learning is the
subject’s realizing that the CS predicts the US.
Note that in the traditional excitatory conditioning
experiment, contiguity and contingency are
confounded. We need to take special steps to deconfound them.
After Rescorla (1968), consider two conditions:
Cond A
CS
US
Time 1
+
+
Time 2
+
Time 3
+
+
Time 4
+
Cond B
CS
US
Time 1
+
+
Time 2
-
Time 3
+
+
Time 4
-
The number of CS-US pairings (contiguity) is equal for
the two conditions; but in Condition B, the US always
appears when the CS is present (and never alone).
19
Contingency is quantified by the following conditionalprobability equation:
Contingency = P(US | CS) – P(US | no CS)
For Condition A (no contingency),
ContingencyA = 2/2 – 2/2 = 0.0
For Condition B (perfect contingency),
ContingencyB = 2/2 – 0/2 = 1.0
Rescorla (1968) taught rats to press a bar for food
pellets. Then he carried out a conditioning procedure in
which the CS was a tone and the US was shock.
Several conditioning groups were formed based on the
probability of shock occurring without the tone (i.e.,
P(US | no CS)) while holding the number of pairings
constant P(US | CS).
Rescorla (1968) evaluated the quality of conditioning
by the amount of suppression of the operant (bar press)
response when the CS was presented. This is called a
Conditioned Emotional Response (CER) procedure.
Suppression is a measure of the amount of fear
produced by the CS, which in turn is a measure of the
quality of the prior conditioning.
20
The CER procedure uses the suppression ratio as its
dependent variable: The more the animal freezes when
the CS is presented during bar pressing, the lower the
suppression ratio; conversely, the less the animal
freezes, the higher the suppression ratio.
Note: Range: 0.0 ≤ SR ≤ 0.5
Results:
If P(US | no CS) = 0, the suppression is at maximum
(suppression ratio = 0) indicating maximum fear.
Conclusion: Conditioning also depends on contingency,
not just contiguity. Both are important.
21
Number of CS-US Pairings
All other factors held constant, the more CS-US
pairings, the greater the likelihood that a CS will come
to elicit a CR (until an asymptote is reached).
Figure 2.11:
This finding accords with our intuitions concerning the
“learning curve” and seems to go nicely with a pure
contiguity view of conditioning (but recall that the
contiguity and contingency views overlap a lot).
CS and US Intensity
In general, as either CS or US intensity increases,
strength of conditioning increases. This effect seems to
be due to salience (intensity relative to background)
rather than absolute stimulus magnitude.
Prior Experience with CS or US
In general, pre-exposure to a CS alone hinders
subsequent conditioning using that CS (called latent
22
inhibition). Same thing with pre-exposure to US (called
pre-exposure effect).
Relevance of CS and US
In defiance of a “pure associationism” doctrine, not just
any CS goes with a given US.
Gorgeous experiment by Garcia and Koelling (1966):
Rats trained to drink a solution from a tube. The
solution had a distinctive taste and drinking was
accompanied by a light and a clicking sound.
Note: Both taste and A/V stimuli were combined
into single CS complex.
After drinking, ½ rats were given shock US and
the other ½ rats given lithium chloride US (which
produces nausea).
Then all rats (now thirsty) were given access to
drinking tubes again. The tubes either had the
original taste CS or the original A/V CS (but not
both). For each tube type, ½ rats had been given
the shock US previously and ½ rats have been
given the lithium chloride US previously.
Results (Licks per minute):
Test CS
Taste
A/V
Previous US
Li-Cl
Shock
140
300
280
70
23
Conclusion: Biological relevance dictates the ease
with which specific CS-US combinations result in
conditioning.
CS as a Compound Stimulus
Overshadowing. If some stimulus of a CS
compound is more salient than the other stimuli,
the more salient stimulus will become the
effective CS and the less salient stimuli will
receive little or no conditioning.
Blocking. If one stimulus in a CS compound has
undergone conditioning alone before it became
part of the compound, that stimulus will prevent
conditioning of the other stimuli in the compound.
Correlation. If only one stimulus in a CS
compound is correlated with the US, only it will
be conditioned; however, if no stimulus in a CS
compound is correlated with the US, but one is a
constant component of the CS, then the constant
component will be conditioned.
Example: Wagner et al. (1968)
24
Results:
Light (L) in Uncorrelated condition serves
as a “background” stimulus and, because
nothing else is correlated with US, becomes
the focus of the subject’s fear response.
Conditioning Without an Explicit US
Higher Order Conditioning
Once a CS is conditioned to a US-UR sequence,
another CS can be conditioned to the previouslyconditioned CS. So we have a chaining effect:
If
And
Then
US (shock)  UR (fear),
CS (tone) is paired with (US-UR)
CS (tone)  CR (fear)
Now if
Then
CS2 (light) is paired with CS (tone)
CS2 (light)  CR (fear)
Essentially CS2 predicts fear, so it comes to elicit it.
25
Classical Inhibitory Conditioning
A really murky area: Little has been done, and even less is
coherent. However, a couple of things emerge:
CS Discriminations. Subjects readily learn that a CS+ is
always followed by a US and that a CS– is never followed by
a US. If the US is aversive, CS+ produces fear and CS–
probably produces a hedonically opposite effect (relief?).
Long CS-US Intervals. The subject can learn that the US
will follow the CS after a long delay. Here the CS (which is
supposed to be excitatory) takes on the properties of a CS–.
26
Extinction
A really important concept in conditioning.
The CS ceases to be a reliable predictor of the US-UR
sequence; as a result, the CR diminishes in magnitude as the
CS is presented alone repeatedly.
Although it is analogous to forgetting (in memory),
extinction is different: Forgetting occurs because a memory
is not used; extinction occurs because a contingency no
longer exists.
Three things suggest that the CS-CR association remains
intact during extinction, even though the CR diminishes:
1) The CS-CR can be reconditioned in fewer trials than the
original conditioning required; 2) a change in context can
allow the CS-CR to return at full strength; 3) “spontaneous
recovery” can occur in which the CR re-emerges in the old
context (although at less than full strength).
Note: One of the best measures of CS-CR conditioning
strength is the number of trials to extinction required for the
CS presented alone.
Variables That Affect Rate of Extinction
1. Number of CS-US acquisition trials.
2. Partial reinforcement (no US on some conditioning
trials) results in more resistance to extinction than
continuous reinforcement (US on every trial). But
continuous reinforcement results in faster conditioning.
27
Clinical Applications of Extinction (treating phobias)
The main problem is to allow extinction to occur
instead of avoidance (which prevents extinction).
Techniques:
Flooding. Subject is immersed in phobic CS’s.
Systematic Desensitization (Wolpe). Relaxation
techniques are learned first, then the client
confronts CS’s in a “fear hierarchy” (beginning
with the least fear-inducing CS). Relaxation is
incompatible with fear. Because fear (due to
original US-UR) does not occur in presence of a
given CS, we have extinction.
28
The pursuit of truth and beauty is a sphere of activity in which we are permitted to
remain children all our lives.
--Albert Einstein
Classical Conditioning: Theoretical Issues
What theory serves as an adequate explanation of classical
conditioning (i.e., what conditions are necessary and
sufficient to explain it)?
There are three basic theoretical positions: contiguity,
contingency, and hybrid approaches.
Contiguity = temporal relationship between CS and
US-UR sequence is what matters
Contingency = the predictive relationship (i.e.,
correlation) between CS and US-UR sequence is what
matters
Rescorla-Wagner model = a hybrid approach
Contiguity theory (Pavlov, 1900’s)
Basic explanation: The temporal overlap of neural
activity of CS and US-UR provides the basis for
association; nature of CS and US don’t matter (just so
they are perceived).
Evidence for: Basic temporal findings for CS and US
(e.g., delayed conditioning > trace conditioning);
optimal SOA is best, etc.
29
Evidence against: Garcia and Koelling (1966) showed
that A/V CS works better with shock US than Li-Cl US;
and Rescorla (1968) showed that US-alone trials disrupt
conditioning even if number of CS-US pairings is
equal.
Another problem: Contiguity can’t handle inhibitory
conditioning: That is, under the theory, a CS– can’t be
neurologically paired with a non-event (nothingness).
Contingency theory (Rescorla, 1960’s)
Basic explanation: Conditioning occurs when
P(US | CS) ≠ P(US | no CS)
Note:
If P(US | CS) > P(US | no CS), then excitatory;
and
if P(US | CS) < P(US | no CS), then inhibitory.
Evidence for: Basically same as contiguity theory. Plus
it accounts for conditioned inhibition (above); and also
accounts for the negative effects of US-alone trials on
conditioning.
Evidence against: Some studies have reported
conditioning without contingency (but not conclusive);
and possible problems interpreting US-alone trials in
Rescorla (1968).
30
Rescorla-Wagner model (1972)
An elegant, powerful model
Assumptions: 1) Conditioning occurs to the extent that
the presence or absence of the US is “surprising”; 2) a
CS is excitatory if it precedes US-UR occurrence, and
inhibitory if it precedes a US-UR non-occurrence; and
3) if the CS is no longer followed by the US-UR
occurrence or non-occurrence, then extinction occurs.
Background of model: The essential idea of “surprise”
is actually from Kamin’s (1969) explanation of the
blocking phenomenon.
That is, if a compound CS consists of a tone + light,
after conditioning with the compound CS, normally,
either the tone or light alone will elicit a CR; however,
if either the tone or the light had been paired with the
US previously, the other stimulus element of the
compound CS will not come to elicit a CR when
presented alone after compound conditioning.
Kamin (1969) argued that when organisms first
experience a US, it comes as a complete surprise. A
surprise is not good, so the animal tries to learn to
predict the US by “looking back” in memory for a
salient event that occurred just prior to US.
The tone becomes the new predictor of the US.
If the tone later becomes part of a compound CS, only
the tone will be noticed and conditioning to other CS
elements will not occur (because the US is already
predicted by the tone and so US is not a surprise).
31
To support this idea, Kamin then did the following:
Pre-training: Tone  shock
Conditioning: Tone + Light  shock
Blocking test: Light  no CR
New experience: Tone + Light  shock … shock
New test: Light  CR present
Reason: second shock was surprise
Formal (Rescorla-Wagner) model:
∆Va ═ αβ (λ ─ Vax)
where:
∆Va is change in associative strength between CS and
US on a given trial (subscript a refers to trial #)
α
is salience of CS
β
is intensity of the US
λ
amount of conditioning that the US can
potentially undergo (i.e., its associative strength or
potential predictability)
Vax
the amount of conditioning that has already
occurred between the US and all other stimuli
(i.e., degree to which US is already predicted)
So the difference (λ ─ Vax) is the US surprise factor for
a given trial.
32
How the model works:
Model: ∆Va ═ αβ (λ ─ Vax) Note: subscript a refers to trial #
Simplifying assumptions:
Salience of CS & intensity of US: let αβ = 0.3
Maximum US associative strength: let λ = 1.0
Prior US conditioning (all stimuli): let V1x = 0.0
Trials:
∆V1 = Δ CS-US strength on T1 = 0.3 (1.0 – 0.00) = 0.30
∆V2 = Δ CS-US strength on T2 = 0.3 (1.0 – 0.30) = 0.21
∆V3 = Δ CS-US strength on T3 = 0.3 (1.0 – 0.51) = 0.15