Download Table of Contents - Neuropsychopharmacology

Learning Mechanisms
Table of Contents
8
8.1
8.2
8.3
8.4
8.5
8.6
8.7
8.8
8.9
Learning Mechanisms ......................................................................................................... 8-1
Stages of High-Level Learning.........................................................................................................8-2
Adaptation ........................................................................................................................................8-5
Associative Learning ........................................................................................................................8-9
Classical Conditioning....................................................................................................................8-14
Operant Conditioning .....................................................................................................................8-18
Reinforcement Learning .................................................................................................................8-20
Extinction........................................................................................................................................8-24
Category of Learning......................................................................................................................8-27
Associative Memory.......................................................................................................................8-32
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
Learning Mechanisms
8 Learning Mechanisms
Objectives
• Understand the different biological mechanisms for learning
• Understand the relationship between learning and memory
Concepts to Learn
• Adaptation
• Associative learning
• Classical conditioning
• Operant conditioning
• Extinction
• Reinforcment learning
• Positive reinforcement
• Negative reinforcement
• Imitation learning
• Associative memory
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-1
Learning Mechanisms
8.1 Stages of High-Level Learning
Objectives
• Understand the different stages of learning
Concepts to Learn
• Stage 1: unconscious incompetent
• Stage 2: conscious incompetent
• Stage 3: conscious competent
• Stage 4: unconscious competent.
Conscious competence theory of learning identified the four stages of learning, progressing
from unconscious incompetent to conscious incompetent to conscious competent to unconscious
competent. At each stage, the level of awareness increases. It started at the level of unaware of
our lack of knowledge to the awareness to what we don’t know. From then, acquisition of
knowledge is actively sought. Once the knowledge is acquired, it arrives at the conscious
awareness of the fact that we have gained that knowledge. With practice, we become an expert
in the subject area, and become so fluent in it that it becomes intuitive to us, and become part of
us so naturally. That is the final stage that we become an expert in it.
Summary
The four stages of high-level learning is the progression from:
• unconscious incompetent
• conscious incompetent
• conscious competent
• unconscious competent.
Q&A
What are the stages of cognitive learning?
In conscious competence theory, there are four stages:
• Stage 1: unconscious incompetent
• Stage 2: conscious incompetent
• Stage 3: conscious competent
• Stage 4: unconscious competent.
The conscious competence theory of learning was proposed by Abraham Maslow in the
1940’s. It represents the progression of cognition in the learning process.
Unconscious incompetent stage represents the ignorant stage – we don’t even know what we
don’t know at all.
Conscious incompetent stage represents the beginning of awareness – we begin to know what
we don’t know.
Conscious competent stage represents the active learning process to acquire knowledge –
now we know what we are talking about.
Unconscious competent stage represents the fluency stage in which we become an expert –
we are so good at it that it becomes second nature without having to think about it.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-2
Learning Mechanisms
What is unconscious incompetent?
It is the stage in which we don’t even know that we don’t have the knowledge.
This is the first stage – the ignorant stage that we have no knowledge of what we don’t know.
We are not aware of our lack of knowledge. This is the naive stage that we don’t know our
deficiency.
This is also the stage that we don’t even know our blind spots. We are totally blind from it.
What is conscious incompetent?
It is the stage in which we are aware of the lack of knowledge.
Conscious incompetent represents the beginning of awareness that we don’t have the
knowledge yet. We are aware of our incompetency (or deficiency) at a particular subject matter.
It is the beginning of the process to seek knowledge. This is one of the reasons why we go to
school to learn something that we don’t know. At this stage, we often feel incompetent and
insecure about the fact that we don’t have the needed knowledge.
At this stage, we are aware of our blind spots, and recognize what is missing in us.
Learning a foreign language is a good example. Before we learn the language, we feel so
incompetent about our ability to understand or speak the language at this stage.
Going to med school is another example. At this stage, you realize there are so many
different diseases and symptoms that are too overwhelming. You go through the clinical round,
and visit the patients, but realize that there are so many different cases each with their own
peculiarity that you don’t know how long it will take you to master all that to make the diagnosis
and provide treatment options.
What is conscious competent?
It is the stage in which we have acquired the knowledge and skills in it.
Conscious competent represents the active learning process to acquire knowledge. When we
have acquired the knowledge, we often feel competent and more secure about what we know, but
we still have to work on it.
For instance, when you aced a course, you feel conscious competent about the subject, but
you still have to rely on the book to reference the materials, even though you understand what
you are reading about. You acquired the skills, but you still need to work on it and think about it
to be fluent in it.
Using foreign language as the example, at this stage, we would have acquired the new
language, but we are still translating the native language into the foreign language conscious
instead of thinking in the new language.
For med school, this is the stage in which you have gone through many clinical rotations, and
finally understand how to make diagnosis and treat the patients. But you still need to have the
medical textbook with you so you can look up the cases and diagnosis from time to time as
reference, even though you can now be independent.
In real life, we would have acquired the needed knowledge in the subject matter, but still
need to quote someone else’s word for it to back up your claims at this stage.
What is unconscious competent?
It is the stage in which we become so fluent in it that we don’t need to think about it to be
good at it.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-3
Learning Mechanisms
Unconscious competent represents the stage in which we become an expert, and the
knowledge is integrated within us. We become so fluent in it that we don’t need to work on it – it
becomes second nature to us. We know it automatically.
This is the expert stage in learning. You don’t need to consult anyone or any reference
materials to know what you are talking about. You know it automatically and intuitively.
For foreign language, at this stage, we are so fluent in this new language that we think and
dream in this new language rather than in our mother tongue. We don’t need to translate between
the languages consciously.
In medicine, you are the expert, the specialist, at this stage. You are so good at it that other
people come to you for consultant and ask for your opinion and advice. You are very
comfortable with what you are doing, and feel very confident about your knowledge in the
subject. You know it without knowing it.
In real life, others will quote you as the expert at this stage, instead of you quoting someone
else’s word to back you up on the credentials. You are the credential.
Review Questions
1) What is the first stage in conscious competence theory of learning?
2) What is the last stage in conscious competence theory of learning?
3) What does each stage of learning represent?
Critical Thinking Questions
1) How does awareness facilitate the learning process?
2) What is the cognitive process that leads to active learning (active acquisition of knowledge)?
3) Why is the last stage of learning become unconscious again?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-4
Learning Mechanisms
8.2 Adaptation
Objectives
• Understand the mechanism for producing adaptation
Concepts to Learn
• Adaptation of response to stimulus
• Sensitization
• Habituation and desensitization
Adaptation is the change of response to the same stimulus that is repeated over time. The
condition is repetition of the stimulus, and the response is a change in the response size. If the
response increases, it is called sensitization. If the response is decreased, it is called habituation
or desensitization. Adaptation only requires one stimulus whereas conditioning requires two
stimuli to produce. Adaptation is the most basic form of learning that is universal to most
animals.
Summary
Adaptation is the simplest form of learning that changes the response size in response to the
repetitive stimulation. The response size can increase (sensitize) or decrease (habituate or
desensitize) as a result of repeating the stimulus over time.
Q&A
What is adaptation?
Adaptation is a change of response due to repeated stimulation.
In adaptation, there is only one stimulus. The stimulus is repeated many times. With
repeated exposure to the stimulus, the response of the animal changes.
In contrast to conditioning or associative learning, it requires two stimuli, whereas adaptation
requires only one stimulus.
What is the difference between adaptation and conditioning?
Adaptation requires only one stimulus whereas conditioning requires two stimuli.
Adaptation is the most fundamental form of learning because it only requires repetitive
stimulation to establish the adaptation. It is the mere repetition of the stimulus within a short
period of time that produces the adaptation.
What is the necessarily condition for adaptation to occur?
The condition is repetitive stimulus over time.
When a stimulus is repeated over and over again, it provides the condition in which
adaptation can occurs. Without adaptation, even if the stimulus is repeated many times, the
response would be exactly the same. But with adaptation, the response is usually reduced.
What form of learning is the most basic and simplest in evolution?
Adaptation is the most fundamental and simplest form of learning in evolution.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-5
Learning Mechanisms
Even though adaptation does not establish any association with any new stimulus, the
response does change according to the stimulus of the environment. The change in response
provides the animal with the advantage of conserving energy by reducing the response if the same
stimulus is repeated over time. The reduction in response may or may not be appropriate as far as
the context of the environment is concerned because it is not associating with other stimuli.
Nonetheless, it does serve some useful function and provides a convenient strategy to cope by
effectively conserve energy.
For instance, gill-withdrawal reflex in the sea slug (Aplasia) can be reduced by repetitive
stimulation of the gill, resulting from reducing the gill-withdrawal after awhile. It is because if
the stimulus is harmless, it conserves energy not to withdraw the gills. This happens in the
natural environment when the sea waves splash into the gills, causing the gill-withdrawal reflex.
But such stimulus is harmless, and withdrawal of the gills does not necessarily protect the animal.
On the other hand, if a predator is touching the gills suddenly, then withdrawal will server a
useful purpose in protecting the animal.
Why do psychologists not consider adaptation as learning whereas physiologists consider it
as basic learning?
It is because the response is automatic in adaptation, psychologists consider it learning as a
cognitive process rather than an unconscious process, whereas physiologists consider it as
learning because it is the basic mechanism of learning in changing the response.
The reason why psychologists do not usually consider this form of learning as “learning” is
because it does not involve any cognitive skills or cognition. It is an automated process.
Psychologists do not even use the term “conditioning” to refer to learning because the change
in response by associating two stimuli does not involve any cognitive process or conscious
control of its learning. They prefer to reserve the term “learning” to higher-learning, such as
conscious declarative learning and instruction procedural learning rather than the implicit
learning subconscious.
On the other hand, physiologists consider adaptation as learning because it is the most basic
biological mechanism in which responses are changed. The change in response to a stimulus is a
form of learning, whether it is conscious or not. The biological substrate is changed, i.e., the
neural circuitry is changed to produce a different output given the same input. The difference is
that if the input is repeated over time, the output is reduced. Whereas if there is no adaptation, the
output would be exactly the same independent of how many times the input is repeated.
What is sensitization?
It is the increase in the response when the stimulus is repeated many times.
That is, over-exposure to the stimulus over time can alter the response. In the case of
sensitization, the response increases compared to the pre-exposed response. This usually happens
when the stimulus is noxious (noci = painful) to the animal.
Increasing the response also means that a smaller stimulus can produce the same response.
Thus, the animal is able to tune into the stimulus and senses the potentially dangerous stimulus
with a smaller stimulus. This enables the animal to prepare for and predict the potential danger
more readily.
What is habituation?
It is the decrease in the response when the stimulus is repeated many times.
An animal can respond lesser to the same stimulus if the stimulus is repeated many times
within a short time. This usually occurs when the stimulus is neutral (i.e., not harmful). When
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-6
Learning Mechanisms
the stimulus does not lead to potential danger, the animal can ignore it since ignoring it saves the
animal energy from responding to a neutral stimulus.
Thus, when the animal decreases its response to a neutral stimulus, it habituates to it.
What is desensitization?
It is habituation.
Desensitization is the opposite of sensitization. It is another name for habituation.
Habituation carries the connotation that the stimulus is repeated over time, which produces the
reduction in response. It emphasizes the process of repetition in habitual behavior.
What is the condition for sensitization?
If the stimulus is noxious, repeating the stimulus would produce sensitization.
This is a form of adaptation that requires only one stimulus repeated over time, as opposed to
associative learning that requires two stimuli to occur.
What is the condition for habituation?
If the stimulus is neutral (not harmful), repeating the stimulus would produce habituation.
This is also a form of adaptation that requires only one stimulus repeated over time, similar to
sensitization, except that the response is opposite.
What is the change in response for adaptation?
It either increases or decreases the response size.
There are two forms of changes: increase or decrease the response size.
When the response is increased, it is called sensitization. When the response is decreased, it
is called habituation.
What are the similarities and differences in the responses between adaptation and
conditioning?
For adaptation, the response is either sensitization or habituation (desensitization).
For conditioning, the response is either seeking or avoidance behavior.
Adaptation produces opposite responses (sensitization vs. habituation) depending on whether
the stimulus is noxious or not. On the other hand, conditioning produces different opposite
responses (goal-seeking vs. avoidance behavior) depending on whether the stimulus is noxious or
not.
Thus, there are many ways an animal can learn or adapt, and the direction of the response
depends on whether the stimulus is noxious or not.
Give some examples of adaptation.
The examples are: reduction in the gill-withdrawal reflex in Aplasia; the desensitization of
the feeling of wearing your socks after awhile.
Note that adaptation learning is found in almost all animals, including invertebrates such as
worms, sea slugs, crayfish, etc. An example is the gill withdrawal reflex in Aplasia in which the
animal stops withdrawing the gill when touched harmlessly for a while. Another example is that
you are not aware of wearing the socks as the day goes on after you put them on in the morning.
Note that not only adaptation is found in most multicellular animals, including lower animals,
such as invertebrates, operant conditioning is also found in these animals as well.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-7
Learning Mechanisms
What is the computational advantage for having adaptation as an automatic response?
Adaptation not only conserves energy consumption by reducing the reflex, but also avoiding
distraction when the stimulus is no longer important any more.
Thus, we do not need to pay attention to the thousands of stimuli that repeatedly impinging
on us every minute and pay more attention to other important stimuli. We have to selectively
identify which stimulus is more important to the survival of the animal than others. Adaptation is
a convenient strategy to automatically suppress those stimuli that are repeated without causing
any potential harm, so new (novel) stimulus will get more attention.
Some of the mental disorders are caused by hypersensitivity to the external stimuli, and
unable to suppress them, leading to sensory overload. If the hypersensitivity is responding to the
perceived potentially harmful stimuli, then it can lead to anxiety disorders, phobia, and even posttraumatic stress disorder. If the hypersensitivity is responding to any stimuli, then it can lead to
paranoia and schizophrenia. These heightened responses are produced by both adaptation and
conditioning, in which the response is increased by associating with some stimuli (events) that
may not have equal significance as the initial event.
Review Questions
1) What is adaptation?
a) What is sensitization?
b) What is habituation?
c) What is desensitization?
d) What is the difference between them?
e) What are the different conditions that produce them?
f) What are the differences in the stimulus?
g) What are the differences in responses?
Analytical Thinking Questions
1) Why does sensitization produce a bigger response whereas habituation produces a smaller
response to the same stimulus?
2) What are the advantages and disadvantages of adaptation and conditioning?
Critical Thinking Questions
1) What are the situations in which adaptation is advantageous to an animal?
a) What are the situations in which adaptation is disadvantageous to an animal?
2) Which type of learning is more efficient and automatic – adaptation or conditioning?
3) Which type of learning is more contextually relevant and appropriate – adaptation or
conditioning?
Creative Thinking Questions
1) If computer adapts to our stimulus, what would happen?
a) Is it good or is it bad that computer adapts to us?
b) Is it good or is it bad that animals adapts to us?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-8
Learning Mechanisms
8.3 Associative Learning
Objectives
• Understand the fundamental principles of associative learning
Concepts to Learn
• Characteristics of associative learning
• Associating two events in proximity
• Caveat of associative learning
• Correlation does not imply causality
Associative learning is one the most basic types of learning in animals. It merely associates
two independent events that happened together in close proximity in time as related. In
evolution, establishing correlation between two adjacent events as significant is one of the first
steps in learning.
The one of characteristics of associative learning is that two events have to occur together
before correlation is established. It is different from adaptation, which requires only one event to
occur by changing the sensitivity of the response. For associative learning, which psychologists
called conditioning, the response is changed only by two independent events that occur close
together. This establishes the correlation between them – from independent events to related
events.
Although correlation does not necessarily imply those events are related, associative learning
merely put some significance between them by assumption, and as convenient first step in
establishing association among random events that is more likely to be related. Whether such
correlation will eventually lead to meaningful significance, such as relatedness or causality, is
dependent on subsequent processing in logical reasoning. Nonetheless, this associative learning
is found to be almost universal in all animals.
Summary
Associative learning is establishing correlation by pairing between two adjacent events in
close proximity in time. It is one of the basic types of learning in which two events are
associated together to produce a response. It is the first step in the learning process that allows
an animal to select importance of two events that occurs together among many other random
events. Although correlation does not necessarily imply causality, establishing correlation is the
first step in the learning process to associate events that are closed together as more important
than other events.
Q&A
What is learning?
Learning is the ability to acquire new knowledge that it did not have before.
There are many types of learning; the most fundamental learning in animals (almost universal
in all animals) is adaptation and associative learning. The other type of learning is declarative
learning (which is how human learn in school by following descriptive instructions).
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-9
Learning Mechanisms
What is associative learning?
Associative learning is the type of learning by associating two events that occur in close
proximity to one another.
This is the one of most fundamental and most basic types of true learning in animals. It is a
form of implicit learning in which the animal learns implicitly by correlating events together
rather than explicitly by following instructions.
This type of learning is often unconscious to the animal – the animal merely slowly adapts to
the relationship between the two events when they are repeated many times, and form an
association between the two.
What is the assumption in associative learning in evolution?
Nature’s assumption is that if two events occur close together in time, then these two events
must have some significance, so they are registered in the brain.
Associative learning is one of the evolutionary consequences that establishes the significance
of two closely adjacent events as more important than other events that happened farther apart in
time. The closer they happen together, the more likely they have significance for the animal.
Thus animals acquired the process for establishing importance of two adjacent events in the
associative learning process. Whether these two events are actually important is not the concern
at this stage of processing. It is the first step in the process. That is why associative learning is so
fundamental in almost all animals.
What are the characteristics of associative learning?
It associates two independent events together to form association between the two.
The characteristic is that it requires a minimum of two events to occur at close proximity in
time to be associated together. Compared to other simpler type of learning, adaptation requires
only one event to occur, whereas associative learning requires two events to occur.
Animals are all endowed with these basic types of learning. Adaptation merely changes the
sensitivity of the response, whereas associative learning acquires associate the context of the
stimuli with the responses by associating the two.
Psychologists often called associative learning as conditioning, while they usually do not
consider adaptation as learning because it does not acquire any context. But physiologists do
consider adaptation as the most basic type of learning (the simplest type), whereas associative
learning as the next level of learning.
What is the process of inducing associative learning?
It is formed by pairing of two events in close proximity in time, and repeating the pairing
between these events many times that form the associative learning.
With repeated presentation of the paired stimuli, the animal will learn to associate the two
stimuli that were uncorrelated before. It is the repeated occurrence of these events that
establishes the correlation (or association) that it did not have before.
An example is the Pavlovian conditioning (also called classical conditioning). Pavlov was a
Russian scientist who discovered how animals learn by a simple experiment: He rings the bell,
and then gives the dog food. By repeating this pairing presentation of food with the bell ringing,
the dog soon begins to expect food whenever the bell rings. All animals learn using this same
mechanism.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-10
Learning Mechanisms
What is the difference between before and after learning?
Before learning, we don’t form any association between two events. After learning, an
association between them is formed.
That is, before learning, things appear to be random to us. After learning, we start to make
association between them. This association may or may not be relevant. If it is a relevant
association, then the learning is appropriate for the situation. If the association is a random
association, then the relationship between them is irrelevant.
If mere association (or coincidence) is the mechanism for biological learning, will an animal
learn the wrong association?
Yes, animals and humans can learn to make a misleading association without knowing it so
long as those events happened to be occurring together.
That is why people often jump to conclusion when they see two things happen together. But
mere coincidence does not necessarily mean that they are related. This coincidental occurrence
can be due to random chance occurrence, even though related events can also happen together
too.
This means that evolution has produced a mechanism for learning about anything, but that
learning may or may not be appropriate or relevant to a given scenario. That is why we can learn
the wrong thing without knowing it. The most common form of this random, irrelevant
association is myths and superstition. The association is formed randomly that eventually
becomes a belief system (i.e., associating things together permanently) even though that
association is a random association.
For example, a person or an animal may form association between thunderstorm and
something bad happening to them, and forms a belief system that thundering is a punishment
from some mysterious origin such as god. That association between thundering and punishment
is merely a coincident, yet the person or animal has learned that one leads to another, i.e., thunder
leads to punishment. It may seem irrational but it is logical according to the associative learning
mechanism, because so long as there is a pairing between two events, an animal will form an
association between them.
What is the caveat of associative learning?
It is making the wrong conclusion by using correlation to imply causality.
Correlation is the necessary condition for causality, but is the insufficient condition. It
requires a lot more than correlation to establish causality. Although causal events are always
correlated, correlated events are not necessarily causal.
How can we avoid forming the wrong association or jumping into conclusion knowing that
associative learning may lead us astray?
In human, we have the cognitive ability to separate out association that occurs by mere
coincidence and non-coincidental association that are truly related to each other.
The difference is to identify the probability of coincidental events; that is, what is the
likelihood of things happened to be together by chance, and factor that into the equation in
making conclusions. If things happen above and beyond the chance level, then the occurrences of
these events may not be coincidental.
What is the rule of thumb in guiding us in separating out what is relevant and what is not
in an association?
Simple rule of thumb: Correlation does not imply causality.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-11
Learning Mechanisms
Just because two things happened together does not mean that they are related or that one
causes another. For instance, the alarm clock rings in the morning when the sun rises. Does it
mean that the alarm clock causes the sun to rise, or the sun causes the alarm clock to ring?
Neither is true, because even though they both happen together without exception, they are
not causal, nor are they related by any means. That is why mere coincidence, or when things
happen together does not mean any relevance or relatedness between them. That is often the
cause of absurdity and how false belief system is established because of the irrelevance of
associating things together that is formed by the biological mechanism in basic learning.
What does correlation and causality have in common?
Correlation is the necessary condition for causality, but correlation is not the sufficient
condition for causality.
Correlation is merely an association between two events. Those events may occur
coincidentally, but not causally. That is why merely association does not imply they are causal.
Why does correlation not imply causality?
It is because there can be other events other than the correlated events are causing things to
happen.
The correlated events can happen by coincidence, i.e., they are random events that happened
to occur in close proximity in time.
For instance, when A and B occurs together, they can form an association between the two.
But A does not necessarily cause B to happen, nor B causes A to happen. Why? Because C can
be causing A and B to happen, but A and B are totally unrelated. They just happened to be
together, which is caused by C, rather than A causing B or B causing A.
What is the most common fallacy in most people’s logical thinking?
Just because two events occur together, it does not mean they are related or causal.
The most common trap is that when we see two things happen together, we conclude that
they are causal rather than the considering the possibility of random chance occurrence that they
happened together. In fact, these two events may be totally unrelated, not to mention they are not
causal.
The reason why most people think they are related is because of the basic mechanism of
associative learning in our neural circuitry that makes association between the two regardless of
whether they are related or causal. Association is the first step in the process, but it is not the end
of the process yet. That is why lower animals often form superstitious beliefs without the higherlevel processing. Humans form the same superstitious beliefs too by forming the erroneous
association to believe they are causal, when they did not follow through with the higher-level
cognitive reasoning process.
Which part of the brain is involved in associative learning?
Associative learning often involves the limbic system, especially the hippocampus for
associative learning and amygdala for emotional learning.
Hippocampus and amygdala of the limbic system is mostly involved in the association of the
environmental events. Other parts of the brain are also involved in different aspects of
associative learning, such as the Nucleus Accumbens in the mesolimbic system is involved in the
reinforcement of the associative learning, and the cortex is involved in the cognitive process of
the associative learning.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-12
Learning Mechanisms
Review Questions
1) What is associative learning?
2) What are the characteristics of associative learning?
3) What are the conditions in which associative learning is established?
4) What is the difference between before and after associative learning?
Critical Thinking Questions
1) Why does correlation not imply causality?
2) Why is associative learning the first step in higher-learning even though establishing
correlation does not necessarily conclude about causal relationship?
3) Why is associating unrelated events together can lead to wrong conclusion?
4) What is the caveat in associating learning to watch out for?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-13
Learning Mechanisms
8.4 Classical Conditioning
Objectives
• Understand how classical conditioning occurs
Concepts to Learn
• Conditions before and after learning
• Response before and after learning
• Unconditioned stimulus
• Unconditioned response
• Conditioned stimulus
• Conditioned response
Classical conditioning is a form of associative learning in which the unconditioned stimulus
(US) is paired with the conditioned stimulus (CS). Pavlov was the first to discover this
phenomenon by his experiment with his dog. He gives the dog food (US) right after he rings the
bell (CS). By repeating this sequence, he can train his dog to beg for food whenever the dog
hears the bell.
The indicator of the dog learning to make the association between bell and food is the
salivation upon hearing the bell, i.e., the expectation of food. The innate salivation response to
the presentation of food is called unconditioned response (UR) while the salivation response in
response to the ringing of the bell is called the conditioned response (CR).
Before learning, the bell and food are not associated, and are considered as random events for
the dog. After learning, the bell takes on significance, which is the signal preceding the food
reward. Thus, the dog makes the association between the bell and food only after the paired
presentation of bell and food. Before that, bell and food are random, unrelated events. After
conditioning, they become related events for the dog.
Summary
Classical conditioning (also called Pavlovian conditioning) is produced by pairing CS with
US, resulting in a CR when CS is presented. Before learning, CS would not elicit a CR. Only
after training will CS elicit a CR. Thus, associative learning is established by paired association
between the CS and CR, whereas before learning, CS and CR are unrelated for the animal.
Q&A
What is Pavlovian conditioning?
It is a form of associative learning. It is also called classical conditioning.
It is called classical conditioning because it is the classical form of learning (the first type of
learning discovered by scientists). It is called “conditioning” in psychology because this type of
learning is acquired by the “conditions” in which the animal is in, i.e., by the specific presentation
of the stimuli.
Because Pavlov was the first person to discover this type of learning, it is called classical
conditioning, or alternatively Pavlovian conditioning.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-14
Learning Mechanisms
How did Pavlov discover classical conditioning?
Pavlov discovered the basic learning mechanisms in animals by performing a simple
experiment in which a dog learns that ringing of the bell would lead to food feeding when the
experimenter repeats ringing the bell followed by presenting the food immediately.
Initially, before the dog learns (or make the association), the bell ring and food are totally
uncorrelated and random. The dog did not have any association between the bell ring and
presentation of food.
During the learning process (which is also called “training” in learning theory), the two
stimuli – bell-ring and presentation-of-food – are paired together. By repeating the process, the
dog eventually will start to associate the ringing of the bell with the presentation of food, and
expect that food will come immediately after the bell-ring. As a result, the dog starts to salivate
whenever it hears the bell.
At the end of the experiment, Pavlov can simply ring the bell (without presenting the food),
and the dog would immediately salivate (i.e., expecting that food would follow).
Why is this type of learning called conditioning?
It is because the learning is conditioned on the stimulus.
The condition for this type of learning is conditioned on the two stimuli. If only one of the
stimuli is presented, no associative learning occurs. It is only when the condition in which both
stimuli are given, then learning occurs.
So the conditions in which the associative learning occurs is used to classify this type of
learning as conditioning compared to adaptation in which it learns unconditionally, i.e., without
any condition of the stimuli for sensitization/desensitization to occur in adaptation.
What is the difference between the initial and final responses after classical conditioning?
Compared to the initial condition, the dog did not salivate when the bell rings. That is, there
is no association between the bell and food. At the conclusion of the training session, the dog
salivates when the bell rings (even without seeing the food).
This indicates that the dog made the association between the bell-ring and presentation-offood. Thus, the dog has learned by association (although the dog may not be conscious of this),
and this type of learning is called associative learning.
What is paired association?
Paired association is the presentation of two stimuli in close proximity in time that produces
a predictable response.
In the case of Pavlov’s experiment, the pair of stimuli is bell-ring and presentation-of-food.
The dog learns the association between the two stimuli. The ability to associate two independent
events together allows the animal to establish new knowledge (stimulus-response function).
There is also a more fundamental kind of learning called “adaptation,” which basically
involved only one stimulus instead of two. By repeating the same stimulus, the animal could
either become more sensitized to it or less sensitized (habituated) to the repeated stimulus.
What are the stimuli that are paired in Pavlov’s experiment?
The paired stimuli are bell sound and food presentation.
If the bell sound and food presentation were presented separately at different time, no
associative learning (conditioning) would occur. It is only when these two events are presented
together at the same time, then associative learning occurs.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-15
Learning Mechanisms
This is the most fundamental type of learning in animal’s behavior. Pavlovian condition is
used in most animal training, such as dog-obedience training, acrobatic performance of dolphins
in Sea World™, in which the animal is rewarded with food each time they perform the
appropriate behavior.
How important is the timing between two stimuli for conditioning to occur?
The two stimuli have to present in close proximity in time in order for conditioning to occur.
Thus, the timing is very important in associative learning. In fact, the bell has to ring before
the food in order for conditioning to occur. If the order is reversed, such as presenting the food
before the bell, conditioning would not be effective.
What is unconditioned stimulus (US)?
It is the stimulus that will elicit an innate response.
In Pavlov’s experiment, the presentation of food is the unconditioned stimulus because it will
always elicit a salivation response. The response does not have to be learned – it is innate.
What is unconditioned response (UR)?
It is the innate response that is elicited by the unconditioned stimulus.
In Pavlov’s experiment, salivation is the unconditioned response because it is solicited by the
presentation of food. The salivation response is the innate response that does not need to be
learned.
What is conditioned stimulus (CS)?
It is the stimulus that does not elicit an innate response initially, but will elicit a learned
response after conditioning.
In Pavlov’s experiment, ringing of the bell is the conditioned stimulus because initially, the
bell will not elicit a salivation response, but after training, the bell sound will elicit salivation
response for the dog.
What is conditioned response (CR)?
It is the learned response that is elicited by the conditioned stimulus.
In Pavlov’s experiment, the salivation response elicited by the bell sound is the conditioned
response. Note that salivation itself is both UR and CR, but the condition in which it is elicited is
different. When it is elicited by food (US), it is called the UR. If it is elicited by the bell (CS),
then it is called CR.
Review Questions
1) What is classical conditioning?
2) How is classical conditioning established?
3) What is the difference between unconditioned stimulus and conditioned stimulus?
4) What is the difference between unconditioned response and conditioned response?
5) What is the difference before and after conditioning?
6) What are the conditions in which classical conditioning occurs?
Analytical Thinking Questions
1) How does animal trainer teach dolphins to jump and flip on command of whistle blowing?
2) Could the dolphins be trained without giving them food?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-16
Learning Mechanisms
Critical Thinking Questions
1) If we can train dogs and dolphins using classical conditioning, can we teach kids using the
same procedure?
2) How do babies learn how to respond to their name?
a) What are the conditions in which they make the association?
b) What are the stimuli?
Creative Thinking Questions
1) How do you train someone to love you?
2) How do you teach others to dislike you?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-17
Learning Mechanisms
8.5 Operant Conditioning
Objectives
• Understand how operant conditioning works
Concepts to Learn
• Characteristics of operant conditioning
• Self-learning without a “teacher”
• Unsupervised learning
• Difference between classical conditioning and operant conditioning
Operant conditioning is similar to classical conditioning except that the animal is learning by
itself when the stimuli (CS and US) are encountered coincidentally by the animal rather than
delivered by a “teacher” deliberately. All the conditions are the same for the associative learning
to occur except the difference is who delivers the CS signal. In classical conditioning, the
“teacher” (or Pavlov) is delivering the CS, whereas in operant conditioning, the animal is
delivering the CS (without a “teacher”).
Operant conditioning usually takes longer time to learn than classical conditioning. It is
because the pairing between CS and US happens by random coincidence; the probability of
coincidental occurrence is usually very low. On the other hand, the pairing in classical
conditioning is intentional and deliberate every time; the probability of paired occurrence is very
high.
Most animals learn by operant conditioning because there is usually no “teacher” is available
to hold their hands to do things to provide the one-on-one pairing. They learn by accident.
Regardless of learning by operant conditioning and classical conditioning, the principle of
associative learning is the same for both.
Summary
Operant conditioning is essentially self-learning without a “teacher.” It is similar to classical
conditioning except that the CS is encountered by the animal itself without the delivery of CS
intentionally by a “teacher.” The result is associative learning, associating the CS with US.
Q&A
What is operant conditioning?
It is the behavior elicited by self-association between the CS and US without a “teacher” who
presents the CS and US.
It is essentially self-learning. The mechanism is the same as classical conditioning in which
CS and US are paired together to produce the subsequent CR. The difference is that no “teacher”
is required in the process, i.e., Pavlov is not the person who rings the bell and presents the food to
the dog, but rather, the dog rings the bell and discovered the food drops in front of him, so he
makes the association by himself.
This is how rats learn how to press lever bar to get food or juice reward. When a rat
accidentally presses the lever, and the food drops in front of him; then with repeated trials, the rat
will learn to press the lever bar to get food reward. This is operant conditioning.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-18
Learning Mechanisms
What is the difference between classical conditioning and operant conditioning?
The difference is that a “teacher” is required to present the CS and US in classical
conditioning, whereas no “teacher” is required in operant conditioning.
In operant conditioning, the animal produces the CS itself (such as the dog rings the bell or
the rat presses the lever), and if the CS happens to coincide with US, then operant conditioning
occurs. Then the dog will learn to ring the bell to get food, or the rat will learn to press the lever
bar to get food.
Which associative learning takes longer time to achieve – classical conditioning or operant
conditioning?
Operant conditioning will usually take longer time to learn because it relies on coincidental
occurrence of simultaneous presentation of CS and US, whereas the presentation of CS and US is
deliberate and consistent in classical conditioning.
Because coincidental pairing of CS and US is random and infrequent, the frequency of CS
and US occurrence simultaneously in operant conditioning is much less that classical
conditioning. Therefore, it usually takes much longer time for operant conditioning to learn than
classical conditioning.
How do we train a rat to press lever bar in experiments?
We use the operant conditioning paradigm, i.e., when a rat accidentally press the lever bar,
food is delivered. After repeated trials, the rat will learn to associate bar-pressing with food
delivery.
The rat essentially learns by itself based on random encountering of the bar-pressing CS, and
the subsequent immediate delivery of food. If we hold the hand of the rat and press the lever bar,
then it is classical conditioning.
But we don’t need to hold the hand of the rat, so we let the rat presses the bar itself by
accident and get the food reward. When that happens, it is operant conditioning. Usually, this
will take longer time to train than classical conditioning because it takes more time for the
accidental pressing of the bar by the rat than holding the hand of the rat to press the bar.
Review Questions
1) What is operant conditioning?
2) What is the difference between operant conditioning and classical conditioning?
a) Which one requires the presence of a “teacher” to learn?
3) Why does operant conditioning take longer time to learn?
4) Why type of associative learning animals use to learn most often – classical or operant
conditioning?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-19
Learning Mechanisms
8.6 Reinforcement Learning
Objectives
• Understand how reinforcer affects behavioral outcomes
Concepts to Learn
• Conditions in which seeking behavior occurs
• Conditions in which avoidance behavior occurs
• Positive and negative reinforcer
• Reward and punishment
• Transfer of learnng from US to CS
• Transfer of knowledge from prior experience to new experience
Reinforcement learning is associative learning that is reinforced by a reinforcer – either
reward (positive reinforcer) or punishment (negative reinforcer). The result of the reinforcement
is seeking behavior toward the reward if it is positively reinforced, or avoidance behavior away
from the punishment if it is negatively reinforced. That is, if the CS is paired with a reward, this
lead to positive reinforcement. If it is paired with a punishment, it leads to negative
reinforcement.
The resulting association is that for positive reinforcement, it will lead to a pleasant
experience, which will be more likely to seek the same behavior or environment subsequently.
For negative reinforcement, it will lead to unpleasant experience, which will be more likely to
avoid that behavior or environment subsequently.
In this way, new learning occurs by transferring the innate reward to the new stimulus as
rewarding for positive reinforcement. The new stimulus is to be sought subsequently. Or
conversely, for negative reinforcement, the new learning is transferring the innate punishment to
the new stimulus as aversive. The new stimulus will be avoided subsequently. Thus, associative
learning is transferring knowledge from US to CS – from a prior experience to a new experience.
Food and water are usually the innate positive reinforcers for animals. Most of the classical
and operant conditionings in animals are elicited by pairing with either food or water that the
animal likes. Rats are trained to press lever bar with operant conditioning rewarded by food or
water in the operant chamber. Subsequently, the same rat can be trained to self-administer drug
as a substitute for food or water by the same reinforcement procedure. If the drug is rewarding,
the animal will continue the same drug-seeking behavior. If the drug is aversive, the animal will
have drug-avoidance behavior.
Summary
There are two types of reinforcement learning – positive reinforcement and negative
reinforcement. Pairing with rewarding stimulus will lead to seeking behavior. Pairing with
punishing/aversive stimulus will lead to avoidance behavior. They both are associative learning.
Reinforcement learning is merely associating the unconditioned rewarding (or punishing)
stimulus with the new conditioned stimulus. The result is that the animal will associate the new
stimulus with either the rewarding experience or the aversive experience.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-20
Learning Mechanisms
Q&A
What is the condition that an animal will learn to seek toward a behavior?
If the animal is rewarded during the paired association, the animal will seek toward that
behavior.
In the case of Pavlovian experiment, since the food is a reward for the dog, the paired
association is rewarded, thus the dog tends to seek toward that behavior, i.e., expecting food.
What is the condition that an animal will avoid a specific behavior?
If the animal is punished during the paired association, the animal will avoid that behavior.
In the case of Pavlovian experiment, if the dog is electrically shocked while paired with the
bell ringing, the dog tends to predict the ringing with a shock. Thus, the dog would tend to avoid
that situation, and respond with an alarm cry whenever the bell rings.
This shows that paired association can be used to produce seeking behavior or avoidance
behavior depending on whether the paired association is rewarded or punished.
What is the difference in condition in which the animal seeks or avoids the conditioned
stimulus?
If it is rewarding, it will lead to seeking behavior.
If it is punishing, it will lead to avoidance behavior.
The difference is whether the US is rewarding or punishing. If CS is paired with a rewarding
US, then the animal will seek toward the CS. If CS is paired with a punishing US, then the
animal will avoid the CS.
What is a reinforcer?
It is the CS that is used to reinforce the behavior. It is either rewarding or punishing.
A reinforcer is the condition in which the conditioning will lead to either seeking behavior or
avoidance behavior.
What is a positive reinforcer?
It is the reward.
Positive reinforcers are used to elicit seeking response toward the CS that is paired with that
reinforcer. For instance, if the bell ring is paired with food (a reward), then the dog will approach
when hearing the bell sound.
What is a negative reinforcer?
It is the punishment.
Negative reinforcers are used to elicit avoidance response away from the CS that is paired
with that reinforcer. For instance, if the bell ring is paired with electric shock (a punishment),
then the dog will freak out when hearing the bell sound.
How do we train a rat to self-administer drug?
After we train the rat to bar-press for food, we can then substitute drug for food as the
reinforcer, and see if the rat will continue the bar-pressing behavior when reinforced by the drug.
If the drug has rewarding property, then the rat will continue the bar-pressing behavior. That
is, the drug acts as a positive reinforcer.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-21
Learning Mechanisms
If the drug has aversive property, then the rat will terminate the bar-pressing behavior. Then
the drug would act as a negative reinforcer.
How do we know if a drug has reinforcing property by experimenting with the rats?
We can train rats to self-administer drug, and see if they increase or decrease their selfadministrative rate.
We can then see if the bar-pressing behavior accelerates or decelerates. If the rat increases
the bar-pressing rate to self-administer more drug, then the drug has positive reinforcing property.
If the rat decreases the bar-pressing rate, then the drug has no or negative reinforcing property.
This is evidence that suggests if the drug could lead to accelerated use or decelerated use,
even though we can’t ask what the rat feels or like, we can infer from the behavior whether the rat
prefers or dislike the drug.
How can you do a taste test on rats?
We simply do an operant conditioning on the rat, and provide the rat with two choices – one
for each food or drink, and observe which lever-bar the rat will press more for the reward.
Most often, this is how we find out what food our pet likes – by giving them alternatives, and
see which food they prefer better by eating it first, and avoid eating the other. This is how we do
a taste test on them.
How can knowledge be transferred in reinforcement learning?
It is transferred from the US to the CS.
Initially, before learning, the innate association is US leading to UR. CS will not lead to CR
initially. With training, i.e., repeated presentation of the paired US and CS, the association is
made between CS and CR. After learning (reinforcement), CS will lead to CR. Thus, knowledge
is transferred from US to CS subsequent to the reinforcement learning.
What is the new knowledge gained from reinforcement learning?
The new knowledge is the association between CS and CR.
Before learning, CS and CR are not related or associated. After learning CS and CR are
related. Presentation of CS will lead to expectation of UR, thus resulting in the production of CR.
Review Questions
1) What is positive reinforcement?
a) What is negative reinforcement?
b) What is the difference between them?
2) What is positive reinforcer?
a) What is negative reinforcer?
b) What is the difference between them?
Critical Thinking Questions
1) Why does your dog beg for food at the dinner table after you fed them food at the table?
a) Why does your dog beg for food when you open the refrigerator door after you fed them
food from the refrigerator?
2) Why don’t you like the people who yelled at you?
a) Why do kids like people who gives them candy?
3) Why do you like the teacher who gives you an A?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-22
Learning Mechanisms
a) Why do you not like the teacher who failed you?
4) How does giving examples help you learn?
a) Is example a CS, US, CR or UR?
b) How does knowledge be transferred using an example is the pairing?
Creative Thinking Questions
1) How do you train animals to listen to your words?
2) How do you train animals to stay within the perimeter of your yard?
a) What does electric fence do?
3) How do you teach children to listen to you?
a) Why do kids not listen to you when you yell at them?
b) How would you talk to them if you want them to listen to you?
4) How do you transfer knowledge from what you have learned to something new?
a) How does using example facilitate this transfer of knowledge?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-23
Learning Mechanisms
8.7 Extinction
Objectives
• Understand how reinforcement learning be reversed
Concepts to Learn
• Conditions for extinction
• Neutral reinforcer
• Relearning vs. unlearning
• Learning of new knowledge
• Pathological condition when extinction is incomplete
Extinction of behavior is reversing the behavior that was previously reinforced. It is
achieved by pairing the stimulus with a “neutral” reinforcer, i.e., neither rewarding nor punishing
repeatedly. The result is a change in expectation from expecting to be rewarded (or punished)
into expecting nothing will happen. It is learning new knowledge rather than deleting the
existing knowledge. That is, once it is learned (reinforced), it cannot be unlearned. Rather it can
only be relearned.
If the reinforced behavior cannot be extinguished, it can lead to mental disorders depending
on the reinforcer. If it is fear reinforcement, then it can lead to anxiety disorders, such as posttraumatic stress disorder, such that the fearful stimulus will always produce a fearful response. If
it is reinforced behavior, such as addiction, then the addictive response is permanent, i.e.,
whenever the stimulus is presented, it will always lead to the addictive response. If the belief
system is reinforced randomly, it can lead to false belief, and if it cannot be extinguished, it leads
to superstition.
Summary
Extinction is reinforcing the behavior with a neutral reinforcer such that the previously
reinforced CS will not lead to CR. It is relearning rather than unlearning. New concept is
formed to correct for the previously reinforced knowledge. If extinction is incomplete, it can
lead to pathological conditions in mental disorders.
Q&A
What is extinction of behavior?
It is reversing the behavior that has been reinforced previously.
When a response is reinforced (either by reward or punishment), that behavior is learned and
remains. In order to reverse that behavior, the behavior can be counter-reinforced such that the
response will no longer be associated with CS.
How can a behavior be extinguished?
It can use the same associative learning paradigm to pair the stimulus repeatedly with a
“neutral reinforcer,” i.e., a stimulus that is neither rewarding nor punishing.
That is, if the stimulus is no longer paired with either a reward or punishment, the response
will eventually extinguished, i.e., stop responding to the original CS.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-24
Learning Mechanisms
Using Pavlov’s dog as the example, to extinguish the salivation behavior, the bell ring is
paired with nothing, i.e., no food is presented. After a few trials, the dog will no long beg for
food or expect food will be delivered when hearing the bell.
Is extinction the same as unlearning or erased?
No, extinction actually learns a new concept, i.e., the new CS is no longer expected to be
associated with the reinforcer. It is re-learned rather than unlearned.
Once a behavior is reinforced (i.e., learned), that cannot be unlearned, i.e., it will not be
reversed back to the naïve state. The knowledge of CS leading to US cannot be erased. Even
though the behavior is changed after extinction by not responding to the CS, the expectation is
change. The expectation is changed from the expecting a reinforcer to expecting nothing, i.e., a
reinforcer will not be delivered.
In other words, once learned, the knowledge is kept. Extinction merely adds more knowledge
to it by saying that the previous knowledge was incorrect. This is new knowledge rather than
return back to the naïve state. In other words, it can only be re-learned.
Using Pavlov’s dog as the example, when the dog hears the bell, it will expect that no food
will be delivered, and stop expecting anything for it. The dog is no longer in a naïve state, i.e.,
before the reinforcement, the bell sound is merely a random event. After reinforcement, the bell
sound means food. After extinction, the bell sound means no food rather than a random sound.
In other word, the dog will recognize that the bell is teasing him after extinction, because he
will not expect any food from it.
What does extinction learn?
It actually learns a new concept – the new concept is that it will not be reinforced (rewarded
or punished) by the stimulus.
Rather than expecting the CR, it expects no CR when the CS is presented. In Pavlov’s
example, the dog will not expect any food when the bell is ring. So the bell takes on a different
meaning (no more expectation of reward) rather than random sound.
What are the consequences when extinction is incomplete?
This could lead to mental disorders, such as post-traumatic stress disorder if fear extinction is
incomplete, or addiction when the reinforced behavior is so potent (addicted) that cannot be
extinguished, or false belief system when the superstitious behavior is reinforced without
extinction.
The reason why they become a disorder is because the association is so reinforced that
whatever the behavior is learned, it cannot be extinguished. So even though association is
important for learning, when the learning is permanent and if it cannot be reversed, then it could
lead to pathological condition because the subsequent behavior is ineffective or inappropriate.
False belief is another example that the associations were reinforced by some random
phenomena, and those associations resist extinction even though those associations do not serve
any useful purposes.
Review Questions
1) How does extinction occur?
2) Is extinction the same as unlearning?
a) What new knowledge is gained from extinction?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-25
Learning Mechanisms
Analytical Thinking Questions
1) What are the stimuli for extinction?
a) What is the US during extinction training?
b) What is the CR after extinction?
Critical Thinking Questions
1) Is the mechanism for establishing extinction the same mechanism as classical or operant
conditioning?
a) What is the difference?
b) What is the similarity?
2) Once we learned something, can that be unlearned or relearned?
3) Is extinction of behavior good or bad?
a) Under what condition is it advantageous to survival?
b) Under what condition is it disadvantageous to survival?
4) What happens if learning is so strong that cannot be reversed?
a) Is learning good or bad?
Creative Thinking Questions
1) How can you undo superstitious behavior?
2) How can you change the false belief system?
3) How can you train your dog not to beg for food at the dinner table any more?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-26
Learning Mechanisms
8.8 Category of Learning
Objectives
• Understand the different types of cognitive learning in human
Concepts to Learn
• Reinforcement learning by corrective feedback
• Imitation learning by observing demonstration
• Instruction learning by conceptualization of abstract generalized rules and theories
• Differences and similarities of these types of learning
• Advantages and disadvantages in each type of learning
Cognitive learning in human can be classified into the categories of reinforcement learning,
imitation learning and instruction learning. Reinforcement learning learns by receiving the
feedback whether the final action is correct or not (rewarded or punished) rather than providing
the feedback in each step of the sequence. That is, the response is reinforced by indicating right
or wrong without explaining why it is correct or incorrect. The feedback is given only for the
final resulting action. Reinforcement learning can be done with or without a “teacher,” i.e.,
supervised or unsupervised. When it is supervised by a “teacher,” then the teacher delivers the
feedback by a positive or negative reinforcer indicating the correct or incorrect answer. When it
is unsupervised, self-learning occurs by observing/receiving the natural consequences of the
action. If the action is detrimental, it is negatively reinforced. If the action leads to pleasurable
or rewarding experience, it is positively reinforced.
Reinforcement learning is one of the most basic types of learning. It is not as effective in
learning complex concepts as the other two types of learning, due to the lack of feedback in the
intermediate steps or sequence of action that leads to the final result. But once reinforced, the
reinforcement is often very potent that resists extinction. Thus, it can interfere with relearning or
correction of erroneous knowledge or behavior if the extinction is incomplete.
Imitation learning is learned by following the example (sequence of steps) demonstrated by
the tutor. It is essentially mimicry by “copy-catting” the tutor’s behavior (or sequence of action).
Imitation often involves motor practice of the same repertoire of behavior in various different
ways. By practicing the variety of behavioral sequence of action, generalization can be made
about the conceptual understanding of the sequence of action. This learning is done by
observation or demonstration of actual behavior rather than done by following abstract
instructions. Play learning is an example of imitation learning that is used by most animals in
acquiring complex behavior.
Instruction learning is acquired by listening or reading the instructions (procedures) of
explaining how things work rather than observing the demonstration of the actual process. It is
the abstract learning in which abstraction of the general principles is delivered rather than
presenting the actual physical process as in imitation learning.
Summary
Reinforcement learning learns by receiving the feedback of the correctness of the final action
without the feedback on the intermediate steps or explanation or demonstration of the sequence
of action.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-27
Learning Mechanisms
Imitation learning learns by observing and mimicking the sequence of actions demonstrated
by the tutor. Subsequent action is learned by making variations of the same presented sequence
to generalize the result. It is learning by example.
Instruction learning learns by listening or reading the abstract explanation of concepts based
on the declaration of the procedures (abstract rules or theories) rather than demonstrating them
with concrete examples.
Q&A
What are the different categories of cognitive learning used by human?
They are:
• reinforcement learning
• imitation learning
• instruction learning
Cognitively, reinforcement learning is type of learning that receives the feedback of whether
the response is correct (by reward) or incorrect (by punishment).
Imitation learning is mimicry and observational learning. It is learned by observing the
demonstration.
Instruction learning is the procedural (or declarative) learning that learns by someone
explaining the concepts.
How does reinforcement learning learned cognitively?
It receives the feedback of whether the response is a correct action or not.
Reinforcement learning essentially learns by receiving the feedback of its action. If the
action is right, the feedback is provided by the positive reinforcement. It is rewarded for it.
The reward may be a direct feedback or indirect feedback. The reward can be delayed. The
reward can be virtual too, i.e., it does not need to be a physical reward, but rather a rewarding
expectation. (We will see later that the expectancy of the reward can become a potent reinforcer,
especially in addictive behavior formation.)
If the action is wrong, the feedback is provided by the negative reinforcement. It is punished
for it. The punishment may also be direct or indirect feedback too. It can also be delayed or
virtual. It is the anticipation of punishment that produces anxiety and fear.
Does the reinforcer feedback need to be immediate, direct or physical?
No, the reinforcer can be immediate or delayed; direct or indirect; physical or virtual
feedback.
When the feedback is delayed, it can produce apprehension or expectation.
When the feedback is an indirect feedback, then the reinforcer can cascade from other
reinforcer.
When the feedback is virtual, it is often created by the expectation or anticipation of the
reward/punishment rather than the physical reward/punishment. If it is punishment, it produces
anxiety. The reward can be virtual, such as a praise or getting a good grade. It does not need to
be physical reward. Similarly, the punishment can be virtual too, such as reprimand or scolding.
What is delayed gratification?
It is the delay in delivery of the reward although the expectation of reward is still present
immediately.
Reprinted from Neuropsychopharmacology by David Tam.
8-28
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
Learning Mechanisms
In other words, the behavior is still reinforced by the immediate virtual reward even though
the real reward is delivered subsequently. The virtual reward is the expectancy of the reinforcer
at some later time. If the expectancy is not present (i.e., if the virtual reward is not present), then
there will not be any reinforcement learning.
What is rote learning?
It is essentially reinforcement learning based on the feedback of what is correct without
explanation of the concepts or the sequence of action that leads to the result.
Rote learning is learned by reinforcing the correct response by memorization of arbitrary
association rather than associating relevant conditions that are appropriate or related to the task.
It is usually devoid of context.
What is imitation learning?
It is mimicry or observational learning.
It learns by observing a demonstration. It is essentially “copy cat” learning. The sequence of
action is learned by observing how others execute the task, and repeats similar actions.
Most animals learn complex behavior by imitation, especially by play and practice the
imitated behavior. They observe the tutor’s sequence of action, and repeat themselves.
What are examples of imitation learning?
Some of the examples are: play, spoken language acquisition, demonstration, and showing
examples.
Play learning is a form of imitation learning in which one also practices what is observed, and
follows the sequence of action of the “tutor”. Most animals learn complex task by imitating their
parents (who act as the tutor).
Language acquisition is a classic example of imitation learning. The child merely mimics
and repeats the sound produced by the parent (or tutor). In fact, other animals, such as birds, also
learn how to vocalize songs by mimicking the sound of the tutor bird. Birds from different
geographical regions also sing with different dialect in their vocalization due to the imitation of
the regional differences.
Demonstration of a concept in the laboratory or in a classroom is also another example. The
demonstration of how to cook in a cooking class is an example.
Using examples to demonstrate a concept is part of the play imitation learning in which the
play act is an example of future real-life example, such as practicing prey and defense for
animals.
What is instruction learning?
It is the procedural learning that is learned by explaining the instructions or concepts of how
to solve a task. It is declarative rather than demonstrating it.
Rather than learn from the demonstration of the sequence of action, it learns from following
the explanation conceptually. It is learned based on abstract rules and procedures of the action
rather than learning by observing the physical demonstration.
Instruction learning is usually abstract learning based on the abstraction of the sequence or
theory rather than learn by demonstrating concrete examples (application of the theory).
What are examples of instruction learning?
Some of the examples are: “do as I say, not as I do,” rules and regulations, abstract theories,
and most textbook descriptions.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-29
Learning Mechanisms
Most of classroom lectures are instruction learning. It describes the abstraction of the
concepts or the procedure for a sequence of action rather than
Which type of learning is least effective in acquiring complex skills?
Reinforcement learning is the least effective because it only provides the feedback of right or
wrong, but does not provide the sequence of action needed to accomplish the task.
Because complex tasks and complex concepts usually requires sequence of action, either
demonstration of such sequence or explanation of such sequence will accelerate the process.
Reinforcement learning only provides feedback of the end-result of the action rather than the
sequence itself. That is why it is not as effective as the other types of learning.
Which type of learning is the strongest learning that resists extinction?
Reinforcement learning is the strongest learning once learned, and it resists extinction,
especially in addictive behavior.
Reinforcement learning can be a powerful form of learning in which whatever action is
rewarded (or punished), that behavior is reinforced strongly independent of whether such action is
appropriate or not. In other words, if the action is learned incorrectly, it will resist extinction, i.e.,
it gets stuck, and forms superstitious behavior.
Addictive behavior is an example of such phenomenon in which the behavior is strongly
reinforced, and it resists extinction, even though the behavior is an inappropriate behavior
contextually.
Addiction is, in fact, an arbitrary reinforced behavior that is reinforced by the potent
reinforcer (the addictive substance).
What type of learning is most efficient in learning complex concepts and skills?
Imitation and instruction learning are equally effective in acquiring complex skills and
concepts.
Imitation is more natural in the acquisition process because it is evolutionary acquired by
most animals. Instruction learning is a recent phenomenon in evolution, which exists in higheranimals. It short-circuits the need to demonstrate with physical examples and learns from the
result of the generalization rather than deriving its general principles and concepts to produce the
complex behavior or acquiring the complex skills.
What is the difference between imitation learning and instruction learning?
Imitation learning derives the generalization by practicing whereas abstraction of the
generalization is presented in instruction learning rather than demonstrated using instances
(examples) applying to the generalized concept.
The difference is that generalization is derived from the variations of action in imitating the
tutor whereas generalized result is given to the learner in instruction learning rather than
demonstrating it with actual examples.
For instance, rules and regulations are abstract generalization when presented as rules and
regulation. Puritan behavior is demonstrating the rules without generalization, and allowing the
learner to generalize what is desirable behavior and what is not.
Acquiring language is an example to illustrate the difference. Children learn to speak the
native language by imitation rather than learning the grammatical or syntactical rules. Children
generalize the grammatical rule by observation and practicing. The imitation is further reinforced
by the feedback when interacting with other speakers.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-30
Learning Mechanisms
On the other hand, learning a language in school setting is often done by instruction learning
by explaining the grammar and syntax rather than letting the learner derives such rules by
generalization.
Using language as an example, because spoken language is also reinforced in the imitation
process, accent is also acquired when the initial vocalization is reinforced so strong that it cannot
be extinguished later on in acquiring the phonemes of another language.
Review Questions
1) What are the differences between reinforcement, imitation and instruction learning?
2) What are the similarities between reinforcement, imitation and instruction learning?
3) Which learning requires demonstration?
4) Which learning requires explanation?
5) Which learning requires direct reinforcer feedback?
6) Which learning takes the most time or repetition to acquire?
Analytical Thinking Questions
1) What are the factors that impede learning?
2) What are the factors that impede re-learning if it were learned incorrectly initiatlly?
Critical Thinking Questions
1) Which type of learning is used most often in school?
2) Which type of learning is used most often in athletic training?
3) Which type of learning is most efficient and natural in acquiring a language?
a) How do we develop accents and dialects in language?
4) If you were to learn a subject well, which is the best approach to learn it?
a) Which type of learning would be most natural?
b) Which type of learning would take the least time?
c) Which type of learning would be ineffective or inefficient?
Creative Thinking Questions
1) If you were to learn another language, what would be the most effective way to learn this
new language?
a) What are the pros and cons of learning a language using reinforcement, imitation and
instruction learning?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-31
Learning Mechanisms
8.9 Associative Memory
Objectives
• Understand how associative memory is formed
Concepts to Learn
• Acquisition by associative learning
• Associative memory consolidation
• Associative retrieval
• Advantages of memory formation
Associative memory is formed by associative learning. Once the association is acquired, the
association is consolidated into associative memory. Associative memory is recalled by the
same association process, i.e., CS leading to CR. Learning without memory is ineffective.
Memory cannot be formed without initial learning. Therefore, associative learning and
associative memory are inter-related and rely on each other to form. Associative learning and
associative memory essentially forms the contextual meaning of the environmental conditions in
which the events and behavioral responses are interrelated. This provides the mechanism in
which higher-learning can occur by transferring the knowledge through cascading the process.
Summary
Associative memory consolidates the associative learning such that associative recall can be
achieved to reproduce the prior learning experience. It allows for the storage of learned
knowledge so that subsequent actions can be altered based on prior association. The context in
which the stimuli are related is established by this process. Thus, conceptual knowledge of the
environmental conditions in which events occur can be acquired and retrieved.
Q&A
What is memory?
Memory is the storage of the learned knowledge.
In order for memory to be stored in animals, that information needs to be acquired (learned)
first. Thus, learning and memory are often interrelated to one another.
Technically, memory is not stored in any discrete form in the brain, but rather memory is
reconstructed by association on demand. Thus memory is dynamically recalled rather than stored
statically in the brain. Memory is distributed in a network of neurons, and retrieved by activating
the network of neurons.
What is associative memory?
It is the type of memory that is stored and retrieve by associating related events or stimuli.
One of the most common and basic types of memory in animals is associative memory in
which memory is stored and retrieved by association. That is, one thing leads to another – a
stimulus would lead to a response that the animal associated with.
This is different from reflex since the stimulus-response function is usually fixed (i.e., in-born
and hardwired). That is, given a stimulus, the response is always the same, such as the knee-jerk
reflex – stretching the tendon will always produce a leg extension.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-32
Learning Mechanisms
In contrast, the stimulus-response function in associative learning and memory is not in-born
or hardwired, rather it is acquired. For example, you can train the dog to expect food when
pairing it with the bell. But you can also train the dog to expect food when pairing it with whistle
or light-flash. Thus, the stimulus can be a bell-ring, a whistle or a light-flash; they don’t need to
the same stimulus, yet you can train the animal to produce the same response. Thus, the stimulusresponse function is different depending on the association (pairing) you repeated to the animal.
How does associative memory formed?
Associative memory is formed by associative learning.
Associative learning is the first step. Learning without memory is useless because it will
have to acquire next time. Memory provides a means of retrieving the learned knowledge
without relearning it. Thus, learning and memory go hand-in-hand. One without the other is
ineffective.
What does associative learning and associative memory establish?
It establishes the context (environmental conditions) in which events occurs.
Before associative learning and associative memory are formed, events are static,
independent and unrelated (random). After associative learning and associative memory are
formed, events are dynamically connected by their contextual meaning, i.e., they are dependent
on each other. That is how we make connections between things when those events seem to be
random and unrelated before. Thus, the contextual significance of these events are formed.
What are the survival advantages of having learning?
The animal can learn to adapt to the environment based on the specific conditions in that
environment rather than pre-programmed by pre-existing assumptions about the environment (as
in reflex action).
Learning allows the animal to respond more appropriately to the environment if the
circumstances of the environment changed. Thus, this provides the animal with a form of
adaptation that is dependent on the environmental factors, allowing the animal to fit in the
environment based on the environmental condition (rather than pre-existing assumptions that is
hardwired and born-with.)
What are the survival advantages of having memory?
The animals would remember what it learned as well as remember where things were, such
as food source, location of predators, etc., so that it can predict the future outcomes based on past
history.
With memory, animals would be able to accumulate knowledge based on past experience
rather than having to learn it every time anew. It becomes more efficient to recall from past
experience and memory to predict the future (assuming that history predicts a future pattern).
With memory, animals often can use the remembered location of food in the past for
searching, rather than doing a random-search. It provides a more efficient way of finding food, as
well as reducing the chances of encountering the unexpected, such as predators, in the random
search process. It also saves more energy in the search process.
Similarly, animals would be able to predict the likelihood of where predators are if they
remember the location of the predators in the past. Analogously, animals would also be to find
mates easier if they remember the location of their own species in the past encounters.
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-33
Learning Mechanisms
What is associative recall?
It is the recall process in which stimulation of one stimulus triggers the recall of another.
The recall process is associative, i.e., one leads to another. With associative recall, events or
correlation can be retrieved from past experience. In other words, events are linked together, and
retrieval is by establishing the sequence or links between these correlations.
The retrieval is done by associating events that were associated together in prior associative
learning. The CS serves as a cue to retrieve the memory. Most often, when you can’t remember
something, but giving you a hint serves as a cue to trigger the conditions in which the associative
learning occurs, thus “ring the bell” and remind you of what had learned before.
That is what re-enactment in crime scene does – it provides the environment cues (CS) that
stimulate the associative recall process. Without the conditions of the stimulus, the memory that
had formerly unretrievable becomes retrievable again.
What is emotional memory?
Emotional memory is the memory is the recalled by the association of a specific emotional
event.
Emotion often serves as a modulator that primes (enhances) the associative learning and
associative memory process. Associative memory is often stored and recalled vividly when it is
under the influence of strong emotional state. This serves a survival advantage in which the
animal can acquire the circumstance in which the emotional response occurs, and responds faster
when it revisits similar scenarios by remembering the conditions and circumstance of the prior
experience.
Emotions and memory are interrelated to each other in animals since memory is often
consolidated with the association of an emotional event in animal (the neural circuit for emotions
and memory are interrelated to each other) such that animals can recall a specific circumstance in
which the emotional state is similar so that it can prepare for the same scenario more readily.
Why does learning require memory and memory requires learning?
If we learn without memory, what was learned would be forgotten. And vice versa, if we
don’t learn the new association, nothing new would be stored.
Thus, associative learning and associative memory are usually interrelated with one another.
In particular, associative learning establishes the correlation between two events (or stimuli), and
the retrieval of such correlation is often by association, i.e., stimulating one stimulus would lead
to the recall of the second stimulus.
Review Questions
1) How does associative memory establish?
2) How does associative recall accomplish?
3) What is the similarity and difference between associative learning and associative memory?
4) Can associative memory be established without associative learning?
Analytical Thinking Questions
1) How can associative recall be facilitated?
2) Is rote learning effective?
a) Is memorization without context effective in learning or memory formation?
b) Why is not effective in terms of association?
c) How does context come into play in associative learning?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-34
Learning Mechanisms
d) What is missing in rote learning?
Critical Thinking Questions
1) What does reminder do in terms of associative memory retrieval process?
2) Why do we tend to forget if the memory process is not refreshed constantly?
3) What is contextual memory?
Creative Thinking Questions
1) If you can’t remember an incident, what would you do to help you remember it?
2) How can you improve your memory when you study for a class?
a) What are the tricks that are effective in helping you consolidate the memory and recall
them?
b) What are the tricks that are ineffective in helping you make associative learning and
associative memory?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-35
Learning Mechanisms
Chapter Review Questions
1) What are the different stages of learning?
a) How does it progress from incompetent stages to competent stages?
2) How does associative learning occur?
3) How does reinforcement learning occur?
4) What is the difference in the response when a behavior is positively reinforced vs. negatively
reinforced?
5) What is the difference between operant conditioning and classical conditioning?
6) What is the difference between conditioned and unconditioned stimuli?
7) What is the difference between conditioned and unconditioned responses?
8) Which type of reinforcement learning produces seeking behavior?
9) Which type of reinforcement learning produces avoidance behavior?
10) How does extinction occur?
Chapter Analytical Thinking Questions
1) Does operant conditioning use the same biological mechanism as classical conditioning in
the learning process?
2) Why does it usually take longer time to learn using operant conditioning than classical
conditioning?
3) Which type of conditioning requires the knowledge of end-result in the training process –
operant conditioning or classical conditioning?
a) Which one requires a “teacher”?
4) How does avoidance behavior be conditioning?
5) What is the advantage of having adaptation in the real world environment?
a) In what circumstances does sensitization or habituation not advantageous in the real
world environment?
b) How does sensitization (or desensitization) to a drug advantageous (or disadvantageous)
to normal behavior?
Chapter Critical Thinking Questions
1) Are some mental disorders (such as anxiety disorder, posttraumatic stress disorder) due to the
inability to extinct fear conditioning?
Chapter Creative Thinking Questions
1) How can you design a reinforcer for yourself so that you can learn this class better?
2) How can you be more aware of (identify) the negative reinforcer for learning in this class that
could help you not to repeat the avoidance behavior from studying?
3) How can you design a cure (or a drug) to treat patients with anxiety disorder (such as
posttraumatic stress disorder) by using the extinction technique?
a) How does reenactment of the traumatic event in virtual reality environment without
pairing it with any aversive stimuli help reduce the anxiety responses?
b) How does a drug that enhances extinction improve recovery from PTSD?
4) How can you un-learn or re-learn something when the original learned behavior was
inappropriate?
Chapter Big Picture Questions
1) If you want to accelerate your learning in this class in a short time and retain it, which type of
learning would you use – reinforcement learning, imitation learning or instruction learning?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-36
Learning Mechanisms
a) Which is more effective?
b) Which is more potent?
Chapter Application of Knowledge
1) How can you improve the consolidation of your associative memory in learning the materials
in this class?
2) How can you improve the recall of your associative memory in the materials of this class?
3) If you have test anxiety, what conditioning would you use to reduce the anxiety?
Chapter Small Group Discussion
1) What is the best way to learn?
a) Discuss the advantages and disadvantages of each type of learning methodology.
b) Which type of learning is more primitive than others?
c) Which type of learning has more lasting effects?
d) If someone learned the class material incorrectly before, what are the different learning
methodology to unlearn the erroneous association?
Reprinted from Neuropsychopharmacology by David Tam.
Copyright © 2010 by David Tam.
Further reproduction is prohibited without the permission of Great River Technologies, Dubuque, IA.
8-37