Download No Slide Title

The Connectionist Approach
(Neural Networks)
• The brain should be modeled with neuronlike structures
• Focus on parallel processing (even if you
have to fake it with a serial computer)
• It can be applied to cognitive processes if
the process is heavily rule-based (language)
Fuzzy Nature of Neurons
• Q: How can perceptions be consistent if neurons
aren’t?
• A: Perceptions arise from many active neurons
and the responses (opinions) are averaged.
– No neuron has the final say
– No neuron is indispensable
Access by Content
• Connectionist networks allow access by
content, rather than address.
– Phone book: Access by address
• You can find a number if you know name
• You can’t find a name if you know a number
• You cant find a number if you know an address
– Access by content allows you to work either
way. Any bit of info can lead to all other bits
Distributed Information Storage
• The “benign” qualities of a tumor live in the same set of
weights and connections as the malignant qualities
Access by Content
• Each piece of information is linked to the other
pieces and, therefore, can activate them (bring
them into awareness).
• Access is fault-tolerant, so small errors in input
can still lead to a correct solution. A
distributed system will look for a “best-fit.”
– An erroneous spelling will not lead to a correct
phone number
What’s so “neural” about it?
1. The basic computational operation in the brain
is one neuron passing a information related to
the sum of it’s input to other neurons
What’s so “neural” about it?
2. Learning changes the strength of the
connections between neurons and thus the
influence that one has on another
What’s so “neural” about it?
3. Cognitive processes (Thinking: memory,
speech, problem solving etc.) involve the basic
computations being performed in parallel by
large numbers of neurons
Not all Cognitive Models are
Neurally Based
• This memory model
just explains the
hypothetical links
between different types
of memory.
• It’s useful in some
cases, but makes no
attempt to link
memory with the
physiology of the brain
Five Assumptions About the Brain
on which Connectionist Models are Based
1. Neurons Integrate Information
–
Collect information from one set of neurons and
pass the combined message on to another set
Five Assumptions About the Brain
on which Connectionist Models are Based
2. Neurons Pass Information about the level of
their input.
–
The information being sent to the axon is
proportional to the activity at the cell body
Five Assumptions About the Brain
on which Connectionist Models are Based
3. Brain structure is layered
Hidden Units
Five Assumptions About the Brain
on which Connectionist Models are Based
4.
5.
The influence of one neuron on another depends
on the strength of the connection between them.
Learning is achieved by changing the strengths
of connections between neurons.
History
• McCulloch and Pitts - Logical operations
with neuron-like components (1943)
Hot Sensor
Cold Sensor
“I feel Hot”
“I feel Cold”
• A static model - Like your calculator
The Connectionist Approach
(Neural Networks)
• Donald Hebb (1949) and synaptic change
– We knew synapses had to change
– Hebb showed us how (in some cases)
Rosenblatt’s Perceptron (1958)
• A simple neural network inspired by
the human retina
Connectionist Terms
• Although both neurons A and B have excitatory
synapses with C, Activity in cell A will have a
bigger effect on C than the same amount of
activity in cell B.
A
C
Symbols and Equations
• Activity (ai) The activity that is passed on
(through the weights) to the other units
– If the Netinput to a unit does not exceed the
threshold, activity is zero
ai
Symbols and Equations
• Weight (wij) Unit j influences unit I by
passing information about its activity
level.
aj
wij
ai
Symbols and Equations
• Input: The input from unit j to unit i is
the product of the activity of unit j (aj)
and the weight of the connection
between them (wij).
• inputij=ajwij
aj
wij
ai
Symbols and Equations
• NetInput: The total input to unit i (netinput)
is found by summing the inputs from all the
units which send input to it.
• netinputi= Σ ajwij
aj
wij
ai
Symbols and Equations
• Activation Function: How is netinputi
converted to activityi?
Linear
Sigmoid
netinputi
netinputi
netinputi
Activity (ai)
Binary
Rosenblatt’s Perceptron (1958)
• Used Hebb’s principles and a form of
teaching to create a learning network
Rosenblatt’s Perceptron (1958)
Stimulus
Activity in Inputs
The Rules of Learning
No is no. No is always No.
If they say, “No” it means a thousand times “No!”
Rosenblatt’s Perceptron (1958)
Stimulus
Activity in Inputs
Mistakes will result in
changes in weights
Perceptron Learning Rule
• “Weight gain and loss”
• Δ wij = [ai(desired)-ai(obtained)]ajε
– No change if the outcome matches the desired result
– More activity leads to greater change
– Change can be accelerated ε
aj
wij
ai
Rosenblatt’s Perceptron (1958)
• A one-layer neural network that learned the
alphabet with simple “Yes” or “No” feedback.
PDP as a Model of Cognition
•
•
•
•
Immune to minor damage
Work when input is noisy or incomplete
Knowledge is distributed
Computations take place in parallel
Conclusions are based on consensus!
Human vs. Network
Network Performance
Human Response Time
Exception
570
HAVE
NAVE
GAVE
6
4
2
530
High
Exception
Regular
Regular
550
8
SUAVE
Mean Squared Error
Mean Naming Latency
590
Low
High
Low
PDP vs. Traditional Computer
• Processing is parallel, not serial
– In a serial system, one broken link stops
everything
• Information is distributed, not local
Connectionist Models are far
from Perfect
• We don’t completely understand how the brain is
wired
– Billions of neurons, trillions of synapses
• We don’t completely understand how and where all
neurotransmitters are used (Inhibition/Excitation).
• Connectionist models are simplified. However, they
are computational and easily tested. The output is
quantitative and can be directly compared to the
quantitative results from behavioral studies – if one
doesn’t work, it should be modified or discarded.