Download SP07 cs188 lecture 2..

CS 188: Artificial Intelligence
Spring 2007
Lecture 27: Neural Computation
5/1/2007
Srini Narayanan– ICSI and UC Berkeley
Announcements
 Reinforcement Learning Q/A
 Wednesday 6 PM, 306 SODA
 Helicopter control talk
 Thursday (4 -5:30 PM) 310 SODA
Sci-Fi AI?
What is AI?
The science of making machines that:
Think like humans
Think rationally
Act like humans
Act rationally
Thinking Like Humans?
 The Cognitive Science approach:
 1960s ``cognitive revolution'': information-processing
psychology replaced prevailing orthodoxy of
behaviorism
 Scientific theories of internal activities of the brain
 What level of abstraction? “Knowledge'' or “circuits”?
 Cognitive science: Predicting and testing behavior of
human subjects (top-down)
 Cognitive neuroscience: Direct identification from
neurological data (bottom-up)
 Both approaches now distinct from AI
 Both share with AI the following characteristic:
 The available theories do not explain (or engender)
anything resembling human-level general intelligence}
 Hence, all three fields share one principal direction!
Images from Oxford fMRI center
Basic Ideas
 Neural Computation
 The brain as a computing device
 Learning in the brain.
 Brain-based Computing
 Brain Machine Interfaces
BRAIN
Motor cortex
Somatosensory cortex
Sensory associative
cortex
Visual associative
cortex
Broca’s
area
Visual
cortex
Primary
Auditory cortex
Wernicke’s
area
Imaging the Brain
Sensory Systems




Vision (nearly 30-50% )
Audition (nearly 10%)
Somatic
Chemical
 Taste
 Olfaction
Motor Systems
 Locomotion
 Manipulation
 Speech
NEURON
Brains ~ Computers
 1000 operations/sec
 100,000,000,000
units
 10,000 connections/
 graded, stochastic
 embodied
 fault tolerant
 evolves, learns
 1,000,000,000
ops/sec
 1-100 processors
 ~ 4 connections
 binary, deterministic
 abstract
 crashes
 designed,
programmed (usually)
Computing in the Brain: Mirror Neuron in F5 (premotor cortex)
Approx 2 s:
Experimenter picks up food
Approx 4 s:
Monkey picks up food
Many human imaging studies showing activation of motor regions
(primary and secondary) during action perception.
Regions overlap with those engaged during action production.
Distributed frontal/parietal activation during viewing of
actions performed with mouth, hand, or foot.
All activations compared to rest baseline.
Mouth
(chewing, biting)
Hand
(grasping, pinching)
Foot
(kicking, jumping)
Buccino et al. 2001)
Observation of action activates premotor cortex in
topographic manner, consistent with motor
topography.
Foot Action
Hand Action
Mouth Action
a) no-object
b) w/ object
MEG (magnetoencephalography) study comparing pianists and nonpianists.
Pianists show activation in primary motor cortex when listening
to piano.
Activation is specific to fingers used to play the notes.
Colored region: MEG signal for pianists
minus non-pianists.
Significance of Mirror Neurons
 Action, Perception, Imagination, and
Understanding share a lot of the same
brain circuits.
 Question:
 How are these circuits learnt?
Models of Learning




Hebbian ~ coincidence
Reinforcement ~ delayed reward
Supervised ~ correction (perceptron, mlp)
Unsupervised-similarity
Long-term Potentiation (LTP)
Rapid and long-term increase in synaptic strength
resulting from the pairing of presynaptic activity with
postsynaptic depolarization
Synaptic Plasticity


Hebb’s Postulate: When an axon
of cell A... excites cell B and
repeatedly or persistently takes
part in firing it, some growth
process or metabolic change
takes place in one or both cells so
that A's efficiency as one of the
cells firing B is increased.
Slices of the hippocampus can be
removed and its CA1 neurons
studied in vitro with recording
electrodes. Rapid, intense
stimulation of presynaptic neurons
evokes action potentials in the
postsynaptic neuron. This is just
what we would expect from the
properties of synapses.
The Hebb rule is found with long term
potentiation (LTP) in the hippocampus
Schafer collateral pathway
Pyramidal cells
1 sec. stimuli
At 100 hz
Neural Correlates of RL
Prefrontal Cortex
Dorsal Striatum (Caudate, Putamen)
Parkinson’s Disease
 Motor control +
initialtion?
Intracranial self-stimulation;
Drug addiction;
Natural rewards
 Reward pathway?
 Learning?
Nucleus Accumbens
(Ventral Striatum)
Amygdala
Ventral Tegmental
Area
Also involved in:
 Working memory
 Novel situations
Substantia Nigra  OCD
 Schizophrenia
 …
Conditioning
Ivan Pavlov
= Conditional stimulus
= Unconditional stimulus
Response = Unconditional response (reflex);
conditional response (reflex)
Dopamine Levels track RL signals
Unpredicted reward
(unlearned/no stimulus)
Predicted reward
(learned task)
Omitted reward
(probe trial)
(Montague et al. 1996)
Current Hypothesis
Phasic dopamine encodes a reward prediction error
 Evidence
 Monkey single cell recordings
 Human fMRI studies
 Current Research
 Better information processing model
 Other reward/punishment circuits including Amygdala (for visual
perception)
 Overall circuit (PFC-Basal Ganglia interaction)
Neural Basis of Intelligence
 How does a system of neurons with
specific processes, connectivity, and
functions support the ability to think,
reason, and communicate?
Take CS 182 in Spring 2008.
Basic Ideas
 Neural Computation
 The brain as a computing device
 Learning in the brain.
 Brain-based Computing
 Determining cognitive states from imaging
data.
 Brain Machine Interfaces
Brain Machine Interfaces
 Sensory Prosthesis
 Brain Computer Interfaces from Brain
signals
Sensory Prosthesis
Visual Prosthesis
Cochlear Implants
BCI using EEG
EEG Control of a robot in a labyrinth
Decoding Cognitive Signals