Download Decision Making Theories and Neuroscience

Decision Making Theories in
Neuroscience
Alexander Vostroknutov
October 2008
Choice in the brain
From Sugrue, Corrado and Newsome
Nature Neuroscience, 2005, Vol 6, May 2005
• Weak motion – chance performance; strong motion – optimal performance
• “Decision making” area should aggregate noisy signal and suggest the
decision
Monkey brain
From Sugrue, Corrado and Newsome
Nature Neuroscience, 2005, Vol 6, May 2005
•
•
•
•
•
LIP area – part of visuo-motor pathway
Its activation is covaried with choice AND modulated by movement strength
during motion
not purely sensory (mistake trials);
not purely decision oriented (modulated by strength of movement)
LIP is where “deliberation” takes place
Three processes of choice
From Bogacz, 2007,
TRENDS in Cog. Sci., Vol 11(3)
• Neurons in Visual cortex provide evidence for alternatives (noisy)
• Intergation takes place (in LIP), removes noise
• The choice is made once certain criterion is reached (confidence level)
Optimal decision making
• This procedure can be formulated as a statistical problem
• Statistical test to optimize decision making
• It can be tested whether the brain implements
optimal test (evolution)
• Links optimal tests with neurobiology (basal ganglia)
• and behavior (speed-accuracy tradeoff)
Optimality criterion
• Sequential Probability Ratio Test (Wald)
• A procedure to distinguish two distributions H0: p=p0 and
H1: p= p1 given a sequence of observations {yn}
• Sum log-likelihood ratios of incoming data and stop once
threshold is reached:
Sn = Sn-1 + log(p0(yn)/p1(yn))
• Given fixed accuracy, SPRT requires the least expected
number of observations
• Animals would be interested in implementing SPRT:
minimizes reaction time
Diffusion model (2 alternatives)
Input A
A-B
Input B
Integrator (I)
I > 5: choose A
I < -5: choose B
• Is there simple way to implement SPRT?
• Integrator accumulates evidence about the difference of inputs
In = In-1 + An - Bn
• Once threshold is reached (|In| > 5), choose A or B
Diffusion Model is optimal
• Continuous limit of SPRT can be described by Wiener
process with drift (Bogacz et al, 2006)
dy = (mA-mB)dt + cdW
• Choose once threshold is reached
(assumed: A and B are normal, same variance)
•
•
•
•
mA is mean of alternative A
This is exactly Diffusion Model!
Thus DM implements SPRT
Given fixed accuracy, DM has the best reaction time
(important for animals)
• Simple to implement in neural networks
(requires only addition and subtraction)
Connection to the brain
• How can we test whether something like diffusion model
is implemented in the brain?
• We have evidence (LIP) of the presence of intergators
• We need evidence for the presence of “criterion
satisfying” region
• Good candidate: basal ganglia
• They resolve competition between cortical and subcortical systems that want expression
• Inhibit all actions; the “winning system” is allowed to
express itself through disinhibition
Diffusion Model (n alternatives)
Input A1
A1 – ln[exp(A2)+exp(A3)]
I1
Input A2
A2 – ln[exp(A1)+exp(A3)]
I2
Input A3
A3 – ln[exp(A1)+exp(A2)]
I3
•
•
•
•
DMn implements optimal MULTI SPRT
Uses exponentiation
Neurons which exponentiate are rare
Good evidence for Diffusion Model
choose whenever
any of these is
higher than
threshold
Evidence
• Bogacz, 2007 reports studies that demonstrate that
neurons in subthalamic nucleus (STN) perform
exponentiation
• STN targets output nuclei of basal ganglia, that “decide”
on which system to allow to act
More evidence
Diffusion Model and Economics
• Difficult to perceive the difference between n and n+1 grains of
sugar
• Non-transitivity of indifference
• Beyond the scope of classical preferences model
• DM suggests a simple and natural way to model this
Diffusion Model and Economics
A, B available:
A
80%
B
20%
price
A
B
A, B, C available:
A
50%
B
50%
C
quality
• Violation of Weak Axiom of Revealed Preference
(recent evidence: Kroll, Vogt, 08)
•
•
•
•
Again, DM with 3 alternatives gives simple explanation
Prospect Theory, Regret do not account for this
Can save the “existence” of underlying preferences
Additional prediction of DM: smaller reaction time in second case
Diffusion Model and Economics
S1 = $1
R1 = ($5, 0.1; $1, 0.89; $0, 0.01)
S2 = ($1, 0.11; $0, 0.89)
R2 = ($5, 0.1; $0, 0. 9)
EU maximizer prefers S’s or R’s
Evidence: S1 > R1 and R2 > S2
• Allais paradox: violation of Expected Utility maximization
• In choice between S1 and R1: information about S1 is accumulated
much faster than about R1: high chance of hitting S1 threshold
• In choice between S2 and R2: information accumulates at
comparable speeds, R2 is almost like S2, only with $5 instead of $1,
high chance to hit R2 threshold first
• Additional prediction of DM: reaction time in S1-R1 choice is shorter
than in S2-R2
• No need to get rid of Expected Utility
Conclusion
• It seems like there is evidence that Diffusion Model is
implemented in the brain
• Sensory inputs are integrated in the respective pre-motor
regions (LIP)
• Basal ganglia check which option should be chosen by
comparing competing “integrators” to the threshold
• Important for economists. DM explains with ease many
different phenomena