Download HE Robbins06

Meeting the Future in Managing
Chronic Disorders: Individually
Tailored Strategies
S.A. Murphy
Herbert E. Robbins Collegiate Professorship in
Statistics Lecture
November 14, 2006
1
Outline
–
–
–
–
Three apparently dissimilar problems
Myopic decision making
Constructing strategies
Challenges
• Unknown, unobserved causes
• Small, expensive data sets
– Discussion
2
Three Apparently Dissimilar
Problems
– Artificial Intelligence: Autonomous Helicopter
Flight
– Management of Chronic Mental Illnesses
– Management of a Welfare Program
3
Artificial Intelligence
• Autonomous Helicopter Flight
– Observations: characteristics of the helicopter (position,
orientation, velocity, angular velocity, ….),
characteristics of the environment (wind speed, wind
angle, turbulence….)
– Actions/treatments: cyclic pitch (causes
forward/backward and sideways acceleration), tilt angle
of main rotor blades (direction), tail rotor pitch control
(turning)
– Rewards: Closeness of helicopter’s flight path to the
desired path; avoidance of crashes(!)
4
Andrew Ng’s Helicopter: http://ai.stanford.edu/~ang/
5
The Management of Chronic Mental
Illnesses
• Treating Patients with Opioid Dependence
(heroin)
– Observations: individual characteristics (withdrawal
symptoms, craving, attendance at counseling sessions,
results of urine tests….), characteristics of the
environment (housing, employment.…)
– Actions/treatments: methadone dose, amount of
weekly group counseling sessions, daily dosing time of
methadone, individual counseling sessions, methadone
taper
– Rewards: minimize opioid use and maximize
health/functionality, minimize cost
6
http://www.nida.nih.gov/perspectives/vol1no1.html 7
Management of a Welfare Program
• “Jobs First” Program in Connecticut
– Observations: individual characteristics (assets, income,
age, health, employment), characteristics of the
environment (domestic violence, incapacitated family
member, # children, living arrangements…)
– Actions/treatments: child care, job search skills
training, amount of cash benefit, medical assistance,
education
– Rewards: maximize employment/independence.
8
9
The Common Thread: Multi-Stage
Decision Making
• Observation, action, observation, action,
observation, action,…………………….
• A strategy tells us how to use the
observations to choose the actions.
• We’d like to develop strategies that
maximize the rewards.
10
Myopic Decision Making
11
Myopic Decision Making
• In myopic decision making, decision makers use strategies
that seek to maximize immediate rewards. Problems:
– Ignore longer term consequences of present actions.
– Ignore the range of feasible future actions/treatments
– Ignore the fact that immediate responses to present actions
may yield information that pinpoints best future actions
•
(A strategy tells us how to use the observations to choose
the actions.)
12
Autonomous Helicopter Flight
The helicopter has veered from flight plan.
•
Myopic action: Choose an acceleration and direction that will
ASAP bring us back to the flight plan.
•
The result: The myopic action results in the helicopter overshooting
the planned flight path and in drastic situations may lead to the
helicopter cycling out of control.
•
The mistake: We did not consider the range of actions we can take
following the initial action. The ability to slow down is
mechanically limited.
•
The message: Use an acceleration that will not return us as quickly
to the planned flight path but will take into account the ability of the
helicopter to slow down and reduce the overshoot.
13
Treatment of Psychosis
•
Myopic action: Choose a treatment that reduces psychosis for
as many people as possible.
•
The result: Some patients are not helped and/or experience
abnormal movements of the voluntary muscles (TDs). The
class of subsequent medications is greatly reduced.
•
The mistake: We should have taken into account the variety
of treatments available to those for whom the first treatment is
ineffective.
•
The message: Use an initial medication that may not have as
large a success rate but that will be less likely to cause TDs.
14
Treatment of Opioid Dependence
•
Myopic action: Choose an intensive multi-component
treatment (methadone + counseling + behavioral
contingencies) that immediately reduces opioid use for as
many people as possible.
•
The result: Behavioral contingencies are
burdensome/expensive to implement and many people may
not need the contingencies to improve.
•
The mistake: We should allow the patient to exhibit poor
adherence prior to implementing the (negative) behavioral
contingencies.
•
The message: Use an initial treatment that may not have as
large an immediate success rate but carefully monitor patient
adherence to ascertain if behavioral contingencies are
15
required.
Constructing Strategies
16
Basic Idea for Constructing a Strategy:
Move Backwards Through Time.
Observations
Treatment
Time 1
Observations
Time 2
Treatment
Time 2
Reward
Time 3
(Pretend you know the distribution of all outcomes!)
17
Challenges
18
Goal
• Combine theory and data in a principled
fashion to construct a good strategy.
• In A.I. scientists combine mechanistic
theories with data from experiments to
construct strategies that maximize the
rewards.
19
Artificial Intelligence
• Scientists who construct strategies in autonomous
helicopter flight use mechanistic theory (physical laws:
momentum=m*v, W=F*d*cos(θ)…) to model the
interrelationships between observations.
– Scientists know many (most?) of the causes of the
observations and know how the observations relate to one
another.
• Scientists can quickly evaluate the strategies (within a
matter of months).
20
Comparatively Less Known
Mechanistic Models in
Behavioral/Social/Medical Sciences
• Scientists who want to use data on
individuals to construct treatment strategies
must confront the fact that non-causal
“associations” occur due to the unknown
causes of the observations.
21
Conceptual Structure in the
Behavioral/Social/Medical Sciences
Unknown
Causes
Observations
Unknown
Causes
Treatment
Time 1
Observations
Time 2
Treatment
Time 2
Reward
Time 3
22
Unknown, Unobserved Causes
(Incomplete Mechanistic Models)
Maturity/
Decision
to join "Adult"
Society
Unknown
Causes
+
+
Binge Drinking
Treatment
Time 1
Binge Drinking
Time 2
Counseling
Time 2
Grades
Time 3
23
Unknown, Unobserved Causes
(Incomplete Mechanistic Models)
• Problem: Non-causal associations between
treatments (here counseling) and rewards
are likely.
• Solution: Construct strategies using data
sets in which randomization is used to
assign treatments to students. This breaks
the non-causal associations yet permits
causal associations.
24
Unknown, Unobserved Causes
(Incomplete Mechanistic Models)
Maturity/
Decision
to join "Adult"
Society
Unknown
Causes
"+"
Observations
Treatment
Time 1
Binge Drinking
Time 2
Counseling
Time 2
Grades
Time 3
25
Unknown, Unobserved Causes
(Constructing Sequences of Treatment)
Unknown
Causes
Observations
Unknown
Causes
Treatment
Time 1
Observations
Time 2
Treatment
Time 2
Reward
Time 3
26
Unknown, Unobserved Causes
(Incomplete Mechanistic Models)
Maturity/
Decision
to join "Adult"
Society
Unknown
Causes
+
-
Binge Drinking
Yes/No
Counseling
re Health
Consequences
-
Binge Drinking
Yes/No
Time 2
Sanctions
+ counseling
Grades
Time 3
27
Unknown, Unobserved Causes
(Incomplete Mechanistic Models)
Unknown
Causes
High SAT
Scores
+
+
Observations
Student
is an superior
athlete
+
Student
admitted to
University
Treatment
Time 2
Grades
28
Unknown, Unobserved Causes
(Incomplete Mechanistic Models)
• The problem: Even when treatments are
randomized, non-causal associations occur in the
data.
• The solution: Statistical methods for constructing
strategies must be conducted in stages as opposed
to “all-at-once.” Statistical methods should
appropriately “average” over the non-causal
associations between treatment and reward.
29
Summary of Solutions To Causal
Problems
• Subjects in your data should be
representative of population of subjects.
• Experiments should randomize actions.
• Develop statistical methods that avoid being
influenced by non-causal associations yet
help you construct the strategy.
30
Expensive Data on a Limited
Number of Individuals
• Scientists who want to use data on
individuals to construct treatment strategies
must provide measures of confidence and
also evaluations of alternative treatment
strategies.
• Methods for constructing strategies are nonsmooth.
31
Basic Idea for Constructing a Strategy:
Move Backwards Through Time.
Observations
Treatment
Time 1
Observations
Time 2
Treatment
Time 2
Reward
Time 3
32
Expensive, Limited Data on
Individuals
• In order to provide measures of confidence and
comparisons of strategies, the statistical methods
for constructing strategies must be regularized.
• A number of statisticians are working hard on this
open question.
• This problem will be one of the foci of the SAMSI
program in June, 2007: http://www.samsi.info/
33
Some Experiments
34
ExTENd
• Ongoing study at U. Pennsylvania (D.
Oslin)
• Goal is to learn how best to help alcohol
dependent individuals reduce alcohol
consumption.
35
ExTENd
When to
abandon
initial treatment?
Observation
Secondary T xt
TDM +
Treatment
is working
R
Prescription
Prescription
Counseling
Quickly
Treatment is
not working
R
Naltrexone +
Counseling
Provide
R
Naltrexone
Treatment
is working
TDM +
R
Prescription
Prescription
Slowly
Counseling
Treatment is
not working
R
Naltrexone +
Counseling
36
Adaptive Treatment for ADHD
• Ongoing study at the State U. of NY at
Buffalo (B. Pelham)
• Goal is to learn how best to help children
with ADHD improve functioning at home
and school.
37
ADHD Study
A1. Continue, reassess monthly;
randomize if deteriorate
Yes
8 weeks
A. Begin low-intensity
behavior modification
A2. Add medication;
bemod remains stable but
medication dose may vary
AssessAdequate response?
No
Random
assignment:
Random
assignment:
A3. Increase intensity of bemod
with adaptive modifications based on impairment
B1. Continue, reassess monthly;
randomize if deteriorate
8 weeks
B. Begin low dose
medication
AssessAdequate response?
No
Random
assignment:
B2. Increase dose of medication
with monthly changes
as needed
B3. Add behavioral
treatment; medication dose
remains stable but intensity
of bemod may increase
with adaptive modifications
38
based on impairment
STAR*D
• This trial is over and the data is being
analyzed (PI: J. Rush).
• One goal of the trial is construct good
treatment sequences for patients suffering
from treatment resistant depression.
www.star-d.org
39
40
Discussion
• The best management of chronic disorders
(poverty, mental illness, other medical conditions)
requires multi-stage decision making.
• Avoid myopic decision making!
– Allow for longer term effects of the treatment
– When comparing treatment options take into account
the effect of future treatments
– Appreciate the value of observing patients outcomes
such as adherence
• Experimental designs and statistical methods are
available.
41
This seminar can be found at:
http://www.stat.lsa.umich.edu/~samurphy/seminars/H.E.Robbins06.ppt
[email protected]
42
Role of the Statistician
• What kinds of data are most useful for developing
strategies?
• How do we use limited and expensive data to
construct good strategies?
• How do we evaluate strategies using the limited
data?
(A strategy tells us how to use the observations to
choose the actions.)
43
Unknown, Unobserved Causes
Unknown
Causes
Observations
Unknown
Causes
Treatment
Time 1
Observations
Time 2
Treatment
Time 2
Reward
Time 3
44
Unknown, Unobserved Causes
Unknown
Causes
Maturity
of Student
+
-
Binge Drinking
Treatment
Time 1
Frequent Drinking
Binge Drinking
Time 2
Treatment
Time 2
Grade
Time 3
45
Unknown, Unobserved Causes
• Problem: We recruit students via flyers
posted in dormitories. Associations between
observations and rewards are highly likely
to be (due to the unknown causes) nonrepresentative.
• Solution: Sample a representative group of
college students.
46