Download Lecture13

Decision theory and Bayesian
statistics. Tests and problem solving
Petter Mostad
2005.11.21
Overview
• Statistical desicion theory
• Bayesian theory and research in health
economics
• Review of tests we have learned about
• From problem to statistical test
Statistical decision theory
• Statistics in this course often focus on estimating
parameters and testing hypotheses.
• The real issue is often how to choose between
actions, so that the outcome is likely to be as good
as possible, in situations with uncertainty
• In such situations, the interpretation of probability
as describing uncertain knowledge (i.e., Bayesian
probability) is central.
Decision theory: Setup
• The unknown future is classified into H possible
states: s1, s2, …, sH.
• We can choose one of K actions: a1, a2, …, aK.
• For each combination of action i and state j, we
get a ”payoff” (or opposite: ”loss”) Mij.
• To get the (simple) theory to work, all ”payoffs”
must be measured on the same (monetary) scale.
• We would like to choose an action so to maximize
the payoff.
• Each state si has an associated probability pi.
Desicion theory: Concepts
• If action a1 never can give a worse payoff,
but may give a better payoff, than action a2,
then a1 dominates a2.
• a2 is then inadmissible
• The maximin criterion
• The minimax regret criterion
• The expected monetary value criterion
Example
states
actions
No birdflu
outbreak
Small birdflu
outbreak
Birdflu
pandemic
No extra
precautions
0
-500
-100000
Some extra
precautions
-1
-100
-10000
Vaccination
of whole pop.
-1000
-1000
-1000
Decision trees
• Contains node (square junction) for each choice of
action
• Contains node (circular junction) for each
selection of states
• Generally contains several layers of choices and
outcomes
• Can be used to illustrate decision theoretic
computations
• Computations go from bottom to top of tree
Updating probabilities by aquired
information
• To improve the predictions about the true states of
the future, new information may be aquired, and
used to update the probabilities, using Bayes
theorem.
• If the resulting posterior probabilities give a
different optimal action than the prior
probabilities, then the value of that particular
information equals the change in the expected
monetary value
• But what is the expected value of new
information, before we get it?
Example: Birdflu
• Prior probabilities: P(none)=95%, P(some)=4.5%,
P(pandemic)=0.5%.
• Assume the probabilities are based on whether the virus
has a low or high mutation rate.
• A scientific study can update the probabilities of the virus
mutation rate.
• As a result, the probabilities for no birdflu, some birdflu,
or a pandemic, are updated to posterior probabilities: We
might get, for example:
P (none | high _ mutation)  80%
P ( some | high _ mutation)  15%
P (none | low _ mutation)  99%
P ( some | low _ mutation)  0.9%
P ( pand . | high _ mutation)  5%
P ( pand . | low _ mutation)  0.1%
Expected value of perfect
information
• If we know the true (or future) state of nature, it is
easy to choose optimal action, it will give a certain
payoff
• For each state, find the difference between this
payoff and the payoff under the action found using
the expected value criterion
• The expectation of this difference, under the prior
probabilities, is the expected value of perfect
information
Expected value of sample
information
• What is the expected value of obtaining updated
probabilities using a sample?
– Find the probability for each possible sample
– For each possible sample, find the posterior
probabilities for the states, the optimal action, and the
difference in payoff compared to original optimal action
– Find the expectation of this difference, using the
probabilities of obtaining the different samples.
Utility
• When all outcomes are measured in monetary
value, computations like those above are easy to
implement and use
• Central problem: Translating all ”values” to the
same scale
• In health economics: How do we translate
different health outcomes, and different costs, to
same scale?
• General concept: Utility
• Utility may be non-linear function of money value
Risk and (health) insurance
• When utility is rising slower than monetary value,
we talk about risk aversion
• When utility is rising faster than monetary value,
we talk about risk preference
• If you buy any insurance policy, you should
expect to lose money in the long run
• But the negative utility of, say, an accident, more
than outweigh the small negative utility of a policy
payment.
Desicion theory and Bayesian theory
in health economics research
• As health economics is often about making
optimal desicions under uncertainty,
decision theory is increasingly used.
• The central problem is to translate both
costs and health results to the same scale:
– All health results are translated into ”quality
adjusted life years”
– The ”price” for one ”quality adjusted life year”
is a parameter called ”willingness to pay”.
Curves for probability of cost
effectiveness given willingness to pay
• One widely used way of
presenting a cost-effectiveness
analysis is through the CostEffectiveness Acceptability
Curve (CEAC)
• Introduced by van Hout et al
(1994).
• For each value of the threshold
willingness to pay λ, the CEAC
plots the probability that one
treatment is more cost-effective
than another.
Review of tests
• Below is a listing of most of the statistical
tests encountered in Newbold.
• It gives a grouping of the tests by
application area
• For details, consult the book or previous
notes!
One group of normally distributed
observations
Goal of test:
Testing mean of
normal distribution,
variance known
Testing mean of
normal distribution,
variance unknown
Testing variance of
normal population
Test statistic:
Distribution:
standard normal:
N (0,1)
X  0
/ n
X  0
sx / n
t-fordelingen, n-1
(n  1) s
Chi-kvadrat, n-1
frihetsgrader
 02
2
x
frihetsgrader:
tn 1
 n21
Comparing two groups of
observations: matched pairs
Assuming normal
distributions, unknown
variance: Compare
means
Sign test: Compare
only which
observations are
largest
Wilcoxon signed rank
test: Compare ranks
and signs of
differences
D  D0
sD / n
tn 1
(D1, …, Dn differences)
S = the number of pairs
with positive difference.
Large samples S *  0.5n
(n>20):
0.5 n
T=min(T+,T-);
T+ / T- are sum of
positive/negative ranks
Bin(n, 0.5)
Large samples:
N (0,1)
Wilcoxon signed rank
statistic
Comparing two groups of
observations: unmatched data
Diff. between pop. means:
Known variances
Diff. between pop. means:
Unknown but equal variances
Diff. between pop. means:
Unknown and unequal
variances
Testing equality of variances
for two normal populations
Assuming identical translated
distributions: test equal
means: Mann Whitney U test
( X  Y  D0 ) /
 x2
( X  Y  D0 ) /
s 2p
nx
( X  Y  D0 ) /
sx2
nx
sx2 / s y2
nx
 y2
 ny

s 2p
ny

s 2y
ny
Standard normal N (0,1)
tnx ny 2
t
see book
for d.f.
Fnx 1,ny 1
Based on sum of ranks of Standard normal (n>10)
obs. from one group; all
N (0,1)
obs. ranked together
Comparing more than two groups of
data
One-way ANOVA: Testing if
all groups are equal (norm.)
SSG /( K  1)
SSW /(n  K )
Kruskal-Wallis test: Testing if Based on sums of ranks
all groups are equal
for each group; all obs.
ranked together
FK 1,n  K
 K2 1
Two-way ANOVA: Testing if
all groups are equal, when
you also have blocking
SSG /( K  1)
SSE /(( K  1)( H  1))
FK 1,( K 1)( H 1)
Two-way ANOVA with
interaction: Testing if groups
and blocking variable interact
SSI /(( K  1)( H  1))
SSE /( HK ( L  1))
F( K 1)( H 1), HK ( L 1)
Studying population proportions
Test of population
proportion in one
group (large
samples)
Comparing the
population
proportions in two
groups (large
samples)
p 0
 0 (1   0 ) / n
px  p y
p0 (1  p0 ) p0 (1  p0 )

nx
ny
(p0 common estimate)
Standard normal
N (0,1)
Standard normal
N (0,1)
Regression tests
Test of regression slope:
Is it * ?
Test on partial regression
coefficient: Is it * ?
Test on sets of partial
regression coefficients:
Can they all be set to
zero (i.e., removed)?
b1  *
sb1
b j  *
sb j
( SSE (r )  SSE ) / r
se2
tn  2
tn K 1
Fr ,n  K  r 1
Model tests
Contingency table test: Test if
there is an association
between the two attributes in
a contingency table
Goodness-of-fit test: Counts
in K categories, compared to
expected counts, under H0
Tests for normality:
•Bowman-Shelton
•Kolmogorov-Smirnov
r
c

(Oij  Eij )2
Eij
i 1 j 1
(Oi  Ei )2

Ei
i 1
(2r 1)( c1)
K
*
 K2 1
*
Tests for correlation
Test for zero population
correlation (normal
distribution)
Test for zero correlation
(nonparametric): Spearman
rank correlation
r n2
1 r
2
Compute ranks of xvalues, and of yvalues, and compute
correlation of these
ranks
tn  2
Special
distribution
Tests for autocorrelation
The Durbin-Watson test
(based on normal
assumption) testing for
autocorrelation in
regression data
The runs test
(nonparametric), testing
for randomness in time
n
2
(
e

e
)
 t t 1
Special distribution
i 2
n
2
e
t
i 1
Counting the number Special distribution,
of ”runs” above and or standard normal
below the median in
N (0,1)
the time series
for large samples
From problem to choice of method
• Example: You have the grades of a class of
studends from this years statistics course,
and from last years statistics course. How to
analyze?
• You have measured the blood pressure,
working habits, eating habits, and exercise
level for 200 middleaged men. How to
analyze?
From problem to choice of method
• Example: You have asked 100 married
women how long they have been married,
and how happy they are (on a specific scale)
with their marriage. How to analyze?
• Example: You have data for how satisfied
(on some scale) 50 patients are with their
primary health care, from each of 5 regions
of Norway. How to analyze?