Download p value

Survey
yes no Was this document useful for you?
   Thank you for your participation!

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts

Statistics wikipedia, lookup

History of statistics wikipedia, lookup

Transcript
P Values - part 3
The P value as a
‘statistic’
Robin Beaumont
1/03/2012
With much help from
Professor Geoff Cumming
P values - Putting it all together
Summary
Reviewso far
• A P value is a conditional probability which
considers a range of outcomes – shown as a
‘area’ in a graph.
• The SEM formula allows us to: predict the
accuracy of your estimate ( i.e. the mean
value of our sample) across a infinite
number of samples!
Summary
far
What is a so
statistic?
• A statistic is just a summary measure,
technically we have reduced a set of data to
one or two values:
• Range (smallest – largest)
• Mean, median etc.
• Inter-quartile range, SD Variance
• Z score, T value, chi square value, F value etc
• P value
T value
• T statistic – different types, simplest 1 sample:
Tstatistic  observed
difference in estimated mean and population value
sampling variability in means
Tstatistic  observed
difference in estimated mean and population value
SEM

observed difference in estimated mean and population value
expected variability in means due to random samping
Signal

Noise
So when t = 0 means 0/anything = estimated and
hypothesised population mean are equal
So when t = 1 observed different same as SEM
So when t = 10 observed different much greater than
SEM
T statistic example
Serum amylase values from a random sample of 15 apparently healthy
subjects. The mean = 96 SD= 35 units/100 ml.
How likely would such a ‘unusual’ sample be obtained from a
population of serum amylase determinations with a mean of 120.
(taken from Daniel 1991 p.202 adapted)
The population value = the null
hypothesis
Tstatistic 
96  120
24


35
9.037
15
2.656
This looks like a rare occurrence?
t density:
s x = 9.037 n =15
120
96
Given that the sample was obtained
from a population with a mean of 120
a sample with a T(n=15) statistic of 2.656 or 2.656 or one more extreme
will occur 1.8% of the time = just
under two samples per hundred on
Shaded
area
average.
=0.0188
....
What does the shaded
Given that the sample was obtained
area mean!
from a population with a mean of 120
0
a sample of 15 producing a mean of 96
2.656
0
-2.656
t
(120-x where x=24) or 144 (120+x
Serum amylase values from a random sample of 15 apparently
healthy subjects. mean =96 SD= 35 units/100 ml.
where x=24) or one more extreme will
How likely would such a unusual sample be obtained from a
population of serum amylase determinations with a mean of 120.
occur 1.8% of the time, that is just
(taken from Daniel 1991 p.202 adapted)
under two samples per hundred on
=P value
average.
P value = 2 · P(t < t| H is true) = 2 · [area to the left of t under a t distribution with df = n − 1]
Original units:
(n−1)
o
P value and probability for the one sample t statistic
p value
= 2 x P(t(n-1) values more extreme than obtained t(n-1) | Ho is true)
= 2 X [area to the left of t under a t distribution with n − 1 shape]
Statistic -> sampling distribution -> PDF -> p value
No sampling distribution! Create a virtual one
P Value Variability
Taking another random sample the P value be different
How different? – Does not follow a normal distribution
Depends upon the probability of the null hypothesis
being true! Remember we have assumed so far that the
null hypothesis is true.
Dance of the p values – Geoff Cummings
Simplified dance of the p values when the null hypothesis is true
Example from Geoff
Cummings dance of the p
values
The take home message is
that we can obtain very small
p values even when the null
hypothesis is true.
Why no CI for the P Value if it varies across trials
• P value -> statistic but
• Not all statistics represent values that are reflected in a
population value
• Other ways of getting an idea of variability across trials:
• Reproducibility Probability Value (RP)
Goodman 1992 and also 2001 journal articles
Hung, O’Neill, Bauer & Kohne 1997 Biometrics journal
Shao & Chow 2002 – Statistics in Medicine journal
Boos & Stefanki 2011 – Journal of the American statistical
association
Cummings 2008 + and book
Cumming’s Reproducibility (replication) Probability Value
Given Pobtained = 0.05
What is the interval in which
we are likely to see 80% of
subsequent P values?
Answer:
We have 80% of seeing
subsequent p values fall within
the zero to 0.22 boundary
0, 0.22 [One sided]
This means that we have a
20% of them being
subsequently > 0.22
What about when the null hypothesis is not true?