Download Powerpoint - Marshall University Personal Web Pages

Marshall University School of Medicine
Department of Biochemistry and Microbiology
BMS 617
Lecture 7 – T-tests
Marshall University Genomics Core Facility
T-tests
• T-tests refer to a family of statistical tests, in
which a mean, or difference in two means, is
assumed to be sampled from a T-distribution
• As a statistical test, T-tests compute p-values
– The probability of seeing a mean, or difference of
means, this large, assuming the null hypothesis is
true
– Interpreting a t-test involves knowing the null
hypothesis
Marshall University School of Medicine
Types of T-test
• One-class T-test:
– Null hypothesis is that the mean of a set of values is equal
to some fixed value
• Two-class T-tests:
– Two groups of values
• Unpaired T-test
– Most common T-test
– Null hypothesis is that the values in each group are sampled from
distributions with equal means
• Paired T-test
– Each sample in the first group is paired with a sample in the second group
– Null hypothesis is that the mean of the difference between samples in
each pair is zero
Marshall University School of Medicine
Example: One-class T-test
• Recall our body-temperature data from earlier
– n=130 samples of body temperature
– Mean m=36.82C, SD s=0.41C
• We wanted to use this to test the hypothesis
that mean body temperature was μ=37C
• Under the assumptions of the one-class t-test,
the value t=(μ-m)/(s/√n) follows a tdistribution with n-1 degrees of freedom
• For this example, t=5.006
Marshall University School of Medicine
Computing a p-value for a given t
• To get a p-value for t=5.006, we either use software or a table
– Need to know the degrees of freedom
– In this case, d.f.=130-1=129
• Tables either give the probability value for a given t and df, or critical tvalues for given probabilities and df
• From tables or software, the probability that t<=5.006 is p=0.9999991
• We want to know the probability of seeing a result this extreme, assuming
the mean is 37C, i.e. assuming t follows a t-distribution with 129 d.f.
• This is the probability either that t>5.006 or that t<-5.006
– A two-tailed, one-class t-test
• P(t>5.006)=1-0.9999991=0.0000009
• So P(t<-5.006)=0.0000009
• and p=2 x 0.0000009 = 0.0000018
Marshall University School of Medicine
Assumptions for a one-class t-test
• The one-class t-test is accurate under the
following assumptions:
– The samples are random (or representative)
– The observations are independent
– The data are accurate
– The data are sampled from a population that is
normally distributed
Marshall University School of Medicine
One-class T-test and the Confidence
Interval
• We saw earlier how to compute a confidence
interval for the mean of this data set
– Calculate w=t*s/√n
– The confidence interval is from m-w to m+w
– t* is the value from the t-distribution for which P(t>t*
or t<-t*)=1-confidence
• e.g. for a 95% confidence interval we want
P(t>t* or t<-t*) =0.05
• So P(t>t*)=0.025. From tables or software, t*=1.979.
• This gives a 95% confidence interval of [36.75, 36.89]
• Knowing the 95% confidence interval does not contain the
null hypothesis value of 37 is equivalent to knowing the pvalue is less than 0.05
Marshall University School of Medicine
Two-class unpaired T-test: Example
• For an example of a two-class, unpaired T-test,
consider the GRHL2 expression data we saw
earlier from Cieply et al., Cancer Research
2012.
• Compared expression of GRHL2 in different
breast cancer cell lines, classified as Basal A,
Basal B, or Luminal.
– Compare the Basal A expression to the Basal B
expression
Marshall University School of Medicine
GRHL2 Expression Data
Basal-A
Basal-B
Cell line
Log2 Expression
Cell line
Log2 Expression
HCC1007
BT-20
1.97
1.86
BT-549
HBL-100
-0.724
-0.62
HCC1143
HCC187
2.04
1.63
HCC1500
HCC38
1.19
0.477
HCC1569
1.44
Hs 578T
-0.813
HCC1937
2.06
MCF10a
0.36
HCC1954
2.19
MCF12a
1.16
HCC2157
1.64
MDA-MB-157
-0.123
HCC3153
1.69
MDA-MB-231
0.623
HCC70
1.73
MDA-MB-435
-0.944
MDA-MB-468
2.26
MDA-MB-436
-0.333
SUM-190PT
1.48
SUM-1315M02
-0.819
SUM-225CWN
3.14
SUM-149PT
1
SUM-159PT
0.581
Marshall University School of Medicine
GRHL2 Expression Data
Marshall University School of Medicine
How an unpaired t-test works
• An unpaired t-test works by computing the
difference of the means of the two samples
• Assuming the null hypothesis – that the
difference of the two means is zero – the
difference of the sample means, divided by a
pooled standard error of the mean, will be
distributed with a t-distribution
Marshall University School of Medicine
Unpaired t-test for the GRHL2 data
Marshall University School of Medicine
Assumptions for the unpaired T-test
• The unpaired T-test works along essentially the
same assumptions as the one-class T-test:
–
–
–
–
The samples are random or representative
The observations are independent
The data are accurate
The values in the populations are at least
approximately normally distributed
• Additionally, the t-test we used here assumes:
– The the populations have the same standard deviation
Marshall University School of Medicine
Assumption of equal variances
• In the Basal A vs Basal B GRHL2 comparison, the
Basal B samples have higher SD (0.7859) than the
Basal A samples (0.4463)
• The t-test we ran assumed the samples came
from populations with equal variances (i.e. equals
standard deviations)
• A test can be run to see if the data are consistent
with the assumption of equal variances
– The distribution of the square of the ratio of the
standard deviations is known under the assumption
that the population variances are equal
Marshall University School of Medicine
If the assumption of equal variances is
violated
• A modified t-test can be used, which doesn’t
make the assumption of equal variances
– Called the “Welch T-test”
– Has less power than the standard unpaired t-test
• As usual, testing your data set in order to decide
which test to use can give misleading results
– Typically will give over-optimistic p-values
• In the ideal world, we would run an experiment
specifically to determine if the assumption of
equal variances holds, then use that to determine
how to analyze our real experiment
Marshall University School of Medicine
Rules of thumb for the assumption of
equal variances
• Unequal variances will only badly affect the t-test if the
number of samples in each group is small and unequal
– In other cases the t-test is very robust to violations of this
assumption
• In practice, I do the following:
– If the number of samples is equal, I use the regular t-test
– If the number of samples in both groups is at least 5, no
matter if they are equal, I use the regular t-test
– If there is reason to believe the variances should be equal
(e.g. if all the variance comes from technical replicates), I
use the standard t-test
– Otherwise, I use the Welch T-test
Marshall University School of Medicine
95% Confidence Intervals and
Unpaired t-tests
Data 2
3
2
1
0
B
as
al
B
B
as
al
A
-1
Marshall University School of Medicine
95% Confidence Intervals and
unpaired T-tests
• The unpaired t-test results computed the difference between the
means and the 95% confidence interval for that difference
– For this example the 95% confidence interval of the difference of the
means was [-2.373, -1.348]
• If this confidence interval doesn’t contain zero, this is equivalent to
p<0.05
• We can also compute the confidence interval for each mean
independently
• If these confidence intervals do not overlap, then the p value is
definitely less than 0.05
– In fact, it must be a lot less…
• If the confidence intervals do overlap, then p may or may not be
less than 0.05
– Cannot deduce anything from the error bars in this case
Marshall University School of Medicine
Other error bars and statistical
significance
• What if the bar chart uses SD or SEM for the error bars?
• SD tells us about the amount of scatter in the data
– Nothing about the precision with which the mean is measured
• Overlapping, or non-overlapping, SD error bars have
nothing to do with statistical significance
• SEM measures the precision with which we approximate
the mean
– But interpretation depends on knowing the sample size
– We can deduce the following:
• If SEM error bars overlap, then the difference is definitely not
statistically significant at p=0.05 (in fact, p is much bigger…)
• If SEM error bars do not overlap, the p may or may not be less than
0.05
Marshall University School of Medicine
Error bars and statistical significance
summary
Error bar type
Conclusion if overlapping
Conclusion if not overlapping
95% CI
No conclusion
p<<0.05
SD
No conclusion
No conclusion
SEM
p>>0.05
No conclusion
Marshall University School of Medicine
Paired t-tests
• Paired t-tests are used when the comparison is between samples
which are paired between the groups
– Before and after treatments on a set of patients
• Pair the “before value” on patient A with the “after value” on patient A
• The “before value” on patient B with the “after value” on patient B, etc
– Studies in which subjects are recruited to two groups in a matched
fashion
• Match a control patient with a treatment patient based on age, sex, weight,
height…
• Difficult type of study to perform
– Twin or sibling studies
– Lab experiments in which treated and control samples are handled in
parallel
• Plate cells, divide into two, treat one half and use the other as control
• Repeat the next day with another plate, etc
Marshall University School of Medicine
Example
• In a recent experiment, we performed expression
profiling on a set of eight mice
– Should not use t-test here without correcting for multiple
hypotheses, but this is a good example for demonstration
• Four litters of mice were bred, and two male mice
selected (at random if necessary) from each litter
• One mouse from each pair was treated and one was
used as a control
• Analyzing these data with a paired test has the
potential to eliminate any litter-litter variation
Marshall University School of Medicine
Example data
Litter
Litter 1
Litter 2
Litter 3
Litter 4
Control
Treated
19.939
19.203
13.307
45.513
• Actual data are (normalized) read counts for the gene of interest
• After sequencing, align all reads to the genome
• Count the number of reads that align to each gene for each sample
Marshall University School of Medicine
124.375
131.550
96.992
112.770
Paired t-test
• There are two distinct null hypotheses we can
make about paired data
– Both say “the data is no different between the two
groups”
– One is that the difference between the values in
the groups is zero (“paired t-test”)
– The other is that the ratio of the values in the
groups is 1 (“ratio paired t-test”)
Marshall University School of Medicine
Paired T-test results
Marshall University School of Medicine
How a paired t-test works
• A paired t-test is really just a one-class t-test!
– Computes the differences for each pair
– And then tests the null hypothesis that those
differences are samples from a normal
distribution with mean zero
• The confidence interval is just the confidence
interval of the mean differences
Marshall University School of Medicine
Ratio paired t-tests
• The ratio paired t-test tests the null hypothesis
that the ratio of the paired values is 1
• This is done simply by a mathematical trick: take
the log of all the ratios
– Then perform a regular paired t-test with the log
ratios instead of the differences
– Log(1)=0
– Software performing a ratio paired t-test takes care of
computing the logs, performing the test, and then
transforming the mean difference of log ratios,
confidence interval of that mean, etc, back to ratio
values
Marshall University School of Medicine
Graphing paired data
• Plotting bar charts, or even column-scatter
plots, of paired data does not show the
pairing between the data values
• A better presentation is a connected column
scatter plot
– Column scatter plot with lines connecting the
paired data points
Marshall University School of Medicine
Connected column scatter plot
Data 4
150
100
50
d
te
Tr
ea
U
nt
re
at
ed
0
Marshall University School of Medicine
t-test summary
• t-tests are a family of statistical hypothesis tests
• Generate a p-value
– Remember how to interpret!
• Null hypotheses:
– For a one class t-test, the null hypothesis is that the samples are
drawn from a normally-distributed population with a specified mean
– For an unpaired t-test, the null hypothesis is that the samples are
drawn from two normally-distributed populations with equal means
– For a paired t-test, the null hypothesis is that the differences between
matched values are samples of a normally-distributed population with
mean zero
• Equivalent to a one class t-test on the differences between matched values
Marshall University School of Medicine