Download Lecture 5

Odds and Ends for
Univariate Analyses, etc.
Lecture #5
Computing Confidence
Intervals via SPSS

For means

if one or more groups

may specify confidence level under “Statistics”
→ Descriptive Statistics
→ Explore (with the grouping variable as a “factor”)

if one group

may specify confidence level under “Options”
→ Compare Means
→ One-Sample T Test

For proportions

????? By hand ?????
Hypothesis Tests via SPSS

Mean (for one group)
Can pick the null value/”Test Value”
 Gives two-sided p-value
and 95% CI (of the mean minus the null value)
→ Compare Means

→ One-Sample T Test (enter Test Value)

Proportion (for one group)

Can pick the null value/”Test Proportion”

Automatically chooses the “first” group as the one of
interest.
Gives two-sided p-value
→ Non-parametric Tests

→ Binomial (pick Test Proportion)
One-sided
Hypothesis Tests



One-sided p-values are not provided by most
software programs. Need to be calculated by
hand based on the two-sided p-values
provided by the program.
When should one do a one-sided instead of a
two-sided test?
Depends on



The purpose of the study
Conventions in the particular field of study
Whether a result in the “other” direction would be
of interest/publishable
Sample Size Considerations
for Confidence Intervals




Suppose that you want to know the value of your
parameter to within 2d units (width of the
confidence interval).
“precision” = d
Set d = (critical value)(standard error)
and solve for n.
Assume we want a 95% confidence interval.


The textbook approximates the critical value (1.96) as 2.
Means: n = (2σ/d)2 = 4(σ/d)2


Choose σ.
Proportions: n = 4 π(1- π) / d2

Choice for π?
CI Example – Means


For multiple sclerosis patients, we wish to
estimate the mean age at which the disease was
first diagnosed. We want 95 percent confidence
interval that is 2 years wide.
Suppose the population variance is 10.


We want a “margin of error” for our estimate of
the mean of 1 year:


n = (2σ/d)2 = 4(σ/d)2 = 4σ2 / d 2
Then, n = 4 (10)/12 = 40
Suppose we want a precision of 0.25 year:

Then, n = 4 (10)/0.252 = 640
CI Example – Means


Is 10 realistic for the variance?
Results in standard deviation of 3.16


If the diagnosis ages have approximately a normal
distribution, 95% of people are diagnosed within
2*2(3.16)=12.64 years of each other!
An alternative?



Suppose we surmise that the range of ages of diagnosis
is roughly 15 to 50.
Recall 95% of diagnosis are within 2 standard deviations
of the mean – and 4 standard deviations of each other.
Then (50-15)/4=8.75 would be a “conservative”
estimate of the standard deviation
CI – Mean Sample Size Revised
With the new standard deviation of 8.75:
 For a “margin of error” of 1 year for
our estimate of the mean:


n = 4 (8.752)/12 = 306.25  307 people
For a precision of 0.25 year:

n = 4 (8.752)/0.252 = 4,900.00 people
CI Example – Proportions




Aim: Estimate the proportion of Americans who
think it was a mistake for the U.S. to send troops
to Iraq.
Want to estimate the proportion in favor with a
0.01 precision.
n = 4π(1-π) /(d 2)
π=?



Can use a “best guess” at the value of the true
proportion, maybe, 0.65.
Can use “most conservative guess, ” i.e., 0.50
Let’s try both
CI – Proportions, cont.

Requiring d = 0.01, using n = 4π(1-π)/(d 2)

With π =.50,
n = 10,000

With π =.65,
n=

Perhaps, we’re requiring too much
precision. Let’s attempt d = 0.025

With π =.50,
n = 1,600

With π =.65,
n=
Comments on Sample Size
for CI’s

Sample size calculations are often an iterative
process.
 Need to balance small precision with realistic sample sizes
 For means, investigators are typically very uncertain about
the value of the standard deviation



set standard deviation = range/4
if the population is expected to be normally distributed, make
adjustments via the empirical rule
When carrying out tests, it is good to be conservative.
 By “conservative,” we mean that you would expect to get
precision that is at least as small as what you planned for.
Conservative Sample Size
Calculations

Keep in mind is the possibility of “dropouts.”


For means,


For example, if you expect a drop-out rate of 10%,
multiply the required sample size by 1.10.
use estimates of the standard deviation that are
somewhat bigger than we really expect it to be.
For proportions,


use 0.50 as the value of the proportion.
if the proportion is expected to be very small or very
large, then use a number that is closer to 0.50 than you
really expect in order to get a conservative sample size.
Sample Size Calculations for
Hypothesis Tests


The book only provides sample size formulas for
comparing two means or proportions. Here we
provide sample size formulas for one-group tests.
Variables needed:




Significance level (α), typically 0.05.
The power of the test (probability of rejecting H0 when
HA is true) = 1- β, typically taken to be 0.80 or 0.90.
Δ, the minimum detectable difference, the minimum
distance between the population mean and the null
value that you wish to detect
SD, standard deviation of a continuous measurement
(for means)
Sample Sizes
for Hypothesis Tests

For population means:
n

2
For population proportions:
n


 2  z  z 2
 (1   )  z  z 2
2
Use the value of the null hypothesis: π = π0
The term (zα+zβ)2 is sometimes referred to as
the “power index”. There is a table of these
values on page 199 of your text book.
E.g.: Sample Size for Hypothesis
Tests for Means


For multiple sclerosis patients, we wish to
prove that the age of diagnosis is less than 30.
Suppose




the population variance is 8.752
We would be interested in detecting a mean that is 5 or
more years less than 30 (i.e., 25, with Δ=5)
We will conduct a one-sided test at a 0.05 significance level
and we want to hold the Type 2 error at 0.10 (power=0.90)
n = (variance)(power index)/ Δ2
= (8.752)(8.6)/25 = 26.3375  27 people
HT for Means Example, cont.

Alternatively, suppose




All else the same
we want to hold the Type 2 error at 0.05 (for a power of
0.95)
n = (variance)(power index)/ Δ2
= (8.752)(10.9)/25 = 33.38125 
people needed
Suppose the population variance is only 7.002

n = (variance)(power index)/ Δ2
= (7.002)(10.9)/25 = 21.364

people needed
E.g.: Sample Size for Hypothesis
Tests for Proportions

Prove that the proportion of Americans who
think it was a mistake for the U.S. to send
troops to Iraq is more than 0.50.




Want to detect differences that are as small as 0.10,
i.e., the we assume the proportion of Americans is at
least 0.60.
0.05 significance level, 0.90 power
In formula, use π=0.50.
n = π(1-π) (power index)/ Δ2
= (.25)(8.6)/ 0.102 = 215 people
E.g. Sample Size for HT’s –
Proportions

COMMENT:
Sample size calculations for
proportions can be highly unreliable if
the resulting sample sizes are less
than those for which the CLT applies.
Alternative sample size calculation
methods exist for these situations.
The Multiple Testing
Problem


If I do 100 hypothesis tests, in which the null hypothesis is
true, in how many do I expect to reject the null hypothesis?
The Bonferroni correction





intended to correct for accumulating error when doing two or more
confidence intervals or hypothesis tests
typically applied when we have data from two or more groups or
when we have multiple variables that are of interest
If you are doing c hypothesis tests, each one must be done
at a level of 1  (1   )1/ c
In order to maintain an overall Type 1 error rate of α, each of
the individual error rates must be done with a Type 1 error
rate of α/c.
The overall Type 1 error rate α is if the probability of
rejecting one or more of the c null hypotheses is α.
Interim Analysis





Suppose that at multiple times throughout the course of
an experiment, we look at the data and stop if we find
significance. These are called interim analyses.
Usually, interim analyses are done for ethical and financial
reasons.
If each interim test of the data is performed at the 0.05
significance level, then by the time the experiment is
finished, the chances of having rejected the null
hypothesis when it is true is more than 5%.
Consequently, the interim analyses and the final analyses
must be done at significance levels that are lower than
the desired overall 0.05 significance level.
Typically, each interim analyses are done at a much
smaller significance level than the final analysis.
Meta-Analysis




Typically, numerous studies are conducted
which look at the same hypothesis.
By looking at a collection of studies, one can
pool information and get a clearer picture of
the “truth” of the hypothesis.
Problem: There are often contradictory studies,
with radically different p-values.
How does one combine the evidence against
the null hypothesis?


via Meta-Analysis
One huge problem with standard MetaAnalyses procedures: Publication Bias
Main vs. Secondary Analyses




Typically when planning a study, you should
choose one to three PRIMARY hypotheses of
interest.
These will be the focus of the main analyses of
your study and should remain fixed over the
course of the study and analyses.
However, you often collect much more data
than is needed to conduct these one to three
hypothesis tests.
Any additional analyses are called secondary
analyses
Secondary Analyses

Problem:




Solution:



Most studies result in nearly an infinite number of
tests or comparisons that can be made.
Eventually a significant result will be found, just by
chance.
P-values are therefore difficult to interpret.
Apply Bonferroni or other similar corrections when
appropriate.
Make it clear when you are performing “secondary
analyses”
Significance in secondary analyses provide great
material for “future research!”
Homework


Read Chapters 11, 12, 13, 22 and 38
Homework Problems

Available at
http://yhkuo.tripod.com/PHCO0504/index.htm