Download Users of statistics

Psych 524, 10/24/05
p. 1/4
One Sample Statistical Inference, Part 2: Effect Size & Confidence Intervals
(based on Kirk, Ch. 10)
Shortcomings of Hypothesis Testing (10.5, 10.6)
1) statistical significance does not imply practical significance
i.e., if our result is statistically significant, this does not mean that it will have
any real-world implications
Example: Returning to the birthweight example, would it be practically
significant to find a difference of .01 pounds (7.1 vs. 7.11)? With what
minimum sample size would we obtain a statistically significant result for
such a difference using α = .05 (two-tailed)?
Effect Size: The magnitude of a difference expressed in standard deviation
units; sample size is no longer influential!
d
X  0
, where (according to Cohen):

d = .2 (to .4999) is a small effect
d = .5 (to .7999) is a medium effect
d = .8 (or greater) is a large effect
Example: What is the effect size for the birthweight example? ( X =7.4, σ=.82,
μ=7.1, n = 100)
Example: What mean would you need to observe for a large effect size?
2) Scientific inference involves knowing the conditional probability that the null
hypothesis (Ho) is true, given that we have obtained a particular data set (D).
In other words, it tells us p(Ho|D).
But, null hypothesis testing tells us the conditional probability of obtaining
these data or more extreme data if the null hypothesis is true. In other words,
it tells us p(D|Ho).
Psych 524, 10/24/05
p. 2/4
As we saw with Bayes’ Theorem, these two probabilities are related, but not
the same.
3) The null hypothesis is (almost) always false because the population value of
your groups will (almost) always be off by a certain number of decimal
places. The result of a null hypothesis test is based on the power of the test
instead of whether there is a true population difference.
4) Use of α as a cutoff score is arbitrary and changes a continuous phenomenon
(low to high uncertainty) into a dichotomous one (reject or fail to reject).
Confidence Intervals (10.5)
Confidence Interval (CI): an interval on the number line within which there is a
high probability that the population parameter (usually a mean) lies
Two-Sided CI’s
the midpoint is the sample mean
the CI is bounded on both sides by a specified number of standard error units
( X )
for one-sample z-tests, the number of standard error units on each side is
specified by the z-score that corresponds with α
95% CI: α = .05, and z = 1.96
99% CI: α = .01, and z = 2.576
other CI’s: use z-table; remember to look up half of α
for other tests (e.g., t-tests), use the corresponding table to look this up
(otherwise, same procedure!)
general formulas for a (1- α)% CI:
X  z / 2 X    X  z / 2 X
or
X
z  / 2
n
X
z  / 2
n
Psych 524, 10/24/05
p. 3/4
Example: Compute a 95% CI for the birthweight example ( X =7.4, σ=.82, μ=7.1,
n = 100)
CI Interpretation
What does a 95% CI tell us?
If we conducted an infinite number of samples, 95% of those samples
would yield CI’s that included the true population mean; this means that 5%
of the samples would not give us a CI that contained the mean.
Do Say... (see p. 331)
“The 95% CI for the mean was XXX to YYY”
“We can be 95% confident that the true mean lies between XXX and YYY”
“Our confidence that the mean lies between XXX and YYY is .95”
“We are 95% confident that the true mean is greater than XXX but less than
YYY”
Do Not Say...
“The probability that the true mean lies between XXX and YYY is .95”
why not? This is a subtlety, but the main point is that once we insert a
sample mean into the CI formula, the interval either does or does not
contain the true population mean
CI’s and Null Hypothesis Testing
CI’s tell us all that null hypothesis tests do...and more!
If μo is not included in the CI, we would reject Ho; if μo is included in the
interval, we fail to reject.
Rather than just this reject/fail to reject information, we have a range of likely
candidates for the true value of the population mean.
Psych 524, 10/24/05
p. 4/4
One-Sided CI’s
When we have a one-sided (directional) hypothesis, the boundaries for our
confidence interval are:
on the side of the alternative hypothesis: infinity
on the side of Ho: specified number of standard error units (  X ); direction
depends on direction of Ho
for one-sample z-tests, the number of standard error units on the Ho side is
specified by the z-score that corresponds with α
95% CI: α = .05, and z = 1.645
99% CI: α = .01, and z = 2.33
other CI’s: use z-table; remember to look up half of α
formula when Ho: μ <= value:
X  z  X  
formula when Ho: μ >= value:
  X  z  X
Example: Assume the following Ho for the birthweight problem:
Ho: μ <= 7.1 ...what is the one sided 95% CI?