Download Slides 9a - My Illinois State

PRINCIPLES OF
HYPOTHESIS TESTING
1
A Quick Review of Important Issues
About Sampling:
Why Sampling?
• To examine the sample’s attributes (sample
statistics) as ESTIMATES of the
population’s characteristics (population
parameters)
 use sample characteristics to make inferences
about the population.
• Estimating, by definition, involves some error
(i.e., sampling error/bias
Resulting from the fact that the sample may not
mirror characteristics of the population).
2
HYPOTHESIS TESTING OFTEN INVOLVES:
a. comparing groups regarding differences in means or proportions, or
b. Examining strength and direction of relationships between two
variables
Common Types of Research Hypotheses and the Related Statistical
Data Analysis Methods:
a. Checking for Presence/Absence of Relationship(s) Among
Variables (and direction/strength of the relationship)
• Bivariate (e.g., Pearson Correlation—r)
Between one variable and another: Y = a + b1 x1
• Multivariate (e.g., Multiple Regression Analysis)
Between one dep. var. and an independent variable,
while holding all other independent variables constant:
Y = a + b1 x1 + b2 x2 + b3 x3 + … + bk xk
b. Checking for Presence/Absence of Difference(s) Among Groups
• Difference(s) in Proportions (Chi Square Test—2)
• Difference(s) in Means (Analysis of Variance)
3
HYPOTHESIS TESTING
QUESTION: When testing hypotheses regarding presence of a relationship/
difference, what does “NULL HYPOTHESIS” (H0) refer to?
Null Hypothesis in most cases states:
“There is no relationship or no difference.”
NOTE: Statistical tests of hypotheses always report
the result of testing the null hypothesis.
The researcher will then have to restate the results in terms of
finding/not finding support for the original research hypothesis.
• If we reject the null, the conclusion is that we have
found a statistically significant relationship/difference.
– When we don’t reject the null, we infer that . . .?
“…any difference/relationship that may be apparent from
sample data is likely to be the result of . . .??
sampling error (i.e., an artifact of the particular sample
being used)”.
4
A Quick Review of Important
Issues About Sampling:
• So, when using sample data to test hypotheses
and make judgments about the population,
there is always a chance for reaching
erroneous conclusions about the population.
• What do we mean by erroneous conclusions?
• What types of erroneous conclusions can we reach when testing
hypotheses?
5
Important Notes About Sampling
Two possible types of erroneous conclusions
from sampling error, when testing hypotheses:
TYPE I ERROR and TYPE II ERROR
6
Important Notes About Sampling
Type I Error?
Rejecting a true “null hypothesis” (erroneously)
•
Rejecting the null when we should not (i.e., when the null is true)
Null Hypothesis?
States “There is NO relationship, there is NO difference, etc.”
“Rejecting the null” refers to concluding…?
Concluding that: “There is a significant relationship/ difference”
7
Important Notes About Sampling
So, type I error (“Rejecting a True Null”) means?
• No relationship/difference exists, but from sample evidence
we come to the conclusion that a significant relationship/
difference does exist.
• Example: A drug is really not effective, but we conclude it is.
Sample
• Conclusion: Type I Error involves finding “something” that
does not really exist—i.e., a case of “False positive”
8
Important Notes About Sampling
Type II Error?
• Accepting a false null hypothesis (or failing to reject a
false null hypothesis)
False Null means?
• “A relationship/difference does in fact exist”.
So, “accepting a false null” (i.e., type II error) means?
• A relationship/difference does in fact exist, but from
sample evidence we fail to detect it (fail to reject the null).
– Come to the conclusion that there is no
relationship/difference (in the population).
• Example; A drug is really effective, but our study shows it
is not.
9
Important Notes About Sampling
Type II Error:
Sample
CONCLUSION: Type II Error represents failing to find
“something” that does exist; it represents a case
of “False Negative.”
10
A Quick Review of Important
Issues About Sampling:
When testing hypotheses, what is the purpose of statistical testing
(significance testing)?
– Statistical tests of significance assess the likelihood of
reaching an erroneous conclusion when using sample data.
In fact, they always assess the likelihood of type I error.
• They assess the probability that the relationship/ difference we have
found (using sample data) may simply be an artifact of the particular
sample we have happened to end up with (i.e., is caused by sampling
error).
– In fact, when using data from the entire population (e.g., a census):
• No chance of sampling error exists
• No need for conducting statistical tests of significance.
11
HYPOTHESIS TESTING
• Statistical tests of significance alway assess
the likelihood/probability of type I error (a )
when using sample data.
Sampl
e
• Once a test is conducted and a is determined, we will
have to decide if we are able/willing to tolerate a (the risk
involved in rejecting the null (and, thereby, to report what…?)
 … that the relationship/difference detected (from sample data), is
too large to be attributed to chance/sampling error.
 That is, decide whether we should consider the
relationship/difference “statistically significant.”
12
Important Notes About Sampling
The probability of committing Type I Error is
called:
• a (alpha) or significance level.
The complement of a:
• 1- a or confidence level.
What does 1- a represent?
 probability of accepting (not rejecting) the null when it
is true:
– concluding NO relationship/difference exists, when
indeed it DOES NOT exist (i.e., chance of not finding
what does not exist)—a correct conclusion
13
HYPOTHESIS TESTING
 So, testing the plausibility of hypothesis/propositions
(i.e., decision to reject/not reject H0) is a probabilistic decision...
?
• It requires us to:
a) Determine the likelihood of null being true (a) and,
thus, the risk (of being wrong) that we would be
taking if we decide to reject the null (a), and
b) Decide whether we are willing/able to tolerate that
risk (a level) by actually rejecting the null…
 and reporting that we have found a
“significant” relationship/difference.
• So, generally speaking, when should we be
tempted to reject the null?
When (a) is ___large or when it is ___small?
15
HYPOTHESIS TESTING
A small a means that…
•…the CHANCE of NULL being TRUE is TOO SMALL to warrant ACCEPTING it .
•…if we decide to reject the null (i.e., conclude that we
have found a relationship/difference), we stand
a relatively small chance of being wrong.
• …rejecting the null is a relatively safe bet.
•…the difference/relationship found is statistically significant .
NOTE:
Small a
rejecting the null
finding a statistically significant
relationship/difference
reporting that the relationship/difference
found (from sample evidence) is too large to be attributed to chance/
sampling error
16
HYPOTHESIS TESTING
BUT HOW DO YOU MEASURE a?
 How would you determine what the actual a
level is (i.e., how much risk of being wrong you
would actually be taking if you were to decide to
reject the null?
 ANSWER…
(a)Look up the actual a from a table of probability distribution
for the test statistic being used,
OR
(b) More conveniently, rely on your statistical software (e.g.,
SPSS) to compute and report the actual a (“Sig.” or
“Prob.”) level for you.
17
HYPOTHESIS TESTING
DECIDING ON AN a that I CAN TOLERATE!!!
• BUT, WAIT ...
How am I supposed to know what odds of
being wrong I should be willing/able to
tolerate as I consider rejecting the null?
 A SIMPLE ANSWER:
a < 5% is conventionally considered to be
a reasonable/small enough risk to be
tolerable in most situations.
18
HYPOTHESIS TESTING
• IS THERE A RULE OF THUMB TO FOLLOW WHEN
TESTING HYPOTHESIS? YES!
• WHAT IS IT?
THE GOLDEN RULE: When testing a hypothesis,
•if the reported a (e.g., “sig.” in SPSS) turns out to
be less than or equal to 0.05, reject the null and
report a statistically significant relationship/difference
 Because the odds of being wrong would be
tolerable).
•Otherwise, refrain from rejecting the null, on the grounds
that the odds of committing an error
(i.e., rejecting a true null) would be prohibitive.
• And, as a result, report . . . ?
19
HYPOTHESIS TESTING
EXCEPTIONS TO THE RULE?
Use a smaller a (e.g., a < 0.01 ) when:
1.
2.
Sample size is relatively large
Consequence of committing type I error is serious/costly
(i.e., False positive results are very costly)
• (e.g., H0: Capital punishment is not a strong
deterrent for criminal behavior.)
Use a larger a (e.g., a < 0.10 ) when:
1.
2.
Sample size is relatively small
Conducting exploratory research whose results provide
the basis for further research
20
HYPOTHESIS TESTING
QUESTIONS OR
COMMENTS
?
21