Download Significance

Week 5
PADM 582
University of La Verne
Soomi Lee, Ph.D
What you
will learn
in Chapter 9
Review
of Concepts
•
•
•
•
•
•
Population vs. Sample
Parameters vs. Statistics
Sampling error
Probability
Properties of the Normal Curve
Z score: computation and interpretation
Significance
PADM 582
University of La Verne
Soomi Lee, Ph.D
What you
will learn
in Chapter 9
Statistical
Significance
1. What significance is and why it is important
– Significance vs. Meaningfulness
2. Type I Error
3. Type II Error
4. How inferential statistics works
The
Concept
Significance
The
Concept of
of Significance
• Any difference between groups that is due to
a systematic influence rather than chance
– Must assume that all other factors that might
contribute to differences are controlled
The
Concept
Significance
The
Concept of
of Significance
“There is a significant difference in attitude toward
maternal employment between adolescents whose
mothers work and adolescents whose mothers do
not work as measured by a test of emotional state.”
Whether or not mothers work is a systematic
factor that makes difference in attitudes towards
working mom between the two groups.
If Only
Perfect…
AreWe
You Were
100% Sure??
• No. When you find that there is a systematic
influence you cannot be 100 % sure.
• Why? You can be wrong because
– There might be possible confounding factors
– A sample does not represent the population.
If OnlySignificance
We WereLevel
Perfect…
• Because you are not 100 percent sure you set
a significance level.
• Significance level
– “Risk associated with not being 100% confident
that what you observe in an experiment is due to
the treatment or what was being tested.”
– likelihood that you are wrong (your hypothesis is
wrong = there is no systematic influence).
If OnlySignificance
We WereLevel
Perfect…
• Sylvia sets her significance level at the 5%.
• Notation: P < .05
“There is a 5% chance that the income difference
between union members and non members is not due
to union membership.”
Can you state her null and research
hypothesis?
Statistical Significance
I am 95% confident that my (research) hypothesis
is right.
= there is a significant difference or relationship.
= there is a systematic influence.
There is a 5% chance that I am wrong.
 There is a 5% chance that I reject the true null
hypothesis.
Statistical Significance
I am 95% confident that my hypothesis is right (=
there is a significant difference or relationship =
there is a systematic influence)
There is a 5% chance that I am wrong.
 There is a 5% chance that I reject the true null
hypothesis.
Type I error
The World’s
Most
Important
Different
Types
of Errors Table
Type I ErrorsType
(Level
of Significance)
I Errors
• The probability of rejecting a null hypothesis
when it is true
• Conventional levels are set between .01 and
.05
• Represented by α
• Usually represented in a report as
p < .05
Type
Errors
Type II
II Errors
• The probability of accepting a null hypothesis
when it is false
• Represented by β
• As your sample characteristics become closer
to the population, the probability that you will
accept a false null hypothesis (type II error)
decreases
Significance
Versus
Meaningfulness
Significance
vs. Meaningfulness
• A study can be statistically significant but not
very meaningful.
Significance
Versus
Meaningfulness
Significance
vs. Meaningfulness
Classroom
Teaching
Computers
Average reading score:
Average reading score:
75.7
75.6
• The average reading scores between the two groups are statistically
significant. (=the result of a systematic influence of the teaching methods)
• Is 0.1 point meaningful? Does this number justify to spend $300,000 on
getting computers?
Significance
Versus
Meaningfulness
Significance
vs. Meaningfulness
• A study can be statistically significant but not
very meaningful
• Statistical significance can only be interpreted
for the context in which it occurred
• Statistical significance should not be the only
goal of scientific research
– Significance is influenced by sample size…we’ll talk
more about this later.
How
Works
HowInference
Inference Works
1. Collect a representative sample of the population.
2. Conduct a statistical test and obtain statistics
– Example: the mean scores for groups, regression
coefficients
3. Make a conclusion as to whether the difference
(or relationship) is the result of chance or the
result of statistically significant differences.
4. Based on the results of the sample, make an
inference about the population.
Test
Testof
of Significance
Significance
1. State the null hypothesis.
2. Set the level of risk associated with the null hypothesis (the
level of significance or Type I error)
3. Select the appropriate test statistic (See textbook p. 211).
4. Compute the test statistic value (obtained value = the result
of a specific statistical test).
5. Determine the value needed to reject the null hypothesis
using the appropriate table of critical values
6. Compare the obtained value to the critical value
– If obtained value is more extreme, reject the null hypothesis
– If obtained value is not more extreme, accept the null hypothesis
Test of Significance
Significance Versus
Meaningfulness
Group Work
t-test
Tests between the Means of Different Groups
PADM 582
University of La Verne
Soomi Lee, Ph.D
How Inference
Goals Works
1. When to use a t test
2. How to compute the observed t value
3. Interpreting the t value and what it means
How
Works
StudyInference
on Eating Disorder
• Question: do different cultures make eating
disorders?
• Compare Indian students and Australian
students.
– Eating Attitudes Test
– Goldfarb Fear of Fat Scale
How
Inference
What
to Use a tWorks
test
• When you are interested in finding out if
there is a difference on the average scores of
one variable between the two groups that
are independent of one another.
How
Inference
What
to Use a tWorks
test
• When you are interested in finding out if
there is a difference on the average scores of
one variable between the two groups that
are independent of one another.
The two groups are not related.
(Each
participant is tested only once.)
How Inference
Works
The Homogeneity
of Variance
Assumption
• Almost every statistical test has a certain
assumptions.
• The assumption of the t test: the amount of
variability in each of the two groups is equal.
** called the homogeneity of variance assumption
How to Compute the Observed t value for
How
InferenceMeans
Works
Independent
𝑡=
𝑋1 − 𝑋2
𝑛1 − 1 𝑠21 + 𝑛2 − 1 𝑠22
𝑛1 + 𝑛2 − 2
𝑛1 + 𝑛2
𝑛1 𝑛2
• Numerator is the difference between the
means.
• Denominator is the amount of variation
within and between each of the two groups.
How Inference
Works Test
Alzheimer’s
Patients Memory
• We are interested in finding out which method is more
effective in helping patients remember the order of daily tasks
when we use different teaching methods.
• Two groups: Patients who were taught using visuals (Group 1)
vs. Patients who were taught using visuals and intense verbal
rehearsal (Group 2).
• Variable: the number of words remembered.
How Inference Works
1. State the null hypothesis.
2. Set the level of risk associated with the null hypothesis (the
level of significance or Type I error)
3. Select the appropriate test statistic (See textbook p. 211).
4. Compute the test statistic value (obtained value = the result
of a specific statistical test).
5. Determine the value needed to reject the null hypothesis
using the appropriate table of critical values
6. Compare the obtained value to the critical value
– If obtained value is more extreme, reject the null hypothesis
– If obtained value is not more extreme, accept the null hypothesis
How
Inference
Finding
the CriticalWorks
Value
• Look up Table B.2. in Appendix B.
• We need to compare the obtained value (our t-value=-.14)
and the critical value.
1. Our first task: to determine the degree of freedom (df)
– The degree of freedom approximate the sample size.
𝑛1 + 𝑛2 − 2
– In our example: the degree of freedom is 30+30-2=58.
2. Find the level of significance (we chose 5%).
3. And choose a two-tailed test because the research
hypothesis is non-directional.
How Inference Works
1. State the null hypothesis.
2. Set the level of risk associated with the null hypothesis (the
level of significance or Type I error)
3. Select the appropriate test statistic (See textbook p. 211).
4. Compute the test statistic value (obtained value = the result
of a specific statistical test).
5. Determine the value needed to reject the null hypothesis
using the appropriate table of critical values
6. Compare the obtained value to the critical value
– If obtained value is more extreme, reject the null hypothesis
– If obtained value is not more extreme, accept the null hypothesis
Compare the obtained value
How
Inference
Works
to the
critical value
• The critical value for rejection the null
hypothesis: 2.001
– What 2.001 represents: the value at which chance is the
most attractive explanation for any of the observed
differences between the two groups given 30 participants
in each group and the .05 significance level.
• The obtained value = -0.14
– If obtained value is more extreme, reject the null hypothesis
– If obtained value is not more extreme, accept the null hypothesis
• We accept the null hypothesis. At the critical
value of 2.001 the null hypothesis is the most
attractive explanation.
How the
Inference
Using
Analysis Works
Toolpak
How
Inference
Interpreting
the pWorks
value
• The p value indicates the probability of t
occurring by chance for a two-tailed test.
– When p > .05  This t value often occurs by chance.
The null hypothesis is the most attractive explanation.
 Accept the null hypothesis (your research
hypothesis is rejected)
– When p <.05  This t value rarely occurs by chance.
There must be something else going on here. The null
hypothesis is not the most attractive explanation. 
Reject the null hypothesis (your research hypothesis
is supported)
How
Inference
Works
Summarize
the Results
The Difference in Memory Test between Two Groups
Group 1
(visual)
Group 2
(visual + Verbal)
Mean
5.43
5.53
Variance
11.70
4.26
30
30
Observations
Degree of freedom
58
t statistic
-0.14
P value
0.89
The results of the analysis shows that although Group2 did have a higher
score than Group 1, that score is not significantly different. The t value
for a two-tailed test is -.14, with an associated p value of .89.
How Inference
Effect Size Works
• Suppose we have a significant difference
between groups.
• The question is not only whether it is
statistically significant but also whether it is
meaningful.
• Effect Size: a measure of how different two
groups are from one another.
How
Inference
Works
Calculating
the Effect
Size
𝑋1 − 𝑋2
𝐸𝑆 =
𝑠
In our Alzheimer’s patients example:
5.43 − 5.53
𝐸𝑆 =
3.42
The effect size is 0.03, a very small effect size.
How Inference
Group WorkWorks
Midterm
• 2 hours
• Total 100 points
1. Create a frequency table, a histogram, and
summary statistics table. Interpret them (1
problem). 40 points
2. Create charts (2 problems). 20 points
3. Interpret statistics (4 problems). 20 points
4. Multiple choice questions (10 problems). 20
points