Download Chapter 16 Slides: Inference in Practice

Chapter 16: Inference in Practice
STAT 1450
16.0 Inference in Practice
Connecting Chapter 16 to our Current
Knowledge of Statistics
▸ Chapter 14 equipped you with the basic tools for confidence interval construction.
▸ Chapter 15 equipped you with the basic tools for tests of significance.
▸ Chapter 16 addresses some of the nuances associated with inference
(our owner’s manual of sorts).
16.1 Conditions for Inference
Conditions for Inference
▸ Random sample:
 Do we have a random sample?
 If not, is the sample representative of the population?
 If not, was it a randomized experiment?
▸ Large enough population: sample ratio
 Is the population of interest at least 20 times larger than the sample?
▸ Large enough sample:
 Are the observations from a population that has a Normal distribution, or one where
we can apply principles from a Normal distribution? Look at the shape of the
distribution and whether there are any outliers present.
16.2 Cautions about Confidence Intervals
Cautions about Confidence Intervals
The margin of error covers only sampling errors.
▸ Undercoverage, nonresponse, or other biases are not reflected in
margins of error.
▸ The source of the data is of utmost importance.
▸ Consider the details of a study before completely trusting a confidence
interval.
16.2 Cautions about Confidence Intervals
Example: Parental Monitoring Software
▸ Many parents elicit the use of various software and passwords to
monitor the ways children use their computers. In a survey of a random
sample of high school students, 16.7% with 3.45% margin of error
expressed an ability to circumvent their parent’s security efforts.
Would you trust a confidence interval based upon this data? Explain.
The Confidence Interval would be (.1325, .2015).
Yes, it is from a random sample.
But, there is likely some under-reporting by the teens.
As mentioned in Chapter 8, people tend to provide conservative answers to provocative
questions.
16.3 Cautions about Significance Tests
Cautions about Significance Tests
▸ When H0 represents an assumption that is widely believed,
small p-values are needed.
▸ Be careful about conducting multiple analyses for a fixed a.
 It is preferred to just run a single test and reach a decision.
16.3 Cautions about Significance Tests
Cautions about Significance Tests
▸ When there are strong consequences of rejecting H0 in favor of HA, we
need strong evidence.
▸ Either way, strong evidence of rejecting H0 requires small p-values.
▸ Depending on the situation, p-values that are below 10% can lead to
rejecting H0.
▸ Unless stated otherwise, researchers assume the de-facto
significance level of 5%.
16.3 Cautions about Significance Tests
Cautions about Significance Tests
▸ The P-value for a one-sided tests is half of the P-value for the
two-sided test of the same null hypothesis and of the same data.
▸ The two-sided case combines two equal areas. The one-sided case
has one of those areas PLUS an inherent supposition by the
researcher of the direction of the possible deviation from H0.
16.3 Cautions about Significance Tests
A Connection between Confidence Intervals and
Significance Tests
▸ Analogous to how we use high levels of confidence for confidence
intervals, we need strong evidence (and very small p-values) to reject
null hypotheses.
▸ Standard levels of confidence are 90%, 95%, and 99%.
▸ Standard levels of significance are 10%, 5%, and 1%.
Recall from last chapter: more than 10% was a “likely” event.
5% to 10% was an “unlikely” event.
< 5% was an “extremely unlikely” event.
16.3 Cautions about Significance Tests
Sample Size affects Statistical Significance
▸ Very large samples can yield small p-values that lead to rejection of H0.
▸ Phenomena that are “statistically significant” are not always “practically
significant.”
16.3 Cautions about Significance Tests
Example: Carry-on luggage
▸ Airlines are now monitoring the amount of carry-on luggage passengers bring with them.
It is believed that the mean weight of carry-on luggage for passengers on multiple hour
flights is 30 lbs. with a standard deviation of 7.5 lbs. A random sample of 500,000
passengers who had recently flown on multiple hour flights had an average carry-on
luggage weight of 29.9 lbs.
▸ The test statistic is -9.43 with a P-value of 0.
▸ There is a statistically significant reason to reject the H0 and believe that the mean
weight of carry-on luggage is not 30 lbs. But, practically, the sample mean (29.9) and the
population mean (30.0) are quite comparable.
16.3 Cautions about Significance Tests
Example: Carry-on luggage
𝑥 − 𝜇0
𝑧=
=
𝜎 𝑛
16.3 Cautions about Significance Tests
Example: Carry-on luggage
𝑥 − 𝜇0
29.9 − 30.0
𝑧=
=
𝜎 𝑛 7.50 500000
16.3 Cautions about Significance Tests
Example: Carry-on luggage
𝑥 − 𝜇0
29.9 − 30.0
𝑧=
=
= −9.43
𝜎 𝑛 7.50 500000
16.3 Cautions about Significance Tests
Cautions about Significance Tests
▸ Be advised that it is better to design a single study and conduct one
test of significance - (yielding one conclusion) than to design 1 study,
and perform multiple analyses until a desired result is achieved.
16.4 Planning Studies: Sample Size for
Confidence Intervals
Sample Size for Confidence Intervals
▸ In Chapter 14, we noticed that the sample size impacts the margin of
error (and thus the width of the confidence interval).
▸ The margin of error is expressed as 𝒙 ± 𝒎
where 𝒎 = 𝒛∗
𝝈
𝒏
▸ Some algebra leads us to a formula for the minimum sample size for a
desired margin of error:
 The z confidence interval for the mean of a Normal population will have a specified
margin of error m when the sample size is
𝑧∗𝜎
𝑛=
𝑚
2
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
▸ In the carry-on luggage example from earlier, a random sample of
500,000 passengers yielded a standard deviation for the sample mean
that was extremely small; resulting in |z| ≈ 9.43.
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
▸ In the carry-on luggage example from earlier, a random sample of
500,000 passengers yielded a standard deviation for the sample mean
that was extremely small; resulting in |z| ≈ 9.43.
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
29.9 – 30 = | -.1| = .1= m
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
𝟕. 𝟓
. 𝟏 = 𝒎 = 𝟗. 𝟒𝟑
29.9 – 30 = | -.1| = .1= m
𝟓𝟎𝟎𝟎𝟎𝟎
Larger n, smaller m.
Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse.
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
𝟕. 𝟓
. 𝟏 = 𝒎 = 𝟗. 𝟒𝟑
29.9 – 30 = | -.1| = .1= m
𝟓𝟎𝟎𝟎𝟎𝟎
Larger n, smaller m.
Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse.
If we desire to double the m (m=.20) we need one-fourth the sample size..
𝑚 = 9.43 ∗
7.5
500000(1 4
= 9.43 ∗
7.5
125000
= .2
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
Larger n, smaller m.
Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse.
If we desire to double the m (m=.20) we need one-fourth the sample size..
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
Larger n, smaller m.
Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse.
If we desire to double the m (m=.20) we need one-fourth the sample size..
𝑚 = 9.43 ∗
7.5
500000(1 4
= 9.43 ∗
7.5
125000
= .2
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
Poll: Would you expect that ________
a) more or b) fewer
passengers would need to be sampled to estimate the mean
weight of carry-on luggage within a margin of error of 0.2 lbs. with
95% confidence?
Larger n, smaller m.
Quadrupling the sample size, divides margin of error in half.. Let’s try the inverse.
If we desire to double the m (m=.20) we need one-fourth the sample size..
𝑚 = 9.43 ∗
9.43 ∗
7.5
500000(1 4
7.5
125000
= 9.43 ∗
7.5
125000
= .2 = 𝑚 = 1.96 ∗
Taking one-25th of 125,000 is about 5000.
= .2
7.5
125000𝑐2
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
▸ Explicitly determine the sample size.
∗
𝑧 𝜎
𝑛=
𝑚
2
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
▸ Explicitly determine the sample size.
∗
𝑧 𝜎
𝑛=
𝑚
2
∗
1.96 7.5
=
.2
2
16.4 Planning Studies: Sample Size for
Confidence Intervals
Example: Carry-on luggage
▸ Explicitly determine the sample size.
∗
𝑧 𝜎
𝑛=
𝑚
2
∗
1.96 7.5
=
.2
2
= 5402.25 ⇒ 5403
Resulting in a much smaller sample size.
16.5 Errors in Significance Testing
Example: The Justice System
Jury Verdict
Truth about the Defendant
Innocent
Guilty
Not Guilty
Guilty
16.5 Errors in Significance Testing
Example: The Justice System
Jury Verdict
Truth about the Defendant
Innocent
Correct decision
Guilty
Not Guilty
Guilty
Correct decision
16.5 Errors in Significance Testing
Example: The Justice System
Jury Verdict
Truth about the Defendant
Innocent
Guilty
Guilty
Error
Correct decision
Not Guilty
Correct decision
Error
16.5 Errors in Significance Testing
Power, Type I Error, and Type II Error
Decision based
on data
Truth about a hypothesis
Ho is true
Reject Ho
Fail to reject Ho Correct decision
Ha is true
16.5 Errors in Significance Testing
Power, Type I Error, and Type II Error
Decision based
on data
Truth about a hypothesis
Ho is true
Reject Ho
Fail to reject Ho Correct decision
Ha is true
Correct Decision
16.5 Errors in Significance Testing
Power, Type I Error, and Type II Error
Decision based
on data
Reject Ho
Truth about a hypothesis
Ho is true
Ha is true
Type I Error (𝛼
Correct Decision
Fail to reject Ho Correct decision Type II Error (𝛽
16.5 Errors in Significance Testing
Power, Type I Error, and Type II Error
▸ Type I Error – the maximum allowable “error” of a falsely rejected H0
(also the significance level, a).
▸ Type II Error – the probability of not rejecting H0, when we should have
rejected it.
▸ Power – the probability that the test will reject H0 when the alternative
value of the parameter is true.
 Note: Increasing the sample size increases the power of a significance test.
▸ Effect size – the departure from a null hypothesis that results in
practical significance.
16.5 Errors in Significance Testing
Example: Coffee consumption
▸ Recall the coffee consumption example from last chapter with standard
deviation of 9.2 oz. A random sample of 48 people drank an average of
26.31 oz. of coffee daily. A significance test of the mean being different
from our original estimate is conducted. Provide examples of a, b, and
power.
16.5 Errors in Significance Testing
Example: Coffee Consumption
▸ Recall the coffee consumption example from last chapter with standard
deviation of 9.2 oz. A random sample of 48 people drank an average of
26.31 oz. of coffee daily. A significance test of the mean being different
from our original estimate is conducted. Provide examples of a, b, and
power.
 𝛼: We reject H0 (𝜇 = “20”), when it was actually true.
 𝛽: We fail to reject H0, when actually the true mean m was not “20.”
 Power: Probability of rejecting H0 for a specific value of 𝜇 ≠ 20.
Five-Minute Summary
▸ List at least 3 concepts that had the most impact on your knowledge of
inference in practice.
___________
_____________
_______________