Download Introduction to Inferential Statistics

Basic Statistics
Introduction to
Inferential Statistics
STRUCTURE OF STATISTICS
TABULAR
DESCRIPTIVE
GRAPHICAL
NUMERICAL
STATISTICS
ESTIMATION
INFERENTIAL
TESTS OF
HYPOTHESIS
Introduction to Inferential
Statistics
• Inferential statistics about the population
mean are usually used to answer one of
two types of questions.
– The first question is, What is the average
“something?” This is Estimation.
• “Something” could be hours spend
studying by online students, speed driven
by teenagers, distance people commute to
work or school, or any number of other
things.
Introduction to Inferential
Statistics
• The second type question about the population
mean is:
– “Am I right or wrong if I guess (hypothesize) the
mean “something” to be {value}? This is Hypothesis
Testing.
• Again, “something” could be hours spend
studying by online students (10 hours), speed
driven by teenagers (too fast*), distance people
commute to work or school (12 miles), or any
number of other things.
*The hypothesized value must be a value not a value
judgment!
Inferential Statistics
Inferential Statistics
Estimation
Hypothesis Testing
Confidence Intervals
t-tests
Relating to the Textbook
• Your textbook treats these two types of
questions as distinctly different, with the
Hypothesis Testing taking a
predominate role.
• I see them as closely linked and in fact,
I will show you how to do both things
with one technique.
REMINDER!!
• Much of what we will cover from here
until the end of the course is not in
sequence with your book. The material
is all there but I will be referring you to
many sections of many chapters as we
progress. You will need to pay careful
attention to the PowerPoint lessons
and be able to use your textbook as a
reference.
Some Definitions for Estimation
• Estimation: Using sample statistics to estimate
population parameters.
• Point Estimate: Use of a single number as the
estimate for unknown parameter (usually never
correct!).
• Interval Estimate: A range of values as the
estimate for the unknown parameter.
• Confidence Interval: An interval estimate
accompanied by a specific level of probability.
An Example of Estimation
Suppose a university administrator is interested in
determining the average IQ of all professors at her
university. It is too costly to test all professors, so she
selects a random sample of 20 professors. Each is given
an IQ test and the results show a sample mean of 135.
Since the test is nationally standardized, she knows that s
for the population is 15.
 How would the administrator estimate the
average IQ for ALL university professors?
Constructing a Confidence
Interval
• The general formula for a confidence
interval uses information from the sample
and our knowledge of the sampling
distribution from the Central Limit
Theorem.
• We then construct an interval in which we
think the population parameter will be.
Confidence Interval Formula
 σ 
CI  X  Z

 n
In words, the confidence interval is determined by
adding and subtracting the bound on the estimate
(the z score representing the level of confidence
times the standard error) to and from the mean from
the sample.
The Area Under the Normal Curve and
the Sampling Distribution of the Means
95%
  1.96s x 

  1.96s x 
X
The Sampling Distribution of the
Means
• From the previous slide we can see that
given our knowledge of the sampling
distribution of the means, we know that
95% of all X ' s that we would obtain from
numerous samples will fall with 1.96
standard errors s x  of the unknown .
Our University Administrator
 Let’s return to our university administrator and
see how she will estimate the average IQ [m] of
all professors at her university.
 She will need to know the
 Shape,
 Mean, and
 Standard Deviation of the
Sampling Distribution!
What about the Shape of the
Sampling Distribution?
If we had repeated taking hundreds of
samples of 20 professors, what does the
CLT tell us will be the shape of the
distribution of sample means from these
samples?
It would be approximately normal or
mound-shaped.
What about the Mean of the
Sampling Distribution?
What does the CLT tell us the mean
of the sampling distribution would be?
It would be the same as the
population, which is  and is
unknown.
What about the Standard Deviation of
the Sampling Distribution?
 What does the CLT tell us the standard deviation
(standard error) of the sampling distribution
would be?
 It would be the same as the population standard
deviation, 15, divided by the square root of the
sample size, 20.
σ
σX  X
n
 15
20
 3.35
We can display this graphically
3.35
3.35
X

σX  σX
n
 15
20
 3.35
Using the standard deviation of the sampling
distribution (standard error), we can determine a
bound on the estimate.
We know that 95% of the observations in a
distribution will fall within 1.96 standard
deviations of the mean, therefore, if we take 1.96
of the standard errors (1.96 x 3.35 = 6.57), we
know the maximum distance that our estimate
will miss the population parameter (error) 95% of
the time.
Using this information, we can determine
the points on this graph where the sample
mean would occur 95% of the time:
95%

  6.57
6.57
  (1.96  3.35)    6.57
  6.57
6.57
  (1.96  3.35)    6.57
X
Let’s illustrate the computations-First, we compute the bound on the error of estimation:
s
1.96  s X  1.96  X
n
 1.96 15
20
 6.57
We then subtract and add it to the sample mean:
s




15
X
X  1.96 

135

1
.
96


128
.
4

n 
20 

s

X  1.96  X

  135  1.96 15
  141.6

n 
20 
The Answer!
• Based on our calculations, the way to
state the estimate is:
The administrator is 95% confident
that the mean IQ for all professors at
her university is between 128.4 and
141.6.
We can show graphically the concept of the
confidence interval.
  6.57

  6.57
X
 Xˆ 
Since there is a 95% chance that the sample mean will be
in this interval, the interval around the sample mean will
capture the population mean () 95% of the time.
Important Concept
• When we construct a confidence
interval, we are not saying that the
parameter  is in the middle but merely
somewhere in that interval!! It is like
throwing a net into the sea, we hope to
catch the fish but we do not know where
the fish is. If we are really hungry, we
better throw a big net! (Which
statistically is to have a higher degree of
confidence).
How often will the 95% confidence
interval capture ?
Answer: 95% of the time
  6.57

  6.57
 Xˆ 
X
Here, the sample mean is as far left as it will fall 95% of the time.
Please note that it still captures (barely) the population mean.
  6.57

 Xˆ 
  6.57
X
Here, the sample mean is as far right as it will fall 95%
of the time. Please note that it still captures (barely)
the population mean.
  6.57

  6.57
X
 Xˆ 
Only 5 times in 100 samples will the obtained
sample mean X be so far away that a 95%
confidence interval will not capture .
  6.57

  6.57
X
 Xˆ 
Summary
The sample mean is the point estimate for the population
mean.
The standard deviation of the sampling distribution is also
called the standard error for the estimate of the mean.
1.96 standard errors provides the 95% bound on the error
of estimation.
If we add and subtract this bound from the sample mean,
we can create a confidence interval.
Finally, we can alter the confidence limits (from 95%)
depending on the distance from the mean of the distribution
that we choose.
Summary in Symbols for Estimating μ
Point Estimate  X
Error of Estimation  s
n
Bound on Error of Estimation  z s
n
Note: z would be 1.96 for 95% Bound, 2.575
for 99% Bound, 1.64 for 90% Bound, etc.
Confidence 
s
X

z

Interval:

n
to X  z s


n
One–Sample Test of Hypothesis
Hypothesis Testing on a Population Mean

population
X
sample
Research Situation
 A particular test has a national mean and
standard deviation of 100 and 15 respectively.
The superintendent of a particular school
system wants to know if the average IQ in her
school system is different than the national
average on this test.
population
Definitions Related to
Hypothesis Testing
• Null Hypothesis: The hypothesis that we will
test statistically. In a single sample problem it
is the “guess” about the population mean ().
– Written as: Ho:  = value.
• Alternative Hypothesis: If the null is not
feasible, then the alternative must be.
– Written as: Ha:   ‘value’, or  < ‘value’, or  >
‘value’
Step by Step: The One-Sample Test
of Hypothesis Using the z-test.
1. State Research Question
2. Establish the Hypotheses
3. Establish Level of
Significance
4. Collect Data
5. Calculate Statistical Test
6. Interpret the Results
1. Stating the Research Problem
Is the average IQ of students in that particular
system different from the national average?
0
population
Difference?
sampling
sample
X
2. Establish the Research
Hypothesis
Research or Alternative The mean IQ is not equal to 100.
Hypothesis
H a :   100
Null Hypothesis
The mean IQ is 100.
H o :   100
Alternative or Research
Hypotheses
• The Alternative Hypotheses may take
either a non-directional form,  = ‘value’.
• The Alternative Hypotheses may be a
directional hypothesis,  > ‘value’ or
 < ‘value’.
• The decision to use a directional
alternative is based on the research
question under investigation.
Errors in Decisions
Our Decision
Null is Really
True
Null is Really
False
Accept Null
Hypothesis
Good Decision Bad Decision
Type II Error
(b)
Reject Null
Hypothesis
Bad Decision
Type I Error
(a)
Good Decision
(Power)
3. Establish the Level of
Significance
 a is the probability of rejecting a true null
hypothesis and will be equal to the area NOT
within the area we would expect to find our
sample mean (e.g., if we use 95% under the
curve, then a is .05).
 a defines what is called the “Rejection
Region” because we will reject the Null if our
calculated z statistic is in that region.
Graphical Depiction of
Rejection Region
Rejection Region
Rejection Region

Hypothesized
Rejection Region
Directional Hypotheses

+1.645
This would represent a directional hypothesis  > ‘value’. The
total area would be on only one side, e.g., .05, thus the critical
value of z would be 1.645 rather than 1.96, giving a greater
likelihood of rejecting the Null Hypothesis.
Rejection Region
Directional Hypotheses
-1.645

This would represent a directional hypothesis  , ‘value’. The total
area would be on only one side, e.g., .05, and again, the critical
value of z would be 1.645 rather than 1.96, giving a greater
likelihood of rejecting the Null Hypothesis.
Rejection Region
• It is determined by the Alpha (a) selected.
a defines how much of the area under the
curve will be in the rejection region.
• The probability of rejecting a TRUE Null
Hypothesis is equal to the area in the
rejection region, since a sample mean will
only be obtained that frequently; if the Null
Hypothesis is True.
4. Collecting the Data
A random sample of 81 students were given the IQ
test and their scores were recorded. The mean of the
sample was 105.
Student
IQ
1
2
109
88
81
122
5. Analyzing the Data:
Calculating the Test
Statistic
The statistic we will calculate to
determine if the Research
Hypothesis is tenable is a
modification of our z score.
Notice the new formula uses the
data from the sampling
distribution X ' s and the population
mean m divided by the standard
error. These are exactly as we
discussed in Estimation.
X X
Z
S
Z
X 
sx
Calculating the Z Statistic
• From our sample of 81 students, we
calculated the sample mean to be 81.
• The population standard deviation is 15.
• Using our Z test formula we can determine
where our sample mean would fall, if the
population mean m is 100.
The Z Statistic
Z
X 
sx
105  100
5


 2.99
15
1.667
81
Thus, our sample mean lies 2.99 standard
deviations (standard errors) above the
population mean of 100.
Locating our Mean on the
Sampling Distribution of Means
95% of all means
 – 1.667
98.33

100
 + 1.667
101.67
105
6. Interpreting the Results
• Since our mean is not in the area where we
would expect 95% of all sample means from
a distribution where the population mean is
100, we would reject the Null Hypothesis
that  = 100 and accept the alternative that it
is different  = 100.
• We would state that we reject the Null
Hypothesis at the .05 level of confidence.
Problems with Z Test
• The z test requires that we know the
population standard deviation, which we
usually do not know.
• The z test is designed for large samples
(n> 30), again which we don’t always
have.
• What is the solution?
Solution
Use a t-distribution rather than the
z-distribution
z
X  0
s
n
(See page 297-298)
X  0
t
s
n
Characteristics
z-distribution
• Mean of 0
• Mound-Shaped
(Normal)
• SD is same as
population except
divided by the square
root of n
t-distribution
• Mean of 0
• Mound-Shaped (Not exactly
Normal)
• SD is same as the sample
except divided by square root
of n
• Thus, t is more variable than
z--depends on degrees of
freedom
Understanding the t-Distribution
Recall the difference between the sample and population variances.
Population

X  μ

σ 
2
2
N
Sample

X  X

S 
2
2
n1
Other than using sample numerical indicators rather than
population numerical indicators, the only difference is that the
sample variance is divided by “n - 1” rather than “N”.
The reason for this when S2 is used to estimate σ2, it tends to
underestimate it. A man named William S. Gosset discovered
that dividing by n-1 corrected this problem. He also discovered
that n-1had greater significance in statistics and it was called the
degrees of freedom.
William Gosset
William Gosset was the quality control
engineer at Guinness Brewery in London in the
early 1900s. For some reason, getting samples
of 30 or more of his produces proved difficult.
This prompted his search for a small-sample
statistic that resulted in his publication of the ttest. He published it under the pen name of
“Student” for a couple of reasons. First,
moonlighting was frowned upon by Guinness.
Second, he wanted to honor his teachers,
particularly Karl Pearson.
Review Degrees of Freedom
Degrees of freedom are the number of observations free to vary, thus the
number of observations that contribute to the variance in a sample.

X X


2
Recall the formula for
the sample variance:
S
2
n 1
In order to compute the variance, we first must compute
the sample mean. We find the sample mean by summing
the scores and dividing by n.
Consider a problem with n = 3
and the sample mean = 5. Thus,
And if we know the
first two numbers are
3 and 8….
X

X
15 3  8  ?

3
3
n
? 15
5 
3
3
The ? must be 4. Thus, only 2 of the 3
scores are free to vary. That is to say
there are n – 1 degrees of freedom.
This concept of degrees of freedom
is used for many different statistics,
not just the t-statistic.
The t-distribution is presented in tables, but
not complete tables like the normal curve
z-scores. This is because it would take a
different table for each different degree of
freedom. Thus, only commonly used alpha
values are tabulated.
New Research
Situation
 The current rate for producing 5 amp fuses at
Moe’s Electric Company is 250 per hour. A
new machine has been purchased and
installed that, according to the supplier, will
increase the production rate. Is the new
machine faster than the old one?
population
Is there significant difference in mean score of
Dependent variable between Sample and
Population?
?
XA
Sample
population
Step by Step: The One-Sample Test
of Hypotheses using the t Test
1. State Research Question
2. Establish the Hypotheses
3. Establish Level of
Significance
4. Collect Data
5. Calculate Statistical Test
6. Interpret the Results
1. Stating Research Problem:
Is the production rate of new machine more than
250 per hour?
0
population
Difference?
sampling
sample
X
2. Setting Hypotheses
Null Hypothesis
The mean production rate of the new machine is equal to
or less than 250 per hour. Notice that the null now
contains the other side of the directional alternative!
Ho : μ  250
Research or Alternative Hypothesis
The mean production rate is greater than 250 per hour.
Ha : μ  250 One-tail test
3. Setting your level of significance
Possible Conclusion
Two-tailed
One-tailed
test of significance
test of
The sample
is significance
from a
population with a mean
H :   250
greater than 1that1 of the null
hypothesis
H1 : 1  250
if a  .05
a  .05
Accepting
H0 : 0  250
0
Rejecting H0 : 0  250
H1 : 1  250
4. Collecting Data
A sample of 10 randomly selected hours from last
month revealed the mean hourly production on the
new machine was 256, with a sample standard
deviation of 4.67 per hour.
Hours
Production
1
2
254
253
10
250
5. Analyzing the Data:
Calculating the Test
Statistic
The t-test formula uses the data X ' s
from the sampling distribution
and the population mean m
divided by the standard error,
which is now defined by using
the sample standard deviation
and not the population standard
deviation as required in the z
test. These are the same as we
discussed in Estimation.
X μ
t
sx
Steps in Calculating the t Test
• From our sample of 10 randomly selected
hours, we calculated the sample mean to
be 256 and the standard deviation to be
4.67.
• Using our t test formula we can determine
where our sample mean would fall, if the
population mean  is 250.
Calculating the t statistic
X μ
t
sx
256  250
6

 4.06
=
4.67
1.48
10
Thus, our obtained mean from our sample is 4.06
standard errors above the hypothesized mean of 250.
Would this value be in our rejection region?
YES!
Graphical representation of our calculations
a  .05
Accepting H0 : 0  250
Rejecting H0 : 0  250
H1 : 1  250
0
4.06
6. Interpreting the Results
a  .05
250
Accepting HH0 :0 : 00 250
Rejecting H0 : 0  250
H1 : 1  250
0
250
Sampling Distribution of
4.066
256
X
Difference in standard errors
4.066 between sample mean and
population mean
6. Interpreting the Results
• Since our sample mean is not in the area
where we would expect 95% of all sample
means from a distribution where the
population mean is 250 and is in our
rejection region, we would reject the Null
Hypothesis that  = 250 and accept the
alternative that it is different  >250.
• We would state that we reject the Null
Hypothesis at the .05 level of confidence.
Assumption Required
Since the n is too small to invoke the Central Limit
Theorem, we can no longer be sure that the sampling
distribution is normal. In fact, we have already
learned that it is not, it is a t-distribution. In order
for this to happen, we must assume that the original
distribution is normally distributed.
The smaller the n, the more important this assumption is. For
example, with an n of 4, normality might be important.
However, by the time n is 30 or more, the normality
assumption is not necessary as the CLT takes over. We can
see this by examining a t-table (Table D).
Assumption Summary
For a one-sample z-test, there is only
one assumption:
●Sample was obtained randomly
For a one-sample t-test, there are two
assumptions:
●Sample was obtained randomly
●Original population is normally distributed
Comparing Estimation and
Hypothesis Testing
• In estimation, we use data from a sample
to estimate where we think, with a
declared level of confidence, that the
population mean to be.
• In hypothesis testing, we use data from a
sample to evaluate the acceptability of the
hypothesized value for the population
mean.
Comparing the Two Formulae
 s 
CI  X  t

 n
X  μ0
t
s
n
Notice that I have changed the earlier
confidence interval formula with the
information from the t distribution. Also
note the same standard error is used in both.
Calculating the t Statistic and the
95% Confidence Interval
X 
t
sx
=
256  250
6

 4.06
4.67
1.48
10
 s 
95%CI  X  t 
  256  1.833(1.48)
 n
*The value of 1.833 (called the critical value) is found in
Table D, page 538, using one-tailed .05 and 9 degrees of
freedom. We must remember that our alternative was that
 was greater than 250, which it was.
Comparing the 95% Confidence
Interval and the t-test.
 95% CI = 256 + 2.7 = 258.7 and
= 256 – 2.7 = 253.3.
 We are 95% confident that the population mean
is between 253.3 and 258.7, based on the data
from our random sample.
 The calculated t statistic was 4.06, meaning that
the obtained mean is 4.06 standard errors above
the hypothesized mean of 250, and therefore we
rejected the null of 250.
A Graphical Comparison
Rejection Region
250
t =4.06
Hypothesized 
253.3
95% confidence Interval
258.7
Arriving at the Same
Conclusion
• Notice that the confidence interval does not
contain the hypothesized value [250], thus it
is a bad guess.
• General rule: If the confidence interval does
not contain the hypothesized value we will
reject the null hypothesis, the same as if we
had calculated the t statistic.
• Thus, we can conduct a test of hypothesis
simply by calculating the confidence interval.