Download Linear regression

Normal Distribution
Normal distribution
We often meet the situation when the
frequency of some occurances depend on the
distance from the mean value  close to mean
are very frequent, away from mean less
frequent:
- Human height, temperature, machined
products, sales, financial data
Lower tail
Upper tail
-3
-2
-1
0
1
2
3
Normal distribution
𝜎
-3
Position
-2
-1
0
𝜇
1
2
3
Position
Normal distribution
𝜇=0
𝜇 = −1
𝜇 = −2
-3
-2
-1
0
1
2
3
Normal distribution
𝜎 𝑠𝑚𝑎𝑙𝑙𝑒𝑟
narrow
𝜎 𝑠𝑚𝑎𝑙𝑙𝑒𝑟
narrow
𝜎 𝑙𝑎𝑟𝑔𝑒𝑟
widen
-3
-2
-1
0
1
2
3
Standard normal curve
Area under the curve
1
-3
-2
Z distribution
𝜇=0
𝜎=1
-1
0
1
2
3
Standard normal curve
0.5
0.8413
0.1587
Z distribution
𝜇=0
𝜎=1
0.0228
0.3413
0.1359
0.0013
0.1359
0.0214
-3
-2
0.9772
0.3413
0.9982
0.0214
-1
0
1
2
3
Standard normal curve
0.5
68.26%
−𝟏𝝈
-3
-2
-1
0
+𝟏𝝈
1
2
3
Z distribution
𝜇=0
𝜎=1
Standard normal curve
0.5
95.44%
−𝟐𝝈
-3
-2
-1
0
+𝟐𝝈
1
2
3
Z distribution
𝜇=0
𝜎=1
Standard normal curve
0.5
99.74
−𝟑𝝈
-3
-2
-1
0
+𝟑𝝈
1
2
3
Z distribution
𝜇=0
𝜎=1
Apple corporation 2012 Daily Returns
1. What is the probability, for any
given day, of a return greater
than 0.5%?
2. What is the probability, for any
given day, of a loss greater than
2%?
3. What is the probability, for any
given day, of a return beween 0
and 1%?
4. What is the probability, for any
given day, of a gain or loss
greater than 3%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
0.5 %
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Apple corporation 2012 Daily Returns
1. What is the probability, for any
given day, of a return greater
than 0.5%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
𝑥−𝜇
𝑧=
𝜎
0.5 − 0.11
𝑧=
= 0.21
1.84
= norm.dist(0.21,0,1,TRUE)=0.58
0.5 %
1 - 0.58
= 0.42
0.58
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Apple corporation 2012 Daily Returns
2. What is the probability, for any
given day, of a loss greater than 2%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
𝑥−𝜇
𝑧=
𝜎
−2 − 0.11
𝑧=
= −1.15
1.84
= norm.dist(-1.15,0,1,TRUE)=0.125
0.125
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Apple corporation 2012 Daily Returns
3. What is the probability, for any
given day, of a return beween 0 and
1%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Apple corporation 2012 Daily Returns
3. What is the probability, for any
given day, of a return beween 0 and
1%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
𝑥−𝜇
𝑧=
𝜎
1 − 0.11
𝑧=
= 0.48
1.84
𝑥−𝜇
𝑧=
𝜎
0 − 0.11
𝑧=
= −0.06
1.84
= norm.dist(0.48,0,1,TRUE)=0.69
= norm.dist(-0.06,0,1,TRUE)=0.48
0.21
0.21
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Apple corporation 2012 Daily Returns
4. What is the probability, for any
given day, of a gain or loss greater
than 3%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Apple corporation 2012 Daily Returns
4. What is the probability, for any
given day, of a gain or loss greater
than 3%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
𝑥−𝜇
𝑧=
𝜎
3 − 0.11
𝑧=
= 1.57
1.84
𝑥−𝜇
𝑧=
𝜎
−3 − 0.11
𝑧=
= −1.69
1.84
= norm.dist(-1.69,0,1,TRUE)= 0.046
= 1- norm.dist(1.57,0,1,TRUE)=0.058
0.104
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Mean and variance
Statistical quality control
High Way Paving inc. Is a company specializing in residential road
surfacing using low noise pavement. Recycled rubber can be added to
asphalt mixtures to reduce road noise. However resistance to flow
must be maintained within very tight limits otherwise it may too thick
or too „watery”. The goal is a viscosity of 3200.
Over several years of production and quality measurements, HWP has
determined that viscosity population mean and standard deviation is:
𝜇 = 3200
𝜎 = 150
Statistical quality control
During manufacture of each batch of asphalt, the quality control
specialist takes 15 samples of the material and tests the viscosity.
There is no way test every kg of asphalt (population). Therefore the
company must take samples.
From those sample HWP must then make conclusions about the entire
batch.
Statistical quality control
Sample 1
Sample 2
Sample 3
Sample 4
𝑥 = 3210.73
𝑥 = 3150.13
𝑥 = 3345.54
𝑥 = 3190.67
Sample 5
Sample 6
Sample 7
Sample 8
Sample 9
𝑥 = 3217.90
𝑥 = 3301.45
𝑥 = 3100.72
𝑥 = 3413.01
𝑥 = 3023.59
Statistical quality control
Sample
Sample mean (𝒙)
Range
Frequency
1
3210.73
2950-3049
I
2
3150.13
3050-3149
I
3
3345.54
3150-3249
IIII
4
3190.67
3250-3349
II
5
3217.90
3350-3449
I
6
3301.45
7
3100.72
8
3413.01
9
3023.59
We call this the:
Sampling distribution
(distribution of sample means)
𝐸 𝑥 = 3217.08
Viscosity sampling distribution
4.5
4
3.5
3
2.5
2
1.5
1
0.5
0
2950-3049
3050-3149
3150-3249
3250-3349
3350-3449
Viscosity sampling distribution
4.5
𝐸 𝑥 =𝜇
4
3.5
As n  „large”
3
2.5
2
𝐸 𝑥 = 𝑡ℎ𝑒 𝑒𝑥𝑝𝑒𝑐𝑡𝑒𝑑 𝑣𝑎𝑙𝑢𝑒 𝑜𝑓 𝑥
𝜇 = 𝑡ℎ𝑒 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑚𝑒𝑎𝑛
1.5
1
0.5
0
2950-3049
3050-3149
3150-3249
3250-3349
3350-3449
If we take many random samples from the population each with its own sample mean and then create a
distribution based of all of those sample means, the mean of that sampling distribution is equal to the mean
of the population.
• The expected value of the sampling distribution of 𝑥 us at best going
to be an estimate of 𝜇,
• We would have to take every sample from the popoulation to match
the population mean perfectly (but what is the point of sampling
then?)
• The best we are going to be able to do is find and interval estimate for
the population mean 𝜇,
• Our interval estimate will be influenced by sample size and the degree
of confidence we are satisfied with.
Standard deviation of 𝑥 sampling distribution
𝜎
𝜎𝑥 =
𝑛
4.5
4
3.5
3
𝜎𝑥 - standard deviation of 𝑥
2.5
2
𝜎 - standard deviation of population
1.5
1
n – sample size
0.5
0
2950-3049
3050-3149
3150-3249
In this case we know standard deviation of the population which is rarely the case.
3250-3349
3350-3449
Standard deviation of 𝑥 sampling distribution
𝜎
𝜎𝑥 =
𝑛
4.5
4
3.5
3
𝜎𝑥 - standard deviation of 𝑥
2.5
2
𝜎 - 150
1.5
1
0.5
n – 15
0
2950-3049
𝜎𝑥 =
𝜎
150
=
= 38.7
𝑛
15
3050-3149
3150-3249
3250-3349
𝑊𝑖𝑡ℎ 𝑖𝑛𝑐𝑟𝑒𝑎𝑠𝑖𝑛𝑔 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒 𝜎𝑥 𝑔𝑒𝑡𝑠 𝑠𝑚𝑎𝑙𝑙𝑒𝑟.
3350-3449
Standard deviation of 𝑥 sampling distribution
A larger sample size decreases standard error.
The values of 𝑥 will have less variation and therefore
be closer to 𝜇.
𝜎
150
𝜎𝑥 =
=
= 6.71
𝑛
500
𝜎𝑥 =
𝜎
150
=
= 38.7
𝑛
15
𝜎𝑥 =
𝜎
150
=
= 12.9
𝑛
135
𝜎𝑥 =
Statistical quality control
𝜎
150
=
= 38.7
𝑛
15
Sample 1
Sample 2
Sample 3
Sample 4
𝜎𝑥 = 38.7
𝜎𝑥 = 38.7
𝜎𝑥 = 38.7
𝜎𝑥 = 38.7
𝑥 = 3210.73
𝑥 = 3150.13
𝑥 = 3345.54
𝑥 = 3190.67
Sample 5
Sample 6
Sample 7
Sample 8
Sample 9
𝜎𝑥 = 38.7
𝑥 = 3217.90
𝜎𝑥 = 38.7
𝑥 = 3301.45
𝜎𝑥 = 38.7
𝑥 = 3100.72
𝜎𝑥 = 38.7
𝑥 = 3413.01
𝜎𝑥 = 38.7
𝑥 = 3023.59
• Standard error is stanard deviation, it allows us to calculate z-scores
and therefore area (probability) under the curve for certain region,
• Any point estimator is an estimation and will contain error,
• This error can be minimized by selecting large sample from the
population from which to estimate a parameter
• Error component means we cannot determine single value of the
parameter, we can only provide a range or interval that may cover the
parameter  confidence interval
• Most often we do not know standard deviation and we have to
estimate it
Hypothesis testing
𝐻0 : 𝜇 = 𝜇0
𝐻𝑎 : 𝜇 ≠ 𝜇0
𝜇 is the true mean of population under analysis
𝜇0 is the hypothesized mean of the population under analysis
Is the true mean the same as the hypothesized mean?
Example 1
A bottled water company states on the product label that each bottle
contains 355 ml of water. Your work for a goverment agency that
protects consumers by testing products volumes. A sample of 50
bottles is tested. Establish null and alternative hypothesis.
What is our assumption?
We assume that 355 ml on the bottle is to be true. So:
𝐻0 : 𝜇 = 355 𝑚𝑙
𝐻𝑎 : 𝜇 ≠ 355 𝑚𝑙
If the data indicates the bottles are being
filled properly, then we fail to reject the
null, fail to reject our assumption. We
are not saying we have proven the null
just that our assumption held up.
Example 2
According to the United States Department of Agriculture, in 2006 the
average farm size in the state of Texas was 2.3 km2. Since the dacadeslong trend as been for farm sizes to increase due to large
agrobussiness, we want to analyze if farm size in 2015 larger than it
was in 2006. Establish null and alternative hypothesis.
What is our assumption?
We assume that there has been no change is farm size since 2006. We
wish to see if the farm has increased since 2006 So:
𝐻0 : 𝜇 ≤ 2.3 𝑘𝑚2
𝐻𝑎 : 𝜇 > 2.3 𝑘𝑚2
Example 3
During the 2010-2011 English Premier League season Manchester
United home matches had an average attendace of 74,691. A club
marketing analyst would like to see if attendance decreased during the
most recent season. Establish null and alternative hypothesis.
What is our assumption?
We assume that attendance remained the same. We wish to see if the
attendanced has decreased since 2010-2011 So:
𝐻0 : 𝜇 ≥ 74,961
𝐻𝑎 : 𝜇 < 74,961
Two scenarios - 𝜎 known or unknown?
1. The population standard deviation 𝜎 is given.
2. The population standard deviation 𝜎 is not given and we have to
estimate it, s.
• When 𝜎 is given (or n > 100) we use the normal standard zdistribution,
• When 𝜎 is not given we use a t-distribution with n-1 degrees of
freedom
• It is better to check data for normality
Z-test for a single mean
𝑥 − 𝜇 0 𝑥 − 𝜇0
𝑧=
=
𝜎
𝜎
𝑥 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑚𝑒𝑎𝑛
𝜇0 − ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑧𝑒𝑑 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑚𝑒𝑎𝑛
𝜎 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛
𝑛 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒
The Standard Normal Curve
SNC is also a sampling distribution of the mean 𝑥.
The distribution of many sample means of given
sample size.
𝒙
-3
-2
-1
0
1
2
3
95% Probability Interval
The are in the tails is
called alpha.
Therefore we have 5%
left evenly divided
between both tails.
𝛼 = 5% 𝑜𝑟 0.05
𝒙
95% of all
sample means
(𝑥) are in here.
5%
= 2.5% 𝑜𝑟 0.025
2
5%
= 2.5% 𝑜𝑟 0.025
2
𝜶
𝟐
𝜶
𝟐
-3
-2
-1
0
1
2
3
95% Probability Interval
By doing so, we can
assign z-scores (t-scores)
to the uppar and lower
boundary of the 95%
interval.
If the population’s
standard deviation 𝜎 we
treat the sampling
distribution as a standard
normal curve (z-curve).
If not we have to
estimate 𝜎 and use t
distribution (t-curve).
𝒙
95% of all
sample means
(𝑥) are in here.
−𝟏. 𝟗𝟔𝝈
𝜶
𝟐
-3
-2
+𝟏. 𝟗𝟔𝝈
𝒙 ± 𝟏. 𝟗𝟔𝝈𝒙
-1
0
1
𝜶
𝟐
2
3
95% Probability Interval
The standard error of the
mean 𝜎𝑥 depends on:
𝜎
• Sample size (𝜎𝑥 = ),
What is the standard
deviation of the sampling
distribution?
𝑛
The standard error of the
mean 𝜎𝑥 (not 𝜎 – the
standard deviation of the
population)
𝒙
95% of all
sample means
(𝑥) are in here.
−𝟏. 𝟗𝟔𝝈
𝜶
𝟐
-3
-2
+𝟏. 𝟗𝟔𝝈
𝒙 ± 𝟏. 𝟗𝟔𝝈𝒙
-1
0
1
𝜶
𝟐
2
3
Two-tailed z-test rejection region
The critical value is determined by 𝛼 and if we are
using z- or t-test distribution.
𝐻0 : 𝜇 = 𝜇0
𝐻𝑎 : 𝜇 ≠ 𝜇0
𝛼 = 0.05
With 𝛼 and 𝜎 known we would consult the z-table and
find the corresponding z-scores for a two-tailed test.
Z = +1.96
Critical Value
𝒙
Z = -1.96
Critical Value
Nonrejection region
Rejection region
𝛼
= 0.025
2
-3
Rejection region
𝛼
= 0.025
2
𝝁𝟎
-2
-1
0
1
2
3
The Student’s T-distribution
1. In general, the t-distribution is shorter
in the middle and fatter in the tails.
2. More probability in the tails, less near
the mean, grater chance of extreme
values.
3. There isn’t just one t-distribution.
4. There is a t-distribution for every
sample size.
5. Degrees of Freedom (n-1).
6. Smaller the sample size, the shorter
and fatter the distribution, more tail
probability.
7. However as n becomes large, the tdistribution tends to z-distribution.
Area (probability) under the curve = 1
When we do not know 𝜎 we have to
estimate. This estimation, this
uncertainty, forces us to use the
Student’s T-dsitribution.
-4
-3
-2
-1
0
1
2
3
4
The Student’s T-distribution 95% Prob. Int.
By doing so, we can assign t-scores to the
upper and lower boundary of the 95%
interval of each sample size.
Area (probability) under the curve = 1
If we do not know the population
standard deviation 𝜎 we treat the
sampling distribution as a t-distribution.
Degrees of Freedom (n-1)
n = 10
𝒙
z = -1.96
z = +1.96
95% of all
sample means
(𝑥) are in here.
t = -2.26
df = 9
t = +2.26
𝑡𝛼 𝑑𝑓 = 9
2
-4
-3
-2
-1
0
1
2
3
4
95% Distribution comparision
Z-distribution, ±𝟏. 𝟗𝟔
n
df
Interval
10
9
±2.262
30
29
±2.045
75
74
±1.993
100
99
±1.984
Sample standard deviation
• When do we do not know the population standard deviation, we use the
stample standard deviation to approximate it,
• This approximation come at a cost though in terms of our interval estimate,
• We must use the t-distribution instead of z-distribution to account for this
estimation of 𝜎,
• Every sample size will have its own t-distribution with degees of freedom
df = n-1,
• Our standard error will now be:
𝒔
𝒔𝒙 = where 𝒔 =
𝒏
𝒙𝒊 −𝝁 𝟐
𝑵
𝒊=𝟏 𝑵−𝟏
Different standard errors
• In the previous examples we knew the population standard deviation 𝜎 and it
was therefore fixed in the standard error formula,
• This meant that all samples of the same size had the same standard error,
• When 𝜎 is uknown we estimate it with the sample standard deviation, s,
• Since every sample have a unique s, samples of the same size do not necessarily
have the same standard error,
• The randomness of sample selection is represented in its standard deviation and
therefore its standard error.
The Student’s T-distribution 95% Prob. Int.
𝑠𝑥 is largely dependent on sample size.
The standard deviation of
sampling distribution now is the
standard error of the mean.
If sample size is small 𝑠𝑥 becomes larger
and thus distribution becomes wider.
𝒔
𝒔𝒙 =
𝒏
𝒙
95% of all
sample means
(𝑥) are in here.
-4
-3
-2
-1
0
1
2
3
4
T-test for a single mean
𝑥 − 𝜇 0 𝑥 − 𝜇0
𝑡= 𝑠 =
𝑠𝑥
𝑛
𝑥 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑚𝑒𝑎𝑛
𝜇0 − ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑧𝑒𝑑 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑚𝑒𝑎𝑛
𝑠 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛
𝑛 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒
Question:
Is this t-test value in the nonrejection
region or the rejection region based on
df = n-1?
Two-tailed t-test rejection region, n = 20
With 𝛼 and 𝜎 not known and 20 samples (df = 20) we
would consult the t-table and find the corresponding
t-scores for a two-tailed test.
𝐻0 : 𝜇 = 𝜇0
𝐻𝑎 : 𝜇 ≠ 𝜇0
𝛼 = 0.05
t = +2.093
Critical Value
𝒙
t = -2.093
Critical Value
Nonrejection region
Rejection region
𝛼
= 0.025
2
-3
Rejection region
𝛼
= 0.025
2
𝝁𝟎
-2
-1
0
1
2
3
General t-distribution properties
1. A smaller sample size means more sampling error.
2. This sampling error due to small n means a higher probability of
extreme sample means.
3. More probability in the tails means the center hump of the tdistribution mu come downward.
4. This process shrinkes the distribution downward and outward and
thus moving critical values.
5. Given the same 𝛼 and s, a smaller n will push the critical values
outward in the tails due the uncertainty associated with small n.
Hypothesis testing procedure
1.
2.
3.
4.
5.
6.
7.
8.
9.
Start with clear research problem.
Establish hypothesis, null and alternative.
Determine appropriate statistical test and sampling distribution.
Choose 𝛼.
State decision the decision rule.
Gather sample data.
Calculate test statistics.
State statistical conclusion.
Make a decision.
Bussiness analyst salaries
A report from 6 years ago indicated that the average gross salary for a
bussiness analyst was $69,873. Since this survey is now outdated, the
Berau of Labor Statistics wished to test this figure against current
salaries to see if the current salaries are statistically different from the
old ones.
Based on this sample, we found s = $14,985. We do not know 𝜎 and
therefore we will estimate it using s.
For this study, the BLS will take a sample of 12 current salaries.
Bussiness analyst salaries
1. Establish Hypothesis
𝐻0 : 𝜇 = $69,873
𝐻𝑎 : 𝜇 ≠ $69,873
2. Determine Appropriate Statistical Test and Sampling Distribution
This will be two-tailed test.
𝒙 − 𝝁𝟎
Salaries can higher or lower.
𝒕= 𝒔
Since 𝜎 is uknown ans n is small
𝒏
we will use t-distribution.
Bussiness analyst salaries
3. Specify the error rate (significance level) 𝛼 = 0.05
4. State the decision rule
If t > 2.201, reject 𝐻0
𝒙
For df = 11
Rejection region
𝛼
= 0.025
2
-3
Rejection region
𝛼
= 0.025
2
Nonrejection region
If t < -2.201, reject 𝐻0
𝝁𝟎
-2
-1
0
1
2
3
Bussiness analyst salaries
5. Gather data n = 12, 𝑥 = $79,180
6. Calculate test statistics
𝑥 = $79,180
𝜇0 = $69,873
𝑠 = $14,985
𝑛 = 12
𝒙 − 𝝁𝟎
𝒕= 𝒔
𝒏
𝑡=
$79,180 − $69,873
= 2.15
$14,985
12
Bussiness analyst salaries
7 and 8. State statistical conclusion
𝐻0 : 𝜇 = $69,873 OK! 𝐻𝑎 : 𝜇 ≠ $69,873
Since the t-statistics is in the nonrejection region we fail to reject
Null hypothesis. It is not „out of the ordinary” that this sample
came from a population 𝜇 = $69,873 when df = 11
𝒙
Rejection region
𝛼
= 0.025
2
Nonrejection region
Rejection region
𝛼
= 0.025
2
-3
𝝁𝟎
-2
-1
0
1
2
3
Bussiness analyst salaries, n = 15
3. Specify the error rate (significance level) 𝛼 = 0.05
4. State the decision rule
If t > 2.145, reject 𝐻0
𝒙
For df = 14
Nonrejction region shrinks!
Rejection region
𝛼
= 0.025
2
-3
Rejection region
𝛼
= 0.025
2
Nonrejection region
If t < -2.145, reject 𝐻0
𝝁𝟎
-2
-1
0
1
2
3
Bussiness analyst salaries
5. Gather data n = 12, 𝑥 = $79,180
6. Calculate test statistics
𝑥 = $79,180
𝜇0 = $69,873
𝑠 = $14,985
𝑛 = 15
𝒙 − 𝝁𝟎
𝒕= 𝒔
𝒏
𝑡=
$79,180 − $69,873
= 2.41
$14,985
12
Bussiness analyst salaries
7 and 8. State statistical conclusion
𝐻0 : 𝜇 = $69,873
𝐻𝑎 : 𝜇 ≠ $69,873 𝑂𝐾!
1. The larger n decreased standard deviation of samplig
distribution thus narrowing it and making 𝑥 stand further out
on its own; more likely to belong to a different population
that does not overlap much with 𝜇0 . Created separation
between 𝑥 and 𝜇0 .
2. The larger n led to higher df. That shrinked nonrejection
region.
Rejection region
𝛼
= 0.025
2
-3
𝒙
Rejection region
𝛼
= 0.025
2
Nonrejection region
𝝁𝟎
-2
-1
0
1
2
3
Starbucks customer satisfaction
Starbucks is interestes in assessing customer satisfaction in the
Toronto. To conduct the study, Starbucks askes 25 customers in the city:
„Compared to other coffe houses in Toronto, would you say the
customer service at Starbucks is much better than average (5), better
then average (4), average (3), worse than average (2), much worse than
average (1)?” („Likert scale”)
The man rating was determined to be 3.5. Based on this sample, the
standard deviation was found to be s = 1.4.
Bussiness analyst salaries
1. Establish Hypothesis
𝐻0 : 𝜇 ≤ 3
𝐻𝑎 : 𝜇 > 3
2. Determine Appropriate Statistical Test and Sampling Distribution
This will be one-tailed test.
𝒙 − 𝝁𝟎
We are interested in better than average rating. 𝒕 =
𝒔
Since 𝜎 is uknown ans n is small
𝒏
we will use t-distribution.
Bussiness analyst salaries, n = 25
3. Specify the error rate (significance level) 𝛼 = 0.1
4. State the decision rule
For df = 24 If t > 2.495, reject 𝐻0
𝒙
Rejection region
Nonrejection region
𝛼 = 0.1
Nonrejction region shrinks!
𝝁𝟎
-3
-2
-1
0
1
2
3
Bussiness analyst salaries
5. Gather data n = 25, 𝑥 = 3.5
6. Calculate test statistics
𝑥 = 3.5
𝜇0 = 3
𝑠 = 1.4
𝑛 = 25
𝒙 − 𝝁𝟎
𝒕= 𝒔
𝒏
𝑡=
3.5 − 3
= 1.79
1.4
25
Bussiness analyst salaries
7 and 8. State statistical conclusion
𝐻0 : 𝜇 ≤ 3OK! 𝐻𝑎 : 𝜇 > 3
We fail to reject null hypothesis that customer satisfaction is
below average.
𝒙 = 𝟑. 𝟓
Rejection region
Nonrejection region
𝛼 = 0.1
t=1.79
𝝁𝟎 = 𝟑
-3
-2
-1
0
1
2
3
The p-value method
Based on our 𝛼 = 0.01 we know that 1% of our
area (probability) is in the upper tail past our
𝑡𝑐𝑟𝑖𝑡 = 2.495.
In the p-value method, we ask how much area (probability) is
above out test statistics of t=1.79.
Using t-table or Excel (T.DIST.RT(1.79,24)) we find that this 0.043
being greater than 0.01.
𝒙 = 𝟑. 𝟓
Since these are greater than 𝛼 = 0.01 we would fail to reject 𝐻0 .
Nonrejection region
Rejection region
𝛼 = 0.1
t=1.79
𝝁𝟎 = 𝟑
-3
-2
-1
0
1
2
3
We want to determine whether our sample mean (330.6) indicates that
this year's average energy cost is significantly different from last year’s
average energy cost of $260
We want to determine whether our sample mean (330.6) indicates that
this year's average energy cost is significantly different from last year’s
average energy cost of $260
ANOVA
Is one mean so far away from the
other two that is not from the same
population?
Suppose we want to compare three
sample means to see if there is a
difference between them
𝑥1
Question:
Do all three of these means come
from a common population?
𝑥2
𝑥3
Is one mean so far away from the
other two that is not from the same
population?
Suppose we want to compare three
sample means to see if there is a
difference between them
𝑥1
𝑥2
𝑥3
Is one mean so far away from the
other two that is not from the same
population?
Means are in different locations to
the overall mean.
𝑥1
𝑥2
𝑥3
We are not asking if they are exactly
equal. We are asking if each mean
likely came from the larger overall
population
Null hypothesis:
𝐻0 : 𝜇1 = 𝜇2 = 𝜇3
𝑥1
Variability AMONG/BETWEEN
the sample means.
𝑥2
𝑥3
Multiple t-test
𝑥1
𝑥2
𝑥3
𝐻0 : 𝑥1 = 𝑥2 ; 𝛼 = 0.05
𝐻0 : 𝑥1 = 𝑥3 ; 𝛼 = 0.05
𝑥1
𝑥2
Pairwise comparision
means three t-tests all
with 𝛼 = 0.05 Type I error
rate at 95% confidence.
Error compound with each t-test:
0.95 0.95 0.95 = 0.857
𝑥3
𝐻0 : 𝑥2 = 𝑥3 ; 𝛼 = 0.05
𝛼 = 1 − 0.857 = 0.143
ANOVA: Analysis of Variance
𝑥1
Variability ratio
Variability
AMONG/BETWEEN
the sample means.
𝑥2
Distance from
overall mean
𝑥3
overall mean
𝑥1
Internal spread
Variability
AMONG/WITHIN
the sample means.
𝑥2
𝑥3
ANOVA: Analysis of Variance
𝑥1
𝑥2
Distance from
overall mean
𝑥3
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐵𝑒𝑡𝑤𝑒𝑒𝑛
=
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑊𝑖𝑡ℎ𝑖𝑛
overall mean
𝑥1
Internal spread
𝑥2
𝑥3
ANOVA: Analysis of Variance
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐵𝑒𝑡𝑤𝑒𝑒𝑛
−→ 𝑇𝑜𝑡𝑎𝑙 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐶𝑜𝑚𝑝𝑜𝑛𝑒𝑛𝑡𝑠
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑊𝑖𝑡ℎ𝑖𝑛
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐵𝑒𝑡𝑤𝑒𝑒𝑛 + 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑊𝑖𝑡ℎ𝑖𝑛 = 𝑇𝑜𝑡𝑎𝑙 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒
Partitioning – separating total variance into its components parts
If the variability BETWEEN the means (distance from the overall mean) in the numerator is relatively large compared to
the variance WITHIN the samples (internal spread) in the denominator, the ration will be much larger than 1. The samples
then most likely do not come from a common population  reject null hypothesis that means are equal.
ANOVA: Analysis of Variance
𝐿𝐴𝑅𝐺𝐸
= 𝑅𝑒𝑗𝑒𝑐𝑡 𝐻0
𝑠𝑚𝑎𝑙𝑙
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐵𝑒𝑡𝑤𝑒𝑒𝑛
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑊𝑖𝑡ℎ𝑖𝑛
𝑠𝑖𝑚𝑖𝑙𝑖𝑎𝑟
= 𝐹𝑎𝑖𝑙 𝑡𝑜 𝑅𝑒𝑗𝑒𝑐𝑡 𝐻0
𝑠𝑖𝑚𝑖𝑙𝑖𝑎𝑟
𝑠𝑚𝑎𝑙𝑙
= 𝐹𝑎𝑖𝑙 𝑡𝑜 𝑅𝑒𝑗𝑒𝑐𝑡 𝐻0
𝐿𝐴𝑅𝐺𝐸
At least one mean is an
outlier and each
distribution is narrow;
distinct from each
other.
Means are close to
overall mean and/or
distr. overlap a bit;
hard to distinguish.
The means are close to
overall mean and/or
distr. melt together.
ANOVA: Analysis of Variance
𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝐵𝑒𝑡𝑤𝑒𝑒𝑛 + 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒 𝑊𝑖𝑡ℎ𝑖𝑛 = 𝑇𝑜𝑡𝑎𝑙 𝑉𝑎𝑟𝑖𝑎𝑛𝑐𝑒
𝐹=
𝐵𝑒𝑡𝑤𝑒𝑒𝑛
𝑊𝑖𝑡ℎ𝑖𝑛
F-ratio!
𝑥1
𝑥2
𝑥3
ANOVA: Analysis of Variance Example
𝑈𝑛𝑖𝑣𝑒𝑟𝑠𝑖𝑡𝑦 𝑠𝑡𝑢𝑑𝑦 𝑠𝑘𝑖𝑙𝑙𝑠
Twenty one students at the Autonomous University of Madrid (AUM) in Spain were selected for an
informal study about student study skills; 7 first year, 7 second year, and 7 third year undergraduates were
randomly selected.
The students were given a study-skills assessment having a maximum score of 100. As researchers we are
interested in whether or not a difference exists somewhere between the three different year levels. We
will conduct this analysis using a One-Way ANOVA.
ANOVA: Analysis of Variance Example
𝑈𝑛𝑖𝑣𝑒𝑟𝑠𝑖𝑡𝑦 𝑠𝑡𝑢𝑑𝑦 𝑠𝑘𝑖𝑙𝑙𝑠
Random
sample
within
each
group
Columns/Groups
Year 1 Scores
Year 2 Scores
Year 3 Scores
82
71
64
93
62
73
61
85
87
74
94
91
69
78
56
70
66
78
53
71
87
ANOVA: Analysis of Variance Example
Random
sample
within
each
group
𝑥1
𝑥2
𝑥3
Year 1 Scores
Year 2 Scores
Year 3 Scores
82
71
64
93
62
73
61
85
87
74
94
91
69
78
56
70
66
78
53
71
87
𝑥1 =?
𝑥2 =?
𝑥3 =?
Overall Mean:
The mean of all
21 scores taken
together.
𝑥 =?
ANOVA: Analysis of Variance Example
Random
sample
within
each
group
𝑥1
𝑥2
𝑥3
Year 1 Scores
Year 2 Scores
Year 3 Scores
82
71
64
93
62
73
61
85
87
74
94
91
69
78
56
70
66
78
53
71
87
𝑥1 = 71.71
𝑥2 = 75.29
𝑥3 = 76.57
Overall Mean:
The mean of all
21 scores taken
together.
𝑥 = 74.52
Variance and the sum of squares
Sample Variance
2
𝑠 =
𝑥−𝜇
𝑛−1
Avarages squared differences
between sample and its
mean.
Sum of Squares
2
𝑆𝑆 =
𝑥−𝜇
2
Partitioning sum of squares
SST
(total) sum of squares
SSC
(column/groups) sum of squares
SSE
(within/error) sum of squares
ANOVA: Analysis of Variance Example
Random
sample
within
each
group
𝑥1
𝑥2
𝑥3
Year 1 Scores
Year 2 Scores
Year 3 Scores
82
71
64
93
62
73
61
85
87
74
94
91
69
78
56
70
66
78
53
71
87
𝑥1 = 71.71
𝑥2 = 75.29
𝑥3 = 76.57
SST
(total) sum of
squares
Overall Mean:
The mean of all
21 scores taken
together.
𝑥 = 74.52
ANOVA: Analysis of Variance
𝑥 = 74.52
71
SST
(total) sum of
squares
78
87
66
78
69
𝑁
88
62
𝑆𝑆𝑇 =
74
93
56
91
70
𝑥𝑖𝑗 − 𝑥
𝑖=1 𝑗=1
73
53
𝐾
82
94
2
ANOVA: Analysis of Variance Example
Random
sample
within
each
group
𝑥1
𝑥2
𝑥3
Year 1 Scores
Year 2 Scores
Year 3 Scores
82
71
64
93
62
73
61
85
87
74
94
91
69
78
56
70
66
78
53
71
87
𝑥1 = 71.71
𝑥2 = 75.29
𝑥3 = 76.57
SSC
(column/groups)
sum of squares
Overall Mean:
The mean of all
21 scores taken
together.
𝑥 = 74.52
ANOVA: Analysis of Variance
𝑥 = 74.52
SSC
(column/groups)
sum of squares
71.77
𝐾
75.29
𝑆𝑆𝐶 =
𝑥𝑘 − 𝑥
𝑗=1
76.57
2
ANOVA: Analysis of Variance Example
Random
sample
within
each
group
𝑥1
𝑥2
𝑥3
Year 1 Scores
Year 2 Scores
Year 3 Scores
82
71
64
93
62
73
61
85
87
74
94
91
69
78
56
70
66
78
53
71
87
𝑥1 = 71.71
𝑥2 = 75.29
𝑥3 = 76.57
SSE
(within/error)
sum of squares
Overall Mean:
The mean of all
21 scores taken
together.
𝑥 = 74.52
ANOVA: Analysis of Variance Example
Random
sample
within
each
group
𝑥1
𝑥2
𝑥3
Year 1 Scores
Year 2 Scores
Year 3 Scores
82
71
64
93
62
73
61
85
87
74
94
91
69
78
56
70
66
78
53
71
87
𝑥1 = 71.71
𝑥2 = 75.29
𝑥3 = 76.57
𝑁
𝐾
𝑆𝑆𝐸 =
𝑥𝑖𝑗 − 𝑥𝑗
𝑖=1 𝑗=1
2
SSE
(within/error)
sum of squares
Overall Mean:
The mean of all
21 scores taken
together.
𝑥 = 74.52
Formulas for ANOVA
SSC
Sum of squares
𝑑𝑓𝑐𝑜𝑙𝑢𝑚𝑛𝑠 = 𝐶 − 1
𝑀𝑆𝐶 =
𝑆𝑆𝐶
𝑑𝑓𝑐𝑜𝑙𝑢𝑚𝑛𝑠
(columns)
SSE
Sum of squares
𝑑𝑓𝑒𝑟𝑟𝑜𝑟 = 𝑁 − 𝐶
𝑀𝑆𝐸 =
𝑑𝑓𝑡𝑜𝑡𝑎𝑙 = 𝑁 − 1
𝐹=
𝑆𝑆𝐸
𝑑𝑓𝑒𝑟𝑟𝑜𝑟
(within/error)
SST
Sum of squares
(total)
N = total observations C = no. of columns
𝑀𝑆𝐶
𝑀𝑆𝐸
Formulas for ANOVA – our case
SSC
Sum of squares
(columns)
𝑑𝑓𝑐𝑜𝑙𝑢𝑚𝑛𝑠 = 2 𝑀𝑆𝐶 =
88.67
2
SSE
Sum of squares
(within/error)
𝑑𝑓𝑒𝑟𝑟𝑜𝑟 = 18
2812.57
18
SST
Sum of squares
(total)
N = 21 C = 3
𝑑𝑓𝑡𝑜𝑡𝑎𝑙 = 20
𝑀𝑆𝐸 =
𝐹=
44.33
156.25
= 44.33
= 156.25
= 0.28
ANOVA chart
Source of Variance
df
SSE
MSE
F
Between (columns)
2
88.87
44.33
0.28
Within (error)
18
2812.57
156.25
Total
20
2901.24
𝐹=
𝑀𝑆𝐶
𝑀𝑆𝐸
𝐹𝛼,𝑑𝑓𝐶,𝑑𝑓𝐸
𝐹=
F-stat larger than F-crit?
𝑑𝑓=2
𝑑𝑓=18
𝐹0.05,2,18 = 𝐹. 𝐼𝑁𝑉. 𝑅𝑇 0.05,2,18 𝑖𝑛 𝐸𝑥𝑐𝑒𝑙 → 𝐹𝑐𝑟𝑖𝑡 = 3.55
NO. Fail to reject 𝐻0 . No significant
difference in mean test score by
Year of Student.
Statistical quality control
Sample 1
Sample 2
Sample 3
Sample 4
𝑥 = 3210.73
𝑥 = 3150.13
𝑥 = 3345.54
𝑥 = 3190.67
Sample 5
Sample 6
Sample 7
Sample 8
Sample 9
𝑥 = 3217.90
𝑥 = 3301.45
𝑥 = 3100.72
𝑥 = 3413.01
𝑥 = 3023.59
Standard deviation of 𝑥 sampling distribution
𝜎
𝜎𝑥 =
𝑛
4.5
4
3.5
3
𝜎𝑥 - standard deviation of 𝑥
2.5
2
𝜎 - standard deviation of population
1.5
1
n – sample size
0.5
0
2950-3049
3050-3149
3150-3249
In this case we know standard deviation of the population which is rarely the case.
3250-3349
3350-3449
Apple corporation 2012 Daily Returns
1. What is the probability, for any
given day, of a return greater
than 0.5%?
2. What is the probability, for any
given day, of a loss greater than
2%?
3. What is the probability, for any
given day, of a return beween 0
and 1%?
4. What is the probability, for any
given day, of a gain or loss
greater than 3%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
0.5 %
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Apple corporation 2012 Daily Returns
3. What is the probability, for any
given day, of a return beween 0 and
1%?
Z distribution
𝜇 = 0.11 %
𝜎 = 1.84 %
𝑥−𝜇
𝑧=
𝜎
1 − 0.11
𝑧=
= 0.48
1.84
𝑥−𝜇
𝑧=
𝜎
0 − 0.11
𝑧=
= −0.06
1.84
= norm.dist(0.48,0,1,TRUE)=0.69
= norm.dist(-0.06,0,1,TRUE)=0.48
0.21
0.21
-3
-2
-5.41% -3.57%
-1
-1.73%
0
0.11%
1
1.95%
2
3.79%
3
5.63%
Z-test for a single mean
𝑥 − 𝜇 0 𝑥 − 𝜇0
𝑧=
=
𝜎
𝜎
𝑥 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑚𝑒𝑎𝑛
𝜇0 − ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑧𝑒𝑑 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑚𝑒𝑎𝑛
𝜎 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛
𝑛 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒
The Student’s T-distribution
1. In general, the t-distribution is shorter
in the middle and fatter in the tails.
2. More probability in the tails, less near
the mean, grater chance of extreme
values.
3. There isn’t just one t-distribution.
4. There is a t-distribution for every
sample size.
5. Degrees of Freedom (n-1).
6. Smaller the sample size, the shorter
and fatter the distribution, more tail
probability.
7. However as n becomes large, the tdistribution tends to z-distribution.
Area (probability) under the curve = 1
When we do not know 𝜎 we have to
estimate. This estimation, this
uncertainty, forces us to use the
Student’s T-dsitribution.
-4
-3
-2
-1
0
1
2
3
4
T-test for a single mean
𝑥 − 𝜇 0 𝑥 − 𝜇0
𝑡= 𝑠 =
𝑠𝑥
𝑛
𝑥 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑚𝑒𝑎𝑛
𝜇0 − ℎ𝑦𝑝𝑜𝑡ℎ𝑒𝑠𝑖𝑧𝑒𝑑 𝑝𝑜𝑝𝑢𝑙𝑎𝑡𝑖𝑜𝑛 𝑚𝑒𝑎𝑛
𝑠 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑑𝑒𝑣𝑖𝑎𝑡𝑖𝑜𝑛
𝑛 − 𝑠𝑎𝑚𝑝𝑙𝑒 𝑠𝑖𝑧𝑒
Question:
Is this t-test value in the nonrejection
region or the rejection region based on
df = n-1?
95% Distribution comparision
Z-distribution, ±𝟏. 𝟗𝟔
n
df
Interval
10
9
±2.262
30
29
±2.045
75
74
±1.993
100
99
±1.984
How stastistically can we predict data?
𝒙 − 𝑑𝑎𝑡𝑎
1. Collect sample
How stastistically can we predict data?
𝒙 − 𝑑𝑎𝑡𝑎
𝒙
1. Collect sample
2. Calculate statistical
parameters
How stastistically can we predict data?
𝒙 − 𝑑𝑎𝑡𝑎
𝝈
𝒙
1. Collect sample
2. Calculate statistical
parameters
How stastistically can we predict data?
𝝈
𝒙
2. Calculate statistical
parameters
How stastistically can we predict data?
𝝈
𝒙
2. Calculate statistical
parameters
3. Fit distribution
How stastistically can we predict data?
In what range
with specific
certainty do
we expect
future results?
4. Predict future results 
confidence interval
Confidence intervals
Gumball guessing game
The oject of the game is to guess how many gumballs are in the jar.
• If you guess within 5 (±5) I will give you the gum plus 50 PLN,
• If you guess within 15 (±15) I will give you the gum plus 15 PLN,
• If you guess within 30 (±30) I will give you the gum.
But before you start guess you have to decide How confident are
you?
Confidence interval
• Note that in the preceding examples we used the terms „confident” and
„interval” in the form of ±,
• When estimating a population parameter using a sample statistics it is never
going to be perfect; there will always be an error,
• To estimate a population parameter without an error we would have to include
all the sample in the population  no practical sense,
• We can express that error, or uncertainty, using an interval estimate:
𝑝𝑜𝑖𝑛𝑡 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒 ± 𝑀𝑎𝑟𝑔𝑖𝑛 𝑜𝑓 𝑒𝑟𝑟𝑜𝑟
C.I. and standard error
• In the previous exmaples we discussed the standard error of the mean,
• To find the standard error of the mean (SEM) we need to know two things: 1)
the population standard deviation and 2) the sample size,
• Most often we do not know the population standard deviation (PSD) and
therefore we have to estimate it,
• Also remember that for any PSD, increasing the sample size reduces standard
error,
• So we are left with idea that the confidence interval will be affected by all these
points: standard deviation, sample size and level of „confidence” we are satisfied
with.
The Standard Normal Curve
SNC is also a sampling distribution of the mean 𝑥.
The distribution of many sample means of give sample
size.
𝒙
-3
-2
-1
0
1
2
3
95% Probability Interval
The are in the tails is
called alpha.
Therefore we have 5%
left evenly divided
between both tails.
𝛼 = 5% 𝑜𝑟 0.05
𝒙
95% of all
sample means
(𝑥) are in here.
5%
= 2.5% 𝑜𝑟 0.025
2
5%
= 2.5% 𝑜𝑟 0.025
2
𝜶
𝟐
𝜶
𝟐
-3
-2
-1
0
1
2
3
95% Probability Interval
By doing so, we can
assign z-scores (t-scores)
to the uppar and lower
boundary of the 95%
interval.
If we the population
standard deviation 𝜎 we
treat the sampling
distribution as a standard
normal curve (z-curve).
If not we have to
estimate 𝜎 and use t
distribution (t-curve).
𝒙
95% of all
sample means
(𝑥) are in here.
−𝟏. 𝟗𝟔𝝈
𝜶
𝟐
-3
-2
+𝟏. 𝟗𝟔𝝈
𝒙 ± 𝟏. 𝟗𝟔𝝈𝒙
-1
0
1
𝜶
𝟐
2
3
95% Probability Interval
The standard error of the
mean 𝜎𝑥 depends on:
𝜎
• Sample size (𝜎𝑥 = ),
What is the standard
deviation of the sampling
distribution?
𝑛
The standard error of the
mean 𝜎𝑥 (not 𝜎 – the
standard deviation of the
population)
𝒙
95% of all
sample means
(𝑥) are in here.
−𝟏. 𝟗𝟔𝝈𝒙
𝜶
𝟐
-3
-2
+𝟏. 𝟗𝟔𝝈𝒙
𝒙 ± 𝟏. 𝟗𝟔𝝈𝒙
-1
0
1
𝜶
𝟐
2
3
95% Probability Interval
As soon as a sample mean steps outside
the dotted region, 𝜇 is no longer in its
interval.
95% of all sample means (𝑥) are in here.
𝝁
-3 -2 -1 0
𝒙𝟏
1 2 3
𝒙𝟐
We take many
samples of the
same size.
𝒙𝟑
𝒙𝟒
𝒙𝟓
𝒙𝟔
Does this sample interval contains 𝜇?
Sample of the
same size have
the same standard
error 𝜎𝑥 . So the
95% „width” is the
same for all
sample of that
size.
Interpretation
• The randomness lies in the elements chosen for the sample, NOT the population
mean.
• It is the probability of obtaining a representative sample.
• The proportion of samples, size n, for which our estimate, the sample mean 𝑥, is
within a certain distance ± of the true population mean, 𝜇.
• The sample mean is either wihin ± interval of the true mean, or it is not (no
probability).
• The confidence interval IS NOT the probability that the population mean lies
within the interval.
Example
To estimate the mean amount spent per customer at a TGI Friday’s data was
collected for 75 customers. We are to assume the population standard deviation is
$4.
1. At 95% confidence, what is the margin of error?
2. If the sample mean is $20, what is the 95% confidence interval for the
population mean (all customers)?
Example
1. At 95% confidence, what is the margin of error?
𝒙 ± 𝟏. 𝟗𝟔𝝈𝒙
𝑛 = 75
𝜎=4
𝜎𝑥 =
𝜎
𝑛
𝜎𝑥 =
4
75
𝑥 ± 1.96 ∗ 0.46
𝑥 ± 0.91
= 0.46
Example
2. If the sample mean is $20, what is the 95% confidence interval for the population
mean (all customers)?
𝑥 ± 0.91
20 ± 0.91
95% Probability Interval
All samples of n = 75 will have 0.91 as
the margin of error, assuming 𝜎 is know
to be 4.
95% of all intervals made using 𝑥 ±
0.91 will contain the uknown
POPULATION MEAN.
If we take another samples of n = 75 and
make intervals 𝑥 ± 0.91, 95 of them will
contain 𝜇.
𝒙
95% of all
sample means
(𝑥) are in here.
𝟎. 𝟎𝟐𝟓
-3
-2
𝟏𝟗. 𝟎𝟗
20 ± 0.91
-1
0
𝟐𝟎
1
95% confident
𝟎. 𝟎𝟐𝟓
2
𝟐𝟎. 𝟗𝟏
3
Example
Brick and mortar stores like Castorama etc. Have a difficult time dealing with the
practice of „show-rooming” whereby customers come in to the store, examine
items, leave and then buy them cheaper on Allegro.
We call these „false customers”. Sales associates spend time with the customers
who have no intention of buying anything at the stores leading to unproductive
time spent.
Management in one company determined that if less than 15% of a salesperon’s 8
hour day (4320 seconds) is spent with „false customers” then show-rooming is a
not a major problem.
Example
To determine if current show-rooming is really a problem bases on the company’s
standards, 125 sales associates are randomly selected to measure their service
times with false customers.
For this problem we are going to assume a population standard deviation, 𝜎 =
1958 seconds. The results are:
Standard error of the mean
n = 125
𝑥 = 3661.5
𝜎 = 1958
𝜎𝑥 =
1958
125
= 175.13
95% Confidence Interval
𝑥 ± 1.96𝜎𝑥
3661.5 ± 1.96 175.13
3661.5 ± 343.25
95% Probability Interval
95% Confidence Interval
𝑥 ± 1.96𝜎𝑥
3661.5 ± 1.96 175.13
3661.5 ± 343.25
We are 95% confident that salesperson
spend between 3318.25 s and 4004.75 s
interacting with false customer each 8 h
day.
It is 5% possible that we will select
sample with very high 4320 sample
mean.
False customers are
not a great enough
drain to justify policy
changes.
𝒙
95% of all
sample means
(𝑥) are in here.
𝟎. 𝟎𝟐𝟓
-3
3661.5 ± 343.25
-2
𝟑𝟑𝟏𝟖. 𝟐𝟓
-1
0
𝟑𝟔𝟔𝟏. 𝟓
1
𝟎. 𝟎𝟐𝟓
2
𝟒𝟎𝟎𝟒. 𝟕𝟓
3
Concern point
4320 seconds
Summing up
• The randomness lies in the elements chosen for the sample, NOT the population
mean.
• It is the probability of obtaining a representative sample.
• The proportion of samples, size n, for which our estimate, the sample mean 𝑥, is
within a certain distance ± of the true population mean, 𝜇.
• The sample mean is either wihin ± internval of the true mean, or it not not (no
probability).
• The confidence interval IS NOT the probability that the population mean lies
within the interval.
The Student’s T-distribution
1. In general, the t-distribution is shorter
in the middle and fatter in the tails.
2. More probability in the tails, less near
the mean, grater chance of extreme
values.
3. There isn’t just one t-distribution.
4. There is a t-distribution for every
sample size.
5. Degrees of Freedom (n-1).
6. Smaller the sample size, the shorter
and fatter the distribution, more tail
probability.
7. However as n becomes large, the tdistribution tends to z-distribution.
Area (probability) under the curve = 1
When we do not know 𝜎 we have to
estimate. This estimation, this
uncertainty, forces us to use the
Student’s T-dsitribution.
-4
-3
-2
-1
0
1
2
3
4
The Student’s T-distribution 95% Prob. Int.
By doing so, we can assign t-scores to the
upper and lower boundary of the 95%
interval of each sample size.
Area (probability) under the curve = 1
If we do not know the population
standard deviation 𝜎 we treat the
sampling distribution as a t-distribution.
Degrees of Freedom (n-1)
n = 10
𝒙
z = -1.96
z = +1.96
95% of all
sample means
(𝑥) are in here.
t = -2.26
df = 9
t = +2.26
𝑡𝛼 𝑑𝑓 = 9
2
-4
-3
-2
-1
0
1
2
3
4
95% Distribution comparision
Z-distribution, ±𝟏. 𝟗𝟔
n
df
Interval
10
9
±2.262
30
29
±2.045
75
74
±1.993
100
99
±1.984
Sample standard deviation
• When do we do not know the population standard deviation, we use the
stample standard deviation to approximate it,
• This approximation come at a cost though in terms of our interval estimate,
• We must use the t-distribution instead of z-distribution to account for this
estimation of 𝜎,
• Every sample size will have its own t-distribution with degees of freedom
df = n-1,
• Our standard error will now be:
𝒔
𝒔𝒙 =
𝒏
Different standard errors
• In the previous examples we knew the population standard deviation 𝜎 and it
was therefore fixed in the standard error formula,
• This meant that all samples of the same size had the same standard error,
• When 𝜎 is uknown we estimate it with the sample standard deviation, s,
• Since every sample have a unique s, samples of the same size do not necessarily
have the same standard error,
• The randomness of sample selection is represented in its standard deviation and
therefore its standard error.
The Student’s T-distribution 95% Prob. Int.
𝑠𝑥 is largely dependent on sample size.
The standard deviation of
sampling distribution now is the
standard error of the mean.
If sample size is small 𝑠𝑥 becomes larger
and thus distribution becomes wider.
𝒔
𝒔𝒙 =
𝒏
𝒙
95% of all
sample means
(𝑥) are in here.
-4
-3
-2
-1
0
1
2
3
4
Example
To estimate the mean amount spent per customer at a TGI Friday’s data was
collected for 15 customers. The sample standard deviation is $4.
1. At 95% confidence, what is the margin of error?
2. If the sample mean is $20, what is the 95% confidence interval for the
population mean (all customers)?
Example
1. At 95% confidence, what is the margin of error?
𝑛 = 15
𝑠=4
𝑑𝑓 = 14
𝒙 ± 𝟐. 𝟏𝟒𝟓𝒔𝒙
𝑠𝑥 =
𝑠
𝑛
𝑠𝑥 =
4
15
= 1.03
𝑥 ± 2.145 ∗ 1.03
𝑥 ± 2.21
Example
2. If the sample mean is $20, what is the 95% confidence interval for the population
mean (all customers)?
𝑥 ± 2.21
20 ± 2.21
95% Probability Interval
𝑛 = 15
𝑠=4
𝑑𝑓 = 14
𝛼 = 0.05
𝑡 = 2.145
𝑥 ± 2.21
𝑥1 ± 2.21
20 ± 2.21
We are not saying
that there is 95%
probability that
population mean is in
the interval.
We are saying that
there is 95%
probability the
interval contains
population mean. The
randomness is in the
interval not the mean.
𝒙
95% of all
sample means
(𝑥) are in here.
20 ± 2.21
-4
-3
-2
17.79
-1
0
20
1
2
3
22.21
4
Example
Brick and mortar stores like Castorama etc. Have a difficult time dealing with the
practice of „show-rooming” whereby customers come in to the store, examine
items, leave and then buy them cheaper on Allegro.
We call these „false customers”. Sales associates spend time with the customers
who have no intention of buying anything at the stores leading to unproductive
time spent.
Management in one company determined that if less than 15% of a salesperon’s 8
hour day (4320 seconds) is spent with „false customers” then show-rooming is a
not a major problem.
Example
To determine if current show-rooming is really a problem bases on the company’s
standards, 30 sales associates are randomly selected to measure their service times
with false customers.
For this problem we are going to use sample standard deviation, 𝑠 =
1958 seconds. The results are:
Standard error of the mean
n = 30
𝑥 = 3661.5
𝜎 = 1958
𝑠𝑥 =
1958
30
= 357.48
95% Confidence Interval
𝑥 ± 2.045𝑠𝑥
3661.5 ± 2.045 357.48
3661.5 ± 731.05
95% Probability Interval
95% Confidence Interval
𝑥 ± 2.045𝑠𝑥
3661.5 ± 2.045 357.48
3661.5 ± 731.05
𝑛 = 75
𝑠=4
𝑑𝑓 = 14
𝛼 = 0.05
𝑡 = 2.145
False customers are in
fact draining
resources. Policy
intervention needed.
𝒙
95% of all
sample means
(𝑥) are in here.
3661.5 ± 731.05
-4
-3
-2
2930.45
-1
1
0
3661.5
We are 95% confident that salesperson
spend between 2930.45 s and 4392.55 s
interacting with false customer each 8 h
day.
It is 5% possible that we will select
sample with very high 4320 sample
mean.
Concern point
4320 seconds
t = 1.84
2
3
4392.55
4
Increase sample size from 30 to 50
95% Confidence Interval
𝑥 ± 2.011𝑠𝑥
3661.5 ± 2.011 276.9
3661.5 ± 556.58
𝑛 = 75
𝑠=4
𝑑𝑓 = 49
𝛼 = 0.05
𝑡 = 2.011
Falase customers are
not a great enough
drain to justify policy
changes.
𝒙
95% of all
sample means
(𝑥) are in here.
3661.5 ± 556.58
-4
-3
-2
3104.92
-1
1
0
3661.5
We are 95% confident that salesperson
spend between 3104.92 s and 4218.08 s
interacting with false customer each 8 h
day.
It is 5% possible that we will select
sample with very high 4320 sample
mean.
Concern point
4320 seconds
t = 2.38
2
3
4218.08
4
Confidence interv al
• Note that in the preceding examples we used the terms „confident” and
„interval” in the form of ±,
• When estimating a population parameter using a sample statistics it is never
going to be perfect; there will always be an error,
• To estimate a population parameter without an error we would have to include
all the sample in the population  no practical sense,
• We can express that error, or uncertainty, using an interval estimate:
𝑝𝑜𝑖𝑛𝑡 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒 ± 𝑀𝑎𝑟𝑔𝑖𝑛 𝑜𝑓 𝑒𝑟𝑟𝑜𝑟
Estimating size of the sample
Margin of Error
𝑃𝑜𝑖𝑛𝑡 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒 ± 𝑀𝑎𝑟𝑔𝑖𝑛 𝑜𝑓 𝑒𝑟𝑟𝑜𝑟
𝜎
2 𝑛
𝑥 ± 𝑧𝛼
𝑧𝛼 − 𝑧 𝑏𝑜𝑢𝑛𝑑𝑎𝑟𝑦 𝑜𝑓 𝑖𝑛𝑡𝑒𝑟𝑣𝑎𝑙 𝑝𝑟𝑜𝑏𝑎𝑏𝑖𝑙𝑖𝑡𝑦
2
𝜎
2 𝑛
−→ 𝑧𝛼
𝐸=
𝑧𝛼
2
𝜎
𝑛
1. We choose E, our margin of error.
2. We choose our confidence probability
boundary 𝑧𝛼 .
2
𝜎
− 𝑠𝑡𝑎𝑛𝑑𝑎𝑟𝑑 𝑒𝑟𝑟𝑜𝑟
𝑛
3. We are given or we estimate population
standard deviation 𝜎.
4. Solve for n.
Margin of Error
𝑃𝑜𝑖𝑛𝑡 𝑒𝑠𝑡𝑖𝑚𝑎𝑡𝑒 ± 𝑀𝑎𝑟𝑔𝑖𝑛 𝑜𝑓 𝑒𝑟𝑟𝑜𝑟
𝜎
2 𝑛
𝐸 = 𝑧𝛼
𝑧𝛼 𝜎 2
𝑧𝛼 𝜎
𝑛=
2
𝐸
n=
2
𝐸
Margin of Error
• Solving for the sample requires the population standard deviation 𝜎. Most ofther
we do not know it so we have to use an estimare or „planning value” in its place.
Options:
1. Estimate 𝜎 from previous studies using the same population of interest.
2. Conduct a pilot study to select a preliminary sample. Use sample standard
deviation from the pilot study.
3. Use a judgment or best guess for 𝜎. A common guess is the data range
(high-low) divided by 4.
Example
How large a sample should be selected to provide a 95% confidence interval with a
marigin of error (E) of 8? Assiuming the population standard deviation is 𝜎 = 36.
𝑧𝛼 𝜎 2
n=
2
𝐸
=
1.96∗36 2
8
= 77.8
To have 95% of our sample means contain 𝜇, we need a sample size of 78.
Example
The question we are asking is:
„What minimum sample size is necessary to produce 95% confidence that the
sample mean is ±8 of the true population mean?”
As we increase sample size we reduce the standard error and our sample most
likely becomes more representative of the population.
In graphical terms, we set the upper and lower boundary. Then we increase sample
size, pulling the distribution in the middle and inward on the sides.
Example
The larger sample size ensures more sample means are within the given margin of
error due…
To the fact that a large sample is more representative of the overall population.
Larger sample size will be required when:
1. A smaller margin of error is required.
2. A higher level of confidence is required.
3. Or both.
95% Probability Interval
95% Confidence Interval
𝑥 ± 1.96𝜎𝑥
3661.5 ± 1.96 175.13
3661.5 ± 343.25
We are 95% confident that salesperson
spend between 3318.25 s and 4004.75 s
interacting with false customer each 8 h
day.
It is 5% possible that we will select
sample with very high 4320 sample
mean.
Falase customers are
not a great enough
drain to justify policy
changes.
𝒙
95% of all
sample means
(𝑥) are in here.
𝟎. 𝟎𝟐𝟓
-3
3661.5 ± 343.25
-2
𝟑𝟑𝟏𝟖. 𝟐𝟓
-1
0
𝟑𝟔𝟔𝟏. 𝟓
1
𝟎. 𝟎𝟐𝟓
2
𝟒𝟎𝟎𝟒. 𝟕𝟓
3
Concern point
4320 seconds