Download `Central Limit Theorem`.

Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
Statistics for Economist
Ch. 13 The Normal Approximation
for Probability Histograms
1. Tossing a Coin and the Normal Distribution
2. Two Kinds of Histograms
3. The Relationship of Two Histograms
4. Central Limit Theorem
5. The Normal Approximation
6. The Scope of the Normal Approximation
7. Conclusion
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
INDEX
STATISTICS
1
Tossing a Coin and the Normal Distribution
2
Two Kinds of Histograms
3
The Relationship of Two Histograms
4
Central Limit Theorem
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
2/31
1. Tossing a Coin and the Normal Distribution
STATISTICS
 When a coin is tossed a large number of times,
the percentage of heads will be closed to 50%.
Ex) 5 Tosses
# of heads
# of ways
0
1
1
5
2
10
3
10
4
5
5
1
If 100 tosses?
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
3/31
1. Tossing a Coin and the Normal Distribution
STATISTICS
 If we toss 100 times
1.Total # of ways
2100  1.27  1030
2. The # of ways with exactly 50 heads
100!
100  99      51

 1.01 1029
50 ! 50! 50  49      1
3. mathematical probability for the # of
patterns with exactly 50 heads
1.01 10 29

 0.08  8%
30
1.27  10
# of ways with 50 heads
total # of ways
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
4/31
1. Tossing a Coin and the Normal Distribution
STATISTICS
 To calculate in the previous way the probability of getting
exactly 50 heads from 100 tosses is inconvenient.
 Any alternatives? When you toss lots of coins, the
distribution of the number or ratio of getting heads
among all tosses is well approximated by the normal
curve.
 We will solve this problem later using the normal curve.
The answer is 7.96%. It is almost same with the previous
answer 8% acquired from the binomial distribution.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
5/31
INDEX
STATISTICS
1
Tossing a Coin and the Normal Distribution
2
Two Kinds of Histograms
3
The Relationship of Two Histograms
4
Central Limit Theorem
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
6/31
STATISTICS
2. Two Kinds of Histograms
Probability Histogram and Empirical Histogram
Probability Histograms
A kind of graph which represents probability, not data. It is
made up of rectangles and the base of each rectangle is
centered at a possible value for the sum of draws. The area of
the rectangles equals a probability of getting the value.
Empirical Histograms
A kind of histogram which represents experimentally acquired
probability from observed data. The areas of each rectangle
represents empirical density. To get density one should divide the
data into intervals, count the frequencies.
The Relationship of the two:
Empirical histograms converge to a probability of histogram
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
7/31
INDEX
STATISTICS
1
Tossing a Coin and the Normal Distribution
2
Two Kinds of Histograms
3
The Relationship of Two Histograms
4
Central Limit Theorem
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
8/31
3. The Relationship of Two Histograms
STATISTICS
Empirical Histogram : Rolling a pair of dice and finding the total number of spots
0.20
0.15
0.10
(a) The first 100 repetitions
0.05
0.00
1
2
3
4
5
6
7
8
9 10 11 12 13 14
0.20
0.15
0.10
(b) For 1,000 repetitions
0.05
0.00
1
2
3
4
5
6
7
8
9 10 11 12 13 14
0.20
0.15
0.10
(c) For 10,000 repetitions
0.05
0.00
1
2
3 4
5
6
7
8
9 10 11 12 13 14 15
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
9/31
STATISTICS
3. The Relationship of Two Histograms
Probability Histogram
 Probability Histogram of sum : the probability
histogram can be found through tossing limitless times
and getting the limit of empirical histograms
 In case of rolling a pair of dice we can calculate the ideal
probability easily.
Probability Histogram
0.20
0.15
0.10
0.05
0.00
1
2
3 4
5
6
7
8
9 10 11 12 13 14 15
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
10/31
INDEX
STATISTICS
1
Tossing a Coin and the Normal Distribution
2
Two Kinds of Histograms
3
The Relationship of Two Histograms
4
Central Limit Theorem
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
11/31
4. Central Limit Theorem
STATISTICS
Cental Limit Theorem
standard units
-3
-2
-1
0
1
10
2
3
50
5
25
Percent per
standard unit (%)
percent per sum (%)
Tossed 100 times
Expected number of heads : 50
SE of the # of the0 heads : 5
0
35
40
45
50
# of heads
55
Very similar to the normal curve.
Tossing a coin 100 times : probability histogram and the normal curve of getting heads
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
12/31
4. Central Limit Theorem
STATISTICS
Central Limit Theorem
9
8
7
6
5
4
3
2
1
0
45
40
35
30
25
20
15
10
5
0
Tossed 100 times
35
40
45
50
55
60
65
4.5
The histograms follow the normal
curve better and better as the
number of tosses goes up.
4
3.5
3
2.5
2
1.5
1
0.5
0
45
40
35
30
25
20
15
10
5
0
Tossed 400 times
170
180
190
200
210
220
230
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
13/31
4. Central Limit Theorem
STATISTICS
Central Limit Theorem
3
45
40
35
30
25
20
15
10
5
0
Tossed 900 times
2.5
2
1.5
1
0.5
0
405
420
435
450
465
480
Tossed
limitlessly?
495
The probability histogram will follow the normal curve if the number of
observations goes up. We call it ‘Central Limit Theorem’.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
14/31
INDEX
STATISTICS
5
The Normal Approximation
6
The Scope of the Normal Approximation
7
Conclusion
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
15/31
STATISTICS
5. The Normal Approximation
Example 1
 A coin will be tossed 100 times.
Estimate the probability of getting
a) exactly 50 heads
b) between 45 and 55 heads inclusive
c) between 45 and 55 heads exclusive
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
16/31
STATISTICS
3. The Relationship of Two Histograms
Empirical Histogram and Probability Histogram
The empirical histograms converge to the ideal
probability histogram. Converge means “Gets closer and
closer to”.
Empirical Histogram
Law of average
n→∞
Probability Histogram
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
17/31
STATISTICS
5. The Normal Approximation
Example 1
a) exactly 50 heads
☞ expected value is 50, SE is 5.
49.5  50
  0.01 ,
50
49.5
50
50.5
50.5  50
 0.01
50
The base of the rectangle
goes from 49.5 to 50.5 on
the number-of-heads scale.
Using the
Normal
Approximation
7.96%
The exact probability is 7.96%.
The approximation is excellent.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
18/31
STATISTICS
5. The Normal Approximation
Example 1
b) between 45 and 55 heads inclusive
c) between 45 and 55 heads exclusive
44.5
50
-1.1
0
55.5
45.5
1.1
-0.9
50
54.5
0 0.9
 63.18 %
 72.87 %
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
19/31
INDEX
STATISTICS
5
The Normal Approximation
6
The Scope of the Normal Approximation
7
Conclusion
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
20/31
6. The Scope of the Normal Approximation
STATISTICS
The Scope of the Normal Approximation
 With regards to drawing from a box, the normal approximation
works perfectly well, so long as you remember one thing. The
more the histogram of the numbers in the box differs from the
normal curve. The more draws are needed before the
approximation takes hold.
 However, the speed of approximation differs according to the
contents contained in the box. When we represent the
contents through a histogram, the more similar to the normal
curve the form is, the faster it follows the normal curve.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
21/31
6. The Scope of the Normal Approximation
STATISTICS
Probability Histogram of a box containg nine 0’s, and one 1
100
50
0
-2
-1
0
1
2
3
4
5
6
0 1
distribution of box
7
25 draws
1
4
7
10
13
16
19
100 draws
22
28
34
40
46
52
58
400 draws
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
22/31
6. The Scope of the Normal Approximation
STATISTICS
when the contents are symmetric
50 %
0
1
2
3
distribution of box
35
40
45
50
55
60
65
25 draws
80
85
90
95
100
105
110
115
120
50 draws
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
23/31
6. The Scope of the Normal Approximation
STATISTICS
when the contents are asymmetric
50 %
-2.25
-1.12
0.00
1.12
2.25
3.37
25회 추출
0
1
25
2 3 4 5 6 7
distribution of box
8
Standard unit
-3.51
50
0
40
50
60
70
80
90
100 110 120 130 140 150 160
sum
Percent per standard unit (%)
Percent per standard unit (%)
Standard unit
-3.37
50
-2.11
-0.70
0.70
2.11
3.51
100 draws
25
25 draws
100 draws
0
275
300
325
350
375
400
sum
425
450
475
500
525
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
24/31
6. The Scope of the Normal Approximation
STATISTICS
The histogram for a product
 Central Limit Theorem does not apply to a product.
☞ The probability histogram for a product will usually be quite
different from normal.
Making the number of rolls larger does not make the histogram
more normal. The histogram for a product does not convergo to the
normal curve.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
25/31
6. The Scope of the Normal Approximation
STATISTICS
Making the magnitude of a sample larger does not make the
probability histogram for a product more normal.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
26/31
INDEX
STATISTICS
5
The Normal Approximation
6
The Scope of the Normal Approximation
7
Conclusion
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
27/31
STATISTICS
7.Conclusion
Central Limit Theorem
 Central Limit Theorem
When drawing at random with replacement from a box the
probability histogram for the sum will follow the normal
curve.
 Even if the contents of the box do not.
 But when we approximate a probability histogram to the normal
curve, the number of draws needed to approximate changes.
 If the distribution of the box is similar to the normal curve, the
number of draws needed is small, but otherwise the number
should be larger.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
28/31
STATISTICS
7.Conclusion
Expexted value and SE
 When the probability histogram does follow the normal
curve, it can be summarized by the expexted value and SE.
The expected value pins the center of the
probability histogram to the horizontal axis,
and the SE fixes its spread.

•
•
•
Expexted value and SE for a sum can be computed from
average of box
SD of box
the number of draws.
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
29/31
7.Conclusion
STATISTICS
Convergence of Histogram (1)
Empirical
Histogram →
Probability
Histogram
Probability
Histogram →
Normal
Distribution
Explanation
In textbook
Sections 13.2-3
Sections 13,4-6
Examples
Figure 13-1
Figures 13-3,5,6,8
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
30/31
STATISTICS
7.Conclusion
Convergence
히스토그램의 수렴
of Histogram
(2)
(2)
Empirical
Histogram →
Probability
Histogram
Conditions
for
convergence
When the number
of repetition
is infinitely large
Probability
Histogram →
Normal
Distribution
When the
number
of draws
.
is infinitely large
.
Relevent rules
Law of average
Central Limit Theorem
(law of large numbers)
Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics Statistics & Econometrics
31/31