Download ECON1003: Analysis of Economic Data - Ka

Lesson 2:
Descriptive Statistics
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-1
2004 Regional Nominal GDP per Capita
Summary Statistics (yuan per person)
Mean
Standard Error
Median
Mode
Standard Deviation
Sample Variance
Kurtosis
Skewness
Range
Minimum
Maximum
Sum
Count
Ka-fu Wong © 2007
14079.39
1912.88
9608.00
#N/A
10650.44
113431828.11
7.14
2.50
51092.00
4215.00
55307.00
436461.00
31
ECON1003: Analysis of Economic Data
Guizhou
Shanghai
Lesson2-2
Outline
Measures of Central Tendency
Measures of Variability
Measures for two variables
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-3
Population Parameters and Sample
Statistics



A population parameter is number calculated from all
the population measurements that describes some
aspect of the population.
 The population mean, denoted , is a population
parameter and is the average of the population
measurements.
A point estimate is a one-number estimate of the value
of a population parameter.
A sample statistic is number calculated using sample
measurements that describes some aspect of the
sample.
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-4
Measures of Central Tendency
Overview
Central Tendency
Mean
Median
Mode
Midpoint of
ranked values
Most frequently
observed value
n
x
x
i1
i
n
Arithmetic
average
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-5
The Mean
Imagine an economy with N
individuals (that is, a population)
with income X1, X2, …, XN
Sample x1, x2, …, xn
Income per capita ()
Population Mean
N

Ka-fu Wong © 2007
X
x
Sample Mean
n
i
i =1
N
ECON1003: Analysis of Economic Data
x
x
i
i =1
n
Lesson2-6
Arithmetic Mean
 The arithmetic mean (mean) is the most common measure of
central tendency
 For a population of N values:
N
x
x1  x 2  ...  x N
μ

N
N
i 1
i
Population
values
Population size
 For a sample of size n:
n
x
x
i 1
n
i
x1  x 2  ...  x n

n
Observed
values
Sample size
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-7
Effect of extreme value on Arithmetic Mean
 The most common measure of central tendency
 Mean = sum of values divided by the number of values
 Affected by extreme values (outliers)
0 1 2 3 4 5 6 7 8 9 10
Mean = 3
1  2  3  4  5 15

3
5
5
Ka-fu Wong © 2007
0 1 2 3 4 5 6 7 8 9 10
Mean = 4
1  2  3  4  10 20

4
5
5
ECON1003: Analysis of Economic Data
Lesson2-8
Median
 In an ordered list, the median is the “middle” number (50% above,
50% below)
0 1 2 3 4 5 6 7 8 9 10
0 1 2 3 4 5 6 7 8 9 10
Median = 3
Median = 3
 Not affected by extreme values
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-9
Finding the Median
 The location of the median:
Median position 
n 1
position in the ordered data
2
 If the number of values is odd, the median is the middle
number
 If the number of values is even, the median is the average of
the two middle numbers
n 1
 Note that 2 is not the value of the median, only the position
of the median in the ranked data
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-10
Mode






A measure of central tendency
Value that occurs most often
Not affected by extreme values
Used for either numerical or categorical data
There may may be no mode
There may be several modes
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Mode = 9
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
0 1 2 3 4 5 6
No Mode
Lesson2-11
Review Example
 Five houses on a hill by the beach
$2,000 K
House Prices:
$2,000,000
500,000
300,000
100,000
100,000
$500 K
$300 K
 Mean:
= ($3,000,000/5)
= $600,000
 Median: middle value of
ranked data
= $300,000
 Mode: most frequent value
= $100,000
$100 K
$100 K
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-12
Which measure of location is the “best”?
 Mean is generally used, unless extreme values (outliers) exist
 Then median is often used, since the median is not sensitive
to extreme values.
 Example: Median home prices may be reported for a region
– less sensitive to outliers
 In census of income, it is often that several digits are reserved
to record income, say, 6 digits. The income will be recorded as
000000 to 999998. 999999 will be reserved for “missing value”.
That is, an individual with income 1234567 will be recorded as
999998. When we have substantial proportion of individual
with income larger than 999998, median will be a better
measure of central tendency.
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-13
Shape of a Distribution
 Describes how data are distributed
 Measures of shape
 Symmetric or skewed
Left-Skewed
Symmetric
Right-Skewed
Mean < Median
Mean = Median
Median < Mean
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-14
Relations between mean and median in
skewed distributions
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
4
5
6
Symmetric: mean = median = 0
-6
-5
-4
-3
-2
-1
0
1
2
3
Symmetric: mean = median = 0
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-15
Relations between mean and median in
skewed distributions
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
4
5
6
Symmetric distributions: mean = median = 0
-6
-5
-4
-3
-2
-1
0
1
2
3
Left skewed distributions: mean (= -1) < median (= 0)
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-16
Relations between mean and median in
skewed distributions
-6
-5
-4
-6
-5
-4
Ka-fu Wong © 2007
-3
-2
-1
0
1
4
5
6
-3
-2
-1
0
1
4
5
6
2
3
Symmetric distributions: mean = median = 0
2
3
Right skewed distributions: mean > median
ECON1003: Analysis of Economic Data
Lesson2-17
Measures of Variability
Variation
Range
Interquartile
Range
Variance
Standard
Deviation
Coefficient
of Variation
 Measures of variation give
information on the spread or
variability of the data values.
Same center, different variation
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-18
Range
 Simplest measure of variation
 Difference between the largest and the smallest observations:
Range = Xlargest – Xsmallest
Example:
0 1 2 3 4 5 6 7 8 9 10 11 12
13 14
Range = 14 - 1 = 13
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-19
Disadvantages of the Range
 Ignores the way in which data are distributed
7
8
9
10
11
Range = 12 - 7 = 5
12
7
8
9
10
11
12
Range = 12 - 7 = 5
 Sensitive to outliers
1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,3,3,3,3,4,5
Range = 5 - 1 = 4
1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,3,3,3,3,4,120
Range = 120 - 1 = 119
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-20
Interquartile Range
 interquartile range eliminates some outlier problems
 Eliminate high- and low-valued observations and
 Calculate the range of the middle 50% of the data
 Interquartile range = 3rd quartile – 1st quartile
IQR = Q3 – Q1
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-21
Interquartile Range
Example:
X
minimum
Q1
25%
12
Median
(Q2)
45
X
maximum
25%
25%
25%
30
Q3
57
70
Interquartile range
= 57 – 30 = 27
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-22
Quartiles
 Quartiles split the ranked data into 4 segments with an equal
number of values per segment
25%
25%
Q1
25%
Q2
25%
Q3
 The first quartile, Q1, is the value for which 25% of the
observations are smaller and 75% are larger
 Q2 is the same as the median (50% are smaller, 50% are larger)
 Only 25% of the observations are greater than the third quartile
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-23
Quartile Formulas
Find a quartile by determining the value in the appropriate
position in the ranked data, where
First quartile position:
Q1 = 0.25(n+1)
Second quartile position:
(the median position)
Q2 = 0.50(n+1)
Third quartile position:
Q3 = 0.75(n+1)
where n is the number of observed values
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-24
Quartiles
 Example: Find the first quartile
Sample Ranked Data: 11 12 13 16 16 17 18 21 22
(n = 9)
Q1 = is in the
0.25(9+1) = 2.5 position of the ranked data
so use the value half way between the 2nd and 3rd values,
so
Ka-fu Wong © 2007
Q1 = 12.5
ECON1003: Analysis of Economic Data
Lesson2-25
Population Variance
 Average of squared deviations of values from the mean
 Population variance:
N
σ2 
Where
 (x
i 1
2

μ)
i
N
μ = population mean
N = population size
xi = ith value of the variable x
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-26
Sample Variance
 Average (approximately) of squared deviations of values from the
mean
 Sample variance:
n
s2 
Where
2
(x

x
)
 i
i1
n -1
X = arithmetic mean
n = sample size
Xi = ith value of the variable X
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-27
Population Standard Deviation
 Most commonly used measure of variation
 Shows variation about the mean
 Has the same units as the original data
 Population standard deviation:
N
σ
Ka-fu Wong © 2007
 (x
i 1
i
 μ)2
N
ECON1003: Analysis of Economic Data
Lesson2-28
Sample Standard Deviation
 Most commonly used measure of variation
 Shows variation about the mean
 Has the same units as the original data
 Sample standard deviation:
n
S
Ka-fu Wong © 2007
2
(x

x
)
 i
i1
n -1
ECON1003: Analysis of Economic Data
Lesson2-29
Calculation Example:
Sample Standard Deviation
Sample
Data (xi) :
10
12
n=8
s
14
15
17
18
18
24
Mean = x = 16
(10  X ) 2  (12  x) 2  (14  x) 2  ...  (24  x) 2
n 1

(10  16) 2  (12  16) 2  (14  16) 2  ...  (24  16) 2
8 1

130
7
Ka-fu Wong © 2007

4.309
ECON1003: Analysis of Economic Data
Lesson2-30
Measuring variation
Small standard deviation
Large standard deviation
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-31
Comparing Standard Deviations
Data A
11
12
13
14
15
16
17
18
19
20 21
Mean = 15.5
s = 3.338
20 21
Mean = 15.5
s = 0.926
20 21
Mean = 15.5
s = 4.570
Data B
11
12
13
14
15
16
17
18
19
Data C
11
12
Ka-fu Wong © 2007
13
14
15
16
17
18
19
ECON1003: Analysis of Economic Data
Lesson2-32
Advantages of Variance and Standard Deviation
 Each value in the data set is used in the calculation
 Values far from the mean are given extra weight
(because deviations from the mean are squared)
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-33
Interpretation and Uses of the Standard
Deviation
 Chebyshev’s theorem: For any set of observations, the
minimum proportion of the values that lie within k
standard deviations of the mean is at least:
1
1- 2
k
where k is any constant greater than 1.
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Chebyshev’s theorem
Chebyshev’s theorem: For any set of observations, the
minimum proportion of the values that lie within k
standard deviations of the mean is at least 1- 1/k2
Ka-fu Wong © 2007
K
Coverage
1
0%
2
75.00%
3
88.89%
4
93.75%
5
96.00%
6
97.22%
ECON1003: Analysis of Economic Data
Lesson2-35
The Empirical Rule
 If the data distribution is bell-shaped, then the interval:
 μ  1σ contains about 68% of the values in the
population or the sample
68%
μ
μ  1σ
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-36
The Empirical Rule

μ  2σ contains about 95% of the values in the
population or the sample
 μ  3σ contains about 99.7% of the values in the
population or the sample
Ka-fu Wong © 2007
95%
99.7%
μ  2σ
μ  3σ
ECON1003: Analysis of Economic Data
Lesson2-37
Why are we concern about dispersion?
 Dispersion is used as a measure of risk.
 Consider two assets of the same expected (mean) returns.
 -2%, 0%,+2%
 -4%, 0%,+4%
 The dispersion of returns of the second asset is larger then
the first. Thus, the second asset is more risky.
 Thus, the knowledge of dispersion is essential for
investment decision. And so is the knowledge of expected
(mean) returns.
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-38
Coefficient of Variation
 Measures relative variation
 Always in percentage (%)
 Shows variation relative to mean
 Can be used to compare two or more sets of data measured in
different units
 s 
CV  
x 
  100%


Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-39
Comparing Coefficient
of Variation
 Stock A:
 Average price last year = $50
 Standard deviation = $5
 Stock B:
s
CVA  
x

$5
 100% 
100%  10%
$50

 Average price last year = $100
 Standard deviation = $5
s
$5
CVB    100% 
100%  5%
$100
x 
Both stocks have the same standard deviation, but stock B is less
variable relative to its price.
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-40
Sharpe Ratio and Relative Dispersion
 Sharpe Ratio is often used to measure the performance of
investment strategies, with an adjustment for risk.
 If X is the return of an investment strategy in excess of the
market portfolio, the inverse of the CV is the Sharpe Ratio.
 An investment strategy of a higher Sharpe Ratio is
preferred.
http://www.stanford.edu/~wfsharpe/art/sr/sr.htm
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-41
Using Microsoft Excel
 Descriptive Statistics can be obtained from Microsoft®
Excel
 Use menu choice:
tools / data analysis / descriptive statistics
 Enter details in dialog box
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-42
Using Excel
Use menu choice:
tools / data analysis /
descriptive statistics
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-43
Using Excel
(continued)
 Enter dialog box
details
 Check box for
summary statistics
 Click OK
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-44
Excel output
Microsoft Excel
descriptive statistics output,
using the house price data:
House Prices:
$2,000,000
500,000
300,000
100,000
100,000
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-45
2004 Regional Nominal GDP per Capita
Summary Statistics (yuan per person)
Mean
Standard Error
Median
Mode
Standard Deviation
Sample Variance
Kurtosis
Skewness
Range
Minimum
Maximum
Sum
Count
Ka-fu Wong © 2007
14079.39
1912.88
9608.00
#N/A
10650.44
113431828.11
7.14
2.50
51092.00
4215.00
55307.00
436461.00
31
Why is it different from
the national per capita
GDP (10561 yuan)?
ECON1003: Analysis of Economic Data
Lesson2-46
Weighted Mean
 The weighted mean of a set of data is
n
w x
x
w
i 1
i
i
w1x1  w 2 x 2  ...  w n x n

 wi
Where wi is the weight of the ith observation
 Use when data is already grouped into n classes, with wi values in the ith
class
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-47
Approximations for Grouped Data
Suppose a data set contains values m1, m2, . . ., mk, occurring with
frequencies f1, f2, . . . fK
 For a population of N observations the mean is
K
μ
 fimi
K
where
i1
N   fi
i1
N
 For a sample of n observations, the mean is
K
x
Ka-fu Wong © 2007
fm
i 1
i
n
i
K
where
n   fi
i1
ECON1003: Analysis of Economic Data
Lesson2-48
2004 Regional Nominal GDP per Capita
Summary Statistics (yuan per person)
Mean
Standard Error
Median
Mode
Standard Deviation
Sample Variance
Kurtosis
Skewness
Range
Minimum
Maximum
Sum
Count
Ka-fu Wong © 2007
14079.39
1912.88
9608.00
#N/A
10650.44
113431828.11
7.14
2.50
51092.00
4215.00
55307.00
436461.00
31
Why is it different from
the national per capita
GDP (10561 yuan)?
National per capita GDP
based on weighted
mean (with population
size as weights):
13107.66 yuan
ECON1003: Analysis of Economic Data
Lesson2-49
Approximations for Grouped Data
Suppose a data set contains values m1, m2, . . ., mk, occurring with
frequencies f1, f2, . . . fK
 For a population of N observations the variance is
K
σ2 
2
f
(m

μ)
i i
i 1
N
 For a sample of n observations, the variance is
K
s2 
Ka-fu Wong © 2007
2
f
(m

x
)
i i
i1
n 1
ECON1003: Analysis of Economic Data
Lesson2-50
The Sample Covariance
 The covariance measures the strength of the linear relationship
between two variables
 The population covariance:
N
Cov (x , y)   xy 
 (x
 The sample covariance:
i1
i
  x )(y i   y )
N
n
Cov (x , y)  s xy 
 (x  x)(y
i1
i
i
 y)
n 1
 Only concerned with the strength of the relationship
 No causal effect is implied
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-51
Interpreting Covariance
 Covariance between two variables:
Cov(x,y) > 0
x and y tend to move in the same direction
Cov(x,y) < 0
x and y tend to move in opposite directions
Cov(x,y) = 0
x and y are independent
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-52
Coefficient of Correlation
 Measures the relative strength of the linear relationship between
two variables
 Population correlation coefficient:
 Sample correlation coefficient:
Ka-fu Wong © 2007
Cov (x , y)
ρ
σXσY
r
Cov (x , y)
sX sY
ECON1003: Analysis of Economic Data
Lesson2-53
Features of Correlation Coefficient, r
 Unit free
 Ranges between –1 and 1
 The closer to –1, the stronger the negative linear relationship
 The closer to 1, the stronger the positive linear relationship
 The closer to 0, the weaker any linear relationship
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-54
Scatter Plots of Data with Various
Correlation Coefficients
Y
Y
Y
X
X
r = -1
r = -.6
Y
r=0
Y
Y
r = +1
Ka-fu Wong © 2007
X
X
X
r = +.3
ECON1003: Analysis of Economic Data
X
r=0
Lesson2-55
Using Excel to Find
the Correlation Coefficient
 Select
Tools/Data Analysis
 Choose Correlation from the
selection menu
 Click OK . . .
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-56
Using Excel to Find
the Correlation Coefficient
(continued)
 Input data range and select appropriate
options
 Click OK to get output
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-57
Interpreting the Result
100
y = 0.6666x + 30.241
R2 = 0.5376
95
Test #2 Score
90
85
80
75
70
70
75
80
85
90
95
100
Test #1 Score
 r = .733
 There is a relatively strong positive linear relationship between test score
#1 and test score #2.
 Students who scored high on the first test tended to score high on second
test.
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-58
Obtaining Linear Relationships
 An equation can be fit to show the best linear relationship between
two variables:
Y = β0 + β1X
where Y is the dependent variable and X is the independent variable
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-59
Least Squares Regression
 Estimates for coefficients β0 and β1 are found to minimize the sum
of the squared residuals
 The least-squares regression line, based on sample data, is
ˆ  b0  b1 x
y
 Where b1 is the slope of the line and b0 is the y-intercept:
sy
Cov(x, y)
b1 
r
2
sx
sx
Ka-fu Wong © 2007
b0  y  b1x
ECON1003: Analysis of Economic Data
Lesson2-60
Lesson 2:
Descriptive Statistics
- END -
Ka-fu Wong © 2007
ECON1003: Analysis of Economic Data
Lesson2-61