• Study Resource
• Explore

Survey

* Your assessment is very important for improving the work of artificial intelligence, which forms the content of this project

Document related concepts
no text concepts found
Transcript
```Lesson 2:
Descriptive Statistics
ECON1003: Analysis of Economic Data
Lesson2-1
2004 Regional Nominal GDP per Capita
Summary Statistics (yuan per person)
Mean
Standard Error
Median
Mode
Standard Deviation
Sample Variance
Kurtosis
Skewness
Range
Minimum
Maximum
Sum
Count
14079.39
1912.88
9608.00
#N/A
10650.44
113431828.11
7.14
2.50
51092.00
4215.00
55307.00
436461.00
31
ECON1003: Analysis of Economic Data
Guizhou
Shanghai
Lesson2-2
Outline
Measures of Central Tendency
Measures of Variability
Measures for two variables
ECON1003: Analysis of Economic Data
Lesson2-3
Population Parameters and Sample
Statistics



A population parameter is number calculated from all
the population measurements that describes some
aspect of the population.
 The population mean, denoted , is a population
parameter and is the average of the population
measurements.
A point estimate is a one-number estimate of the value
of a population parameter.
A sample statistic is number calculated using sample
measurements that describes some aspect of the
sample.
ECON1003: Analysis of Economic Data
Lesson2-4
Measures of Central Tendency
Overview
Central Tendency
Mean
Median
Mode
Midpoint of
ranked values
Most frequently
observed value
n
x
x
i1
i
n
Arithmetic
average
ECON1003: Analysis of Economic Data
Lesson2-5
The Mean
Imagine an economy with N
individuals (that is, a population)
with income X1, X2, …, XN
Sample x1, x2, …, xn
Income per capita ()
Population Mean
N

X
x
Sample Mean
n
i
i =1
N
ECON1003: Analysis of Economic Data
x
x
i
i =1
n
Lesson2-6
Arithmetic Mean
 The arithmetic mean (mean) is the most common measure of
central tendency
 For a population of N values:
N
x
x1  x 2  ...  x N
μ

N
N
i 1
i
Population
values
Population size
 For a sample of size n:
n
x
x
i 1
n
i
x1  x 2  ...  x n

n
Observed
values
Sample size
ECON1003: Analysis of Economic Data
Lesson2-7
Effect of extreme value on Arithmetic Mean
 The most common measure of central tendency
 Mean = sum of values divided by the number of values
 Affected by extreme values (outliers)
0 1 2 3 4 5 6 7 8 9 10
Mean = 3
1  2  3  4  5 15

3
5
5
0 1 2 3 4 5 6 7 8 9 10
Mean = 4
1  2  3  4  10 20

4
5
5
ECON1003: Analysis of Economic Data
Lesson2-8
Median
 In an ordered list, the median is the “middle” number (50% above,
50% below)
0 1 2 3 4 5 6 7 8 9 10
0 1 2 3 4 5 6 7 8 9 10
Median = 3
Median = 3
 Not affected by extreme values
ECON1003: Analysis of Economic Data
Lesson2-9
Finding the Median
 The location of the median:
Median position 
n 1
position in the ordered data
2
 If the number of values is odd, the median is the middle
number
 If the number of values is even, the median is the average of
the two middle numbers
n 1
 Note that 2 is not the value of the median, only the position
of the median in the ranked data
ECON1003: Analysis of Economic Data
Lesson2-10
Mode






A measure of central tendency
Value that occurs most often
Not affected by extreme values
Used for either numerical or categorical data
There may may be no mode
There may be several modes
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Mode = 9
ECON1003: Analysis of Economic Data
0 1 2 3 4 5 6
No Mode
Lesson2-11
Review Example
 Five houses on a hill by the beach
\$2,000 K
House Prices:
\$2,000,000
500,000
300,000
100,000
100,000
\$500 K
\$300 K
 Mean:
= (\$3,000,000/5)
= \$600,000
 Median: middle value of
ranked data
= \$300,000
 Mode: most frequent value
= \$100,000
\$100 K
\$100 K
ECON1003: Analysis of Economic Data
Lesson2-12
Which measure of location is the “best”?
 Mean is generally used, unless extreme values (outliers) exist
 Then median is often used, since the median is not sensitive
to extreme values.
 Example: Median home prices may be reported for a region
– less sensitive to outliers
 In census of income, it is often that several digits are reserved
to record income, say, 6 digits. The income will be recorded as
000000 to 999998. 999999 will be reserved for “missing value”.
That is, an individual with income 1234567 will be recorded as
999998. When we have substantial proportion of individual
with income larger than 999998, median will be a better
measure of central tendency.
ECON1003: Analysis of Economic Data
Lesson2-13
Shape of a Distribution
 Describes how data are distributed
 Measures of shape
 Symmetric or skewed
Left-Skewed
Symmetric
Right-Skewed
Mean < Median
Mean = Median
Median < Mean
ECON1003: Analysis of Economic Data
Lesson2-14
Relations between mean and median in
skewed distributions
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
4
5
6
Symmetric: mean = median = 0
-6
-5
-4
-3
-2
-1
0
1
2
3
Symmetric: mean = median = 0
ECON1003: Analysis of Economic Data
Lesson2-15
Relations between mean and median in
skewed distributions
-6
-5
-4
-3
-2
-1
0
1
2
3
4
5
6
4
5
6
Symmetric distributions: mean = median = 0
-6
-5
-4
-3
-2
-1
0
1
2
3
Left skewed distributions: mean (= -1) < median (= 0)
ECON1003: Analysis of Economic Data
Lesson2-16
Relations between mean and median in
skewed distributions
-6
-5
-4
-6
-5
-4
-3
-2
-1
0
1
4
5
6
-3
-2
-1
0
1
4
5
6
2
3
Symmetric distributions: mean = median = 0
2
3
Right skewed distributions: mean > median
ECON1003: Analysis of Economic Data
Lesson2-17
Measures of Variability
Variation
Range
Interquartile
Range
Variance
Standard
Deviation
Coefficient
of Variation
 Measures of variation give
variability of the data values.
Same center, different variation
ECON1003: Analysis of Economic Data
Lesson2-18
Range
 Simplest measure of variation
 Difference between the largest and the smallest observations:
Range = Xlargest – Xsmallest
Example:
0 1 2 3 4 5 6 7 8 9 10 11 12
13 14
Range = 14 - 1 = 13
ECON1003: Analysis of Economic Data
Lesson2-19
 Ignores the way in which data are distributed
7
8
9
10
11
Range = 12 - 7 = 5
12
7
8
9
10
11
12
Range = 12 - 7 = 5
 Sensitive to outliers
1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,3,3,3,3,4,5
Range = 5 - 1 = 4
1,1,1,1,1,1,1,1,1,1,1,2,2,2,2,2,2,2,2,3,3,3,3,4,120
Range = 120 - 1 = 119
ECON1003: Analysis of Economic Data
Lesson2-20
Interquartile Range
 interquartile range eliminates some outlier problems
 Eliminate high- and low-valued observations and
 Calculate the range of the middle 50% of the data
 Interquartile range = 3rd quartile – 1st quartile
IQR = Q3 – Q1
ECON1003: Analysis of Economic Data
Lesson2-21
Interquartile Range
Example:
X
minimum
Q1
25%
12
Median
(Q2)
45
X
maximum
25%
25%
25%
30
Q3
57
70
Interquartile range
= 57 – 30 = 27
ECON1003: Analysis of Economic Data
Lesson2-22
Quartiles
 Quartiles split the ranked data into 4 segments with an equal
number of values per segment
25%
25%
Q1
25%
Q2
25%
Q3
 The first quartile, Q1, is the value for which 25% of the
observations are smaller and 75% are larger
 Q2 is the same as the median (50% are smaller, 50% are larger)
 Only 25% of the observations are greater than the third quartile
ECON1003: Analysis of Economic Data
Lesson2-23
Quartile Formulas
Find a quartile by determining the value in the appropriate
position in the ranked data, where
First quartile position:
Q1 = 0.25(n+1)
Second quartile position:
(the median position)
Q2 = 0.50(n+1)
Third quartile position:
Q3 = 0.75(n+1)
where n is the number of observed values
ECON1003: Analysis of Economic Data
Lesson2-24
Quartiles
 Example: Find the first quartile
Sample Ranked Data: 11 12 13 16 16 17 18 21 22
(n = 9)
Q1 = is in the
0.25(9+1) = 2.5 position of the ranked data
so use the value half way between the 2nd and 3rd values,
so
Q1 = 12.5
ECON1003: Analysis of Economic Data
Lesson2-25
Population Variance
 Average of squared deviations of values from the mean
 Population variance:
N
σ2 
Where
 (x
i 1
2

μ)
i
N
μ = population mean
N = population size
xi = ith value of the variable x
ECON1003: Analysis of Economic Data
Lesson2-26
Sample Variance
 Average (approximately) of squared deviations of values from the
mean
 Sample variance:
n
s2 
Where
2
(x

x
)
 i
i1
n -1
X = arithmetic mean
n = sample size
Xi = ith value of the variable X
ECON1003: Analysis of Economic Data
Lesson2-27
Population Standard Deviation
 Most commonly used measure of variation
 Shows variation about the mean
 Has the same units as the original data
 Population standard deviation:
N
σ
 (x
i 1
i
 μ)2
N
ECON1003: Analysis of Economic Data
Lesson2-28
Sample Standard Deviation
 Most commonly used measure of variation
 Shows variation about the mean
 Has the same units as the original data
 Sample standard deviation:
n
S
2
(x

x
)
 i
i1
n -1
ECON1003: Analysis of Economic Data
Lesson2-29
Calculation Example:
Sample Standard Deviation
Sample
Data (xi) :
10
12
n=8
s
14
15
17
18
18
24
Mean = x = 16
(10  X ) 2  (12  x) 2  (14  x) 2  ...  (24  x) 2
n 1

(10  16) 2  (12  16) 2  (14  16) 2  ...  (24  16) 2
8 1

130
7

4.309
ECON1003: Analysis of Economic Data
Lesson2-30
Measuring variation
Small standard deviation
Large standard deviation
ECON1003: Analysis of Economic Data
Lesson2-31
Comparing Standard Deviations
Data A
11
12
13
14
15
16
17
18
19
20 21
Mean = 15.5
s = 3.338
20 21
Mean = 15.5
s = 0.926
20 21
Mean = 15.5
s = 4.570
Data B
11
12
13
14
15
16
17
18
19
Data C
11
12
13
14
15
16
17
18
19
ECON1003: Analysis of Economic Data
Lesson2-32
Advantages of Variance and Standard Deviation
 Each value in the data set is used in the calculation
 Values far from the mean are given extra weight
(because deviations from the mean are squared)
ECON1003: Analysis of Economic Data
Lesson2-33
Interpretation and Uses of the Standard
Deviation
 Chebyshev’s theorem: For any set of observations, the
minimum proportion of the values that lie within k
standard deviations of the mean is at least:
1
1- 2
k
where k is any constant greater than 1.
ECON1003: Analysis of Economic Data
Chebyshev’s theorem
Chebyshev’s theorem: For any set of observations, the
minimum proportion of the values that lie within k
standard deviations of the mean is at least 1- 1/k2
K
Coverage
1
0%
2
75.00%
3
88.89%
4
93.75%
5
96.00%
6
97.22%
ECON1003: Analysis of Economic Data
Lesson2-35
The Empirical Rule
 If the data distribution is bell-shaped, then the interval:
 μ  1σ contains about 68% of the values in the
population or the sample
68%
μ
μ  1σ
ECON1003: Analysis of Economic Data
Lesson2-36
The Empirical Rule

μ  2σ contains about 95% of the values in the
population or the sample
 μ  3σ contains about 99.7% of the values in the
population or the sample
95%
99.7%
μ  2σ
μ  3σ
ECON1003: Analysis of Economic Data
Lesson2-37
Why are we concern about dispersion?
 Dispersion is used as a measure of risk.
 Consider two assets of the same expected (mean) returns.
 -2%, 0%,+2%
 -4%, 0%,+4%
 The dispersion of returns of the second asset is larger then
the first. Thus, the second asset is more risky.
 Thus, the knowledge of dispersion is essential for
investment decision. And so is the knowledge of expected
(mean) returns.
ECON1003: Analysis of Economic Data
Lesson2-38
Coefficient of Variation
 Measures relative variation
 Always in percentage (%)
 Shows variation relative to mean
 Can be used to compare two or more sets of data measured in
different units
 s 
CV  
x 
  100%


ECON1003: Analysis of Economic Data
Lesson2-39
Comparing Coefficient
of Variation
 Stock A:
 Average price last year = \$50
 Standard deviation = \$5
 Stock B:
s
CVA  
x

\$5
 100% 
100%  10%
\$50

 Average price last year = \$100
 Standard deviation = \$5
s
\$5
CVB    100% 
100%  5%
\$100
x 
Both stocks have the same standard deviation, but stock B is less
variable relative to its price.
ECON1003: Analysis of Economic Data
Lesson2-40
Sharpe Ratio and Relative Dispersion
 Sharpe Ratio is often used to measure the performance of
investment strategies, with an adjustment for risk.
 If X is the return of an investment strategy in excess of the
market portfolio, the inverse of the CV is the Sharpe Ratio.
 An investment strategy of a higher Sharpe Ratio is
preferred.
http://www.stanford.edu/~wfsharpe/art/sr/sr.htm
ECON1003: Analysis of Economic Data
Lesson2-41
Using Microsoft Excel
 Descriptive Statistics can be obtained from Microsoft®
Excel
tools / data analysis / descriptive statistics
 Enter details in dialog box
ECON1003: Analysis of Economic Data
Lesson2-42
Using Excel
tools / data analysis /
descriptive statistics
ECON1003: Analysis of Economic Data
Lesson2-43
Using Excel
(continued)
 Enter dialog box
details
 Check box for
summary statistics
 Click OK
ECON1003: Analysis of Economic Data
Lesson2-44
Excel output
Microsoft Excel
descriptive statistics output,
using the house price data:
House Prices:
\$2,000,000
500,000
300,000
100,000
100,000
ECON1003: Analysis of Economic Data
Lesson2-45
2004 Regional Nominal GDP per Capita
Summary Statistics (yuan per person)
Mean
Standard Error
Median
Mode
Standard Deviation
Sample Variance
Kurtosis
Skewness
Range
Minimum
Maximum
Sum
Count
14079.39
1912.88
9608.00
#N/A
10650.44
113431828.11
7.14
2.50
51092.00
4215.00
55307.00
436461.00
31
Why is it different from
the national per capita
GDP (10561 yuan)?
ECON1003: Analysis of Economic Data
Lesson2-46
Weighted Mean
 The weighted mean of a set of data is
n
w x
x
w
i 1
i
i
w1x1  w 2 x 2  ...  w n x n

 wi
Where wi is the weight of the ith observation
 Use when data is already grouped into n classes, with wi values in the ith
class
ECON1003: Analysis of Economic Data
Lesson2-47
Approximations for Grouped Data
Suppose a data set contains values m1, m2, . . ., mk, occurring with
frequencies f1, f2, . . . fK
 For a population of N observations the mean is
K
μ
 fimi
K
where
i1
N   fi
i1
N
 For a sample of n observations, the mean is
K
x
fm
i 1
i
n
i
K
where
n   fi
i1
ECON1003: Analysis of Economic Data
Lesson2-48
2004 Regional Nominal GDP per Capita
Summary Statistics (yuan per person)
Mean
Standard Error
Median
Mode
Standard Deviation
Sample Variance
Kurtosis
Skewness
Range
Minimum
Maximum
Sum
Count
14079.39
1912.88
9608.00
#N/A
10650.44
113431828.11
7.14
2.50
51092.00
4215.00
55307.00
436461.00
31
Why is it different from
the national per capita
GDP (10561 yuan)?
National per capita GDP
based on weighted
mean (with population
size as weights):
13107.66 yuan
ECON1003: Analysis of Economic Data
Lesson2-49
Approximations for Grouped Data
Suppose a data set contains values m1, m2, . . ., mk, occurring with
frequencies f1, f2, . . . fK
 For a population of N observations the variance is
K
σ2 
2
f
(m

μ)
i i
i 1
N
 For a sample of n observations, the variance is
K
s2 
2
f
(m

x
)
i i
i1
n 1
ECON1003: Analysis of Economic Data
Lesson2-50
The Sample Covariance
 The covariance measures the strength of the linear relationship
between two variables
 The population covariance:
N
Cov (x , y)   xy 
 (x
 The sample covariance:
i1
i
  x )(y i   y )
N
n
Cov (x , y)  s xy 
 (x  x)(y
i1
i
i
 y)
n 1
 Only concerned with the strength of the relationship
 No causal effect is implied
ECON1003: Analysis of Economic Data
Lesson2-51
Interpreting Covariance
 Covariance between two variables:
Cov(x,y) > 0
x and y tend to move in the same direction
Cov(x,y) < 0
x and y tend to move in opposite directions
Cov(x,y) = 0
x and y are independent
ECON1003: Analysis of Economic Data
Lesson2-52
Coefficient of Correlation
 Measures the relative strength of the linear relationship between
two variables
 Population correlation coefficient:
 Sample correlation coefficient:
Cov (x , y)
ρ
σXσY
r
Cov (x , y)
sX sY
ECON1003: Analysis of Economic Data
Lesson2-53
Features of Correlation Coefficient, r
 Unit free
 Ranges between –1 and 1
 The closer to –1, the stronger the negative linear relationship
 The closer to 1, the stronger the positive linear relationship
 The closer to 0, the weaker any linear relationship
ECON1003: Analysis of Economic Data
Lesson2-54
Scatter Plots of Data with Various
Correlation Coefficients
Y
Y
Y
X
X
r = -1
r = -.6
Y
r=0
Y
Y
r = +1
X
X
X
r = +.3
ECON1003: Analysis of Economic Data
X
r=0
Lesson2-55
Using Excel to Find
the Correlation Coefficient
 Select
Tools/Data Analysis
 Choose Correlation from the
 Click OK . . .
ECON1003: Analysis of Economic Data
Lesson2-56
Using Excel to Find
the Correlation Coefficient
(continued)
 Input data range and select appropriate
options
 Click OK to get output
ECON1003: Analysis of Economic Data
Lesson2-57
Interpreting the Result
100
y = 0.6666x + 30.241
R2 = 0.5376
95
Test #2 Score
90
85
80
75
70
70
75
80
85
90
95
100
Test #1 Score
 r = .733
 There is a relatively strong positive linear relationship between test score
#1 and test score #2.
 Students who scored high on the first test tended to score high on second
test.
ECON1003: Analysis of Economic Data
Lesson2-58
Obtaining Linear Relationships
 An equation can be fit to show the best linear relationship between
two variables:
Y = β0 + β1X
where Y is the dependent variable and X is the independent variable
ECON1003: Analysis of Economic Data
Lesson2-59
Least Squares Regression
 Estimates for coefficients β0 and β1 are found to minimize the sum
of the squared residuals
 The least-squares regression line, based on sample data, is
ˆ  b0  b1 x
y
 Where b1 is the slope of the line and b0 is the y-intercept:
sy
Cov(x, y)
b1 
r
2
sx
sx