Download Numerical Summary Measures of Variability for Data

Ch2.2, Ch2.3 Numerical Summary Measures of Variability for Data
Topics:
 Measures of variability (spread):
o Deviation
o Variance/standard deviation
o Interquartile range
 Statistical definition of Outliers: the 1.5 x IQR criterion for outliers
-----------------------------------------------------------------------------------------------------------
II: Measures of Variability (Spread)
(1) Deviation: the difference between the observation and the mean
Deviation of xi is defined as di  xi  x .
What is the mean deviation ( d ) ?
1
1
 d1  d 2  ...  d n    x1  x  x2  x  ...  xn  x 
n
n
1
1
  x1  x2  ...  xn  nx    nx  nx   0
n
n
d
So d does not measure the spread in the data. We might use the average of
absolute deviation:
1
| d1 |  | d 2 | ... | d n | .
n
But this measure is mathematically inconvenient. Then we consider the following
definition.
(2) Variance / Standard Deviation
Sample Variance: the sample variance s 2 of n observations is average
(using n-1) of squared deviations
n
s2 
 x
i 1
 x
2
i
n 1
1
► An alternative form convenient for calculation
s2 

1
  x i n( x ) 2 
n 1 
Sample Standard Deviation (SD): the sample standard deviation s
is the square root of s 2 :
n
s
 x
i 1
i
 x
2
n 1
Ex. Grades of 9 students on a HW assignment: 86,85,81,82,84,84,83,84,87.
x  84 . What is the SD? (Also know that
9
x
i 1
2
i
 63532
 9

s 2 =   xi2  n  ( x ) 2  / 8  63532 – 9 * 842 = 28/8=3.5
 i 1

s  s 2  3.5  1.87.
 Interpretation: A random student’s score is about 1.87 points from the
mean score (84).
Comments:
1. SD should be used as a measure of spread only when mean is used as the
measure of center
2. SD=0 implies all data points are the same (no variability)
3. Like mean, SD is strongly influenced by outliers
Ex. If a coding error makes 87 to be 870, then x =182 and SD becomes 278…
(3) Interquartile Range (IQR)
(a) Quartiles: Q1 = first quartile = median of the lower half of the
data. Q3 = third quartile = median of the upper half of the data
(Q2 = median).
IQR = Q3 – Q1 (good spread measure in the presence of outliers)
2
To find quartiles:
1. Sort the data and divide data points into 2 halves (If there are odd
number of observations, include the median in each half.)
2. Lower quartile  Q1= median of the lower half of the data.
3. Upper quartile  Q3 = median of the upper half of the data.
(b) Inter-Quartile Range (IQR)

IQR = Q3 – Q1

Interpretation of IQR: Measure of variability (spread) of data,
similar to s but usually larger than s.
Ex. Grades of 9 students on a HW assignment: 86,85,81,82,84,84,83,84,87.
The sorted hw scores are:
81 82 83 84 84 85 86 87. So median = 84, Q1 = 83, Q3 = 85, IQR = 85
– 83 = 2.
3
Ex. Rainfall in NC in the some 15 months
1|0
2 | 25
3 | 45
Stem: one digit
4 | 11667
Leaf: tenths digit
5 | 449
6|0
7|
8|2
Find quartiles and the IQR
Remark 1: The 1.5 x IQR Criterion for Outliers
An observation is called an outlier if it is 1.5*IQR larger than Q3 or 1.5*IQR
smaller than Q1.
Extreme outlier may indicate data entry error or unusual characteristics in
the data that need careful investigation (if it is 3*IQR larger than Q3 or
3*IQR smaller than Q1.
Ex. Use the 1.5xIQR rule to check if there is any outlier in the Rainfall dataset.
Q1 = 3.45, Q3 = 5.4, IQR = 5.4 – 3.45 = 2.925.
Q1 – 2.925 = 3.45 – 2.925 = 0.525 (no data point smaller than 0.525)
Q3 + 2.925 = 5.4 + 2.925 = 8.325 (no data point larger than 8.325)
Remark 2: Standard numerical summaries of a data set includes sample size,
center, and spread.
For reasonably symmetric distribution with no outliers, use x , s
For the rest situation, use x, IQR
4
Remark 3: The 5-number summary
min, Q1, Median, Q3, max
Remark 4: Change of Unit
1. Adding (or subtracting) a constant to each observation will NOT change
the measures of spread, such as SD and IQR, (but does change the measures
of center and quartiles)
If the new unit = 60 + the old unit; do the spread of the data change?
2. Multiplying each observation by a constant a does multiply measures of
spread (SD and IQR) by |a|.
5
Conclusion: If new unit is aX + b, then the new spread in terms of IQR is |a|
times the original IQR
( Recall that the new center is ax  b )
Ex. Temperatures read in Fahrenheit and the SD temperature is s, and IQR is r.
What are the new SD and IQR if we switch to Centigrade? Note that
5
C  F  32  .
9
The new SD is 5s/9.
The new IQR = 5r/9.
6