Download Descriptive Statistics

Descriptive Statistics
Prof. David Ramsey
e-mail: [email protected]
Home page: www.ioz.pwr.edu.pl/pracownicy/ramsey
February 27, 2017
1 / 69
Brief Course Outline
1. Data Collection and Descriptive Statistics.
2. Probability Theory
3. Statistical Methods of Estimation
2 / 69
1 Data Collection and Descriptive Statistics
Populations of objects and individuals show variation with respect
to various traits (e.g. height, political preferences, the working life
of a light bulb, investment risk).
It is impractical (or impossible) to observe all the members of the
population. In order to describe the distribution of a trait in the
population, we select a sample.
On the basis of the sample we gain information on the population
as a whole.
3 / 69
1.1 Types of Variables
Qualitive variables: These are normally categorical
(non-numerical) variables. We distinguish between two types of
qualitative variables:
a) nominal: these variables are not naturally ordered in any way
(e.g. i. department - mechanical engineering, mathematics,
economics ii. industrial sector).
b) ordinal: there is a natural order for such categorisations e.g.
with respect to smoking, people may be categorised as 1:
non-smokers, 2: light smokers and 3: heavy smokers.
It can be seen that the higher the category number, the more an
individual smokes.
Exam grades are ordinal variables.
4 / 69
Quantitative Variables
These are variables which naturally take numerical values (e.g.
age, height, number of children). Such variables can be measured
or counted.
As before, we distinguish between two types of quantitative
variables.
a) Discrete variables: these are variables that take values from a
set that can be listed (most commonly integer values, i.e. they are
variables that can be counted). For example, number of children,
the results of die rolls.
5 / 69
b) Continuous variables
These are variables that can take values in a given range to any
number of decimal places (such variables are measured, normally
according to some unit e.g. height, age, weight).
It should be noted that such variables are only measured to a given
accuracy (i.e. height is measured to the nearest centimetre, age is
normally given to the nearest year).
If a discrete random variable takes many values (i.e. the
population of a town), then for practical purposes it is treated as a
continuous variable.
6 / 69
1.2 Collection of data
Since it is impractical to survey all the individuals in a population,
we need to base our analysis on a sample.
A population is the entire collection of individuals or objects we
wish to describe.
A sample is the subset of the population chosen for data
collection.
A sampling frame is a list that is used to choose a sample (e.g.
electoral register, telephone book, list of addresses).
A unit is a member of the population.
A variable is any trait that varies from unit to unit.
The sample size is the number of individuals in a sample and is
denoted by n.
7 / 69
1.2.1 Parameters and Statistics
A parameter is an unkown number describing a population. For
example, it may be that 37% of the population of eligible voters
(the electorate) wish to vote for PiS (we do not, however, observe
this population proportion).
A statistic is a number describing a sample. For example, 35% of
a sample may wish to vote for PiS. This is the sample proportion.
Statistics may be used to describe a population, but they only
estimate the real parameters of the population.
8 / 69
Parameters and Statistics - Precision of Statistics
Naturally, the statistics from a sample will show some variation
around the appropriate parameters
e.g. 37% of the population wish to vote for PiS, but only 35% in
the sample.
The greater the sample size, the more precise the results (suppose
we take a large number of samples of size n, the larger n the less
variable the sample proportion from the various samples, i.e. the
more replicable the results).
9 / 69
Parameters and Statistics - Bias
However, there may be intrinsic bias from two possible sources:
a) Sampling bias - when a sample is chosen in a way
such that some members of the population are more
likely to be chosen than others. e.g. The support for
PO is stronger in urban centres and in the west of
Poland. Hence, if we carry out a survey in Wroclaw
and use this to estimate the support for PO in
Poland as a whole, we will tend to overestimate
support for PO.
10 / 69
Parameters and Statistics - Bias
b) Non-Sampling Bias This results from mistakes in
data entry and/or how interviewees react to being
questioned. For example, it has been noted that
support of the government is often underestimated
by opinion polls. This may well be due to the fact
that supporters of the government are more likely to
hide their preferences.
11 / 69
An Example of Sampling Bias - Estimation of the
Population Mean
The sampling bias can be eliminated by choosing a sample in a
more appropriate way, but not non-sampling bias.
e.g. Suppose the population of interest is the Polish population as
a whole and the variable of interest is height.
Suppose I base my estimate of the mean height of the population
on the mean height of a sample of students (i.e. the sampling
frame or means of selecting a sample is inappropriate). Since
students tend to be on average taller than the population as a
whole, I will systematically overestimate the mean height in the
population.
That is to say, if I consider many samples of students of say size
100, a large majority of such samples would give me an
overestimate of the mean height of the population as a whole.
12 / 69
Non-Sampling Bias
The following may be sources of non-sampling bias:
1. Lack of anonymity.
2. The wording of a question.
3. The desire to give an answer that would please the
interviewer. For example, surveys may systematically
overestimate the willingness of individuals to pay
extra for environmentally friendly goods, as stating
that you are prepared to pay more is seen to be the
”politically correct” answer.
13 / 69
Precision and Bias
It should be noted that bias is a characteristic of the way in which
data are collected, not a single sample.
Increasing the sample size will improve the precision of an
estimate, but will not affect the bias.
Returning to the example of height. As the sample size increases,
the sample mean becomes more replicable. However, if we are
estimating the mean height of the entire population based on
samples of students, there will always be a tendency to
overestimate the mean height of the population.
14 / 69
Sampling - Conclusion
In order for a survey to give accurate results, it is not sufficient
that the sample is large. We should ask the following questions:
1. Is the sample representative of the required
population? (sampling bias).
2. Is the proportion of non-respondents large? If so, are
non-respondents likely to differ from respondents and
how? (non-sampling bias).
3. Are the answers to questions reliable? (possibility of
”political correctness”, reaction to the questioner, i.e.
non-sampling bias).
4. Who is publishing the results? (they are likely to
stress what looks good to them).
15 / 69
1.3 Descriptive Statistics - 1.3.1 Qualitative (Categorical
Data)
We may describe qualitative data using
a) Frequency tables.
b) Bar charts.
c) Pie charts.
16 / 69
Frequency tables
Frequency tables display how many observations fall into each
category (the frequency column), as well as the relative frequency
of each category (the proportion of observations falling into each
category).
Let ni denote the number of observations in category i. The
relative frequency of category i in percentage terms is fi , where
fi =
100ni
n
If there are missing data, we may also give the relative frequencies
in terms of the actual number of observations, n0 i.e.
fi 0 =
100ni
n0
17 / 69
Frequency tables
For example 200 students were asked which of the following
musical acts they preferred: Arctic Monkeys, Alt-J or Adele. The
answers may be presented in the following frequency table:
Band
Arctic Monkeys
Alt-J
Adele
Frequency
62
66
72
Relative Frequency (% )
62 × 100/200 = 31
66 × 100/200 = 33
72 × 100/200 = 36
18 / 69
Bar chart
In a bar chart the height of a bar represents the relative frequency
of a given category (or the number of observations in that
category).
19 / 69
Pie chart
The size of a slice in a pie chart represents the relative frequency
of a category. 100% corresponds to 360 degrees. Hence, the angle
made by the slice representing category i is given (in degrees) by
αi , where
100ni
360ni
αi = 3.6fi = 3.6 ×
=
n
n
20 / 69
Pie chart
These graphs were obtained using the SPSS (PASW) package.
21 / 69
1.3.2 Graphical Presentation of Quantitative Data
Discrete data can be presented in the form of frequency tables
and/or bar charts (as above).
The distribution of continuous data can be presented using a
histogram.
The histogram estimates the probability density function of a
continuous random variable (see later).
22 / 69
Histograms for continuous variables
In order to draw a histogram for a continuous variable, we need to
categorise the data into intervals of equal length. The end points
of these intervals should be round numbers.
√
The number of categories used should be approximately n
(normally between 5 and 20 categories are used).
For
√ example, if we have 30 observations then we should use about
30 ≈ 5.5 categories. Hence, 5 and 6 are sensible choices for the
number of categories.
Let k be the number of categories.
23 / 69
Histograms
In order to choose the length of each interval, L, we use
xmax − xmin
,
k
where xmax is smallest ”round” number larger than all the
observations and xmin is the largest ”round” number smaller than
all the observations.
L≈
If necessary, we can ”fine tune” by rounding L upwards, so that the
intervals are of ”nice” length and the whole range of the data is
covered.
24 / 69
Histograms
The intervals used are
[xmin , xmin + L], (xmin + L, xmin + 2L], . . . , (xmax − L, xmax ].
In general the lower end-point of an interval is assumed not to
belong to that interval (to avoid a number belonging to two
classes).
25 / 69
Histograms
A histogram is very similar to a bar chart. The height of the block
corresponding to an interval is the relative frequency of
observations in that block.
Thus, the height of a block is the number of observations in that
interval divided by the total number of observations.
26 / 69
Example 1.1
We observe the height of 20 individuals (in cm). The data are
given below
172, 165, 188, 162, 178, 183, 171, 158, 174, 184,
167, 175, 192, 170, 179, 187, 163, 156, 178, 182.
Draw a histogram representing these data.
27 / 69
Example 1.1
First we choose the number of classes and the corresponding
intervals.
√
20 ≈ 4.5, thus we should choose 4 or 5 intervals.
28 / 69
Example 1.1
The tallest individual is 192cm tall and the shortest 156cm.
200cm is the smallest round number larger than all the
observations and 150cm is the largest round number smaller than
all the observations.
To calculate the length of the intervals
L=
200 − 150
.
k
Taking k to be 4, L = 12.5. Taking k = 5, L = 10 (a ”nicer”
length).
Hence, it seems reasonable to use 5 intervals of length 10, starting
at 150.
29 / 69
Example 1.1
If we assume that the upper endpoint of an interval belongs to
that interval, then we have the intervals [150,160], (160, 170],
(170,180], (180,190], (190,200].
Now we count how many observations fall into each interval and
hence the relative frequency of observations in each interval.
30 / 69
Example 1.1
Height (x)
150 ≤ x ≤ 160
160 < x ≤ 170
170 < x ≤ 180
180 < x ≤ 190
190 < x ≤ 200
No. of Observations
2
5
7
5
1
Rel. Frequency
100 × 2/20 = 10
100 × 5/20 = 25
100 × 7/20 = 35
100 × 5/20 = 25
100 × 1/20 = 5
31 / 69
Example 1.2
The histogram is given below:
32 / 69
Interpretation of the histogram of a continuous variable
A histogram is an estimator of the density function of a variable
(see the chapter on the distribution of random variables in Section
2).
The distribution of height seems to be reasonably symmetrical
around 175cm.
33 / 69
1.3.3 Symmetry and Skewness of Distributions
From a histogram we may infer whether the distribution of a
random variable is symmetric or not.
The histogram of height shows that the distribution is reasonably
symmetric (even if the distribution of height in the population were
symmetric, we would normally observe some small deviation from
symmetry in the histogram, as we observe only a sample).
34 / 69
Right-Skewed distributions
A distribution is said to be right-skewed if there are observations a
long way to the right of the ”centre” of the distribution, but not a
long way to the left.
The distribution of wages is right-skewed, since a small proportion
of individuals will earn several times more than the mean wage.
35 / 69
A right-skewed distribution
36 / 69
Left-skewed distributions
A distribution is said to be left-skewed if there are observations a
long way to the left of the ”centre” of the distribution, but not a
long way to the right.
For example, the distribution of the lifetime of individuals in
Western countries is left-skewed.
This is due to the fact that the majority of individuals will live
between 70-100 years.
No-one will live much longer, but a minority of individuals will die
at a young age.
37 / 69
A Left-skewed Distribution
38 / 69
1.4 Numerical Methods of Describing Quantitative Data
We consider two types of measure:
1. Measures of centrality - give information regarding
the location of the centre of the distribution (the
mean, median).
2. Measures of variability (dispersion) - give information
regarding the level of variation (the range, variance,
standard deviation, interquartile range).
39 / 69
1.4.1 Measures of centrality
1. The Sample Mean, x.
Suppose we have a sample of n observations, the mean is given by
the sum of the observations divided by the number of observations.
n
1X
x=
xi ,
n
i=1
where xi is the value of the i-th observation.
40 / 69
The Population Mean
µ denotes the population mean. If there are N units in the
population, then
PN
xi
µ = i=1 ,
N
where xi is the value of the trait for individual i in the population.
µ is normally unknown. The sample mean x (a statistic) is an
estimator of the population mean µ (a parameter).
41 / 69
2. The sample median Q2
In order to calculate the sample median, we first order the
observations from the smallest to the largest. The order statistic
x(i) is the i-th smallest observation in a sample (i.e. x(1) is the
smallest observation and x(n) is the largest observation).
The notation for the median comes from the fact that the median
is the second quartile (see quartiles in the section on measures of
dispersion).
42 / 69
The sample median Q2
If n is odd, then the median is the observation which appears in
the centre of the ordered list of observations. Hence,
Q2 = x(0.5[n+1]) .
If n is even, then the median is the average of the two observations
which appear in the centre of the ordered list of observations.
Hence,
Q2 = 0.5[x(0.5n) + x(0.5n+1) ]
One half of the observations are smaller than the median and one
half are greater.
43 / 69
The sample median
One advantage of the median as a measure of centrality is that it
is less sensitive to extreme observations (which may be errors) than
the mean. When the distribution is skewed, it is preferable to use
the median as a measure of centrality.
e.g. the median wage rather than the average wage should be used
as a measure of what the ”average man on the street earns”.
The distribution of wages is right-skewed and the small proportion
of people who earn very high wages will have a significant effect on
the mean. The mean is greater than the median.
For left-skewed distributions the mean is less than the median.
44 / 69
1.4.2 Measures of Dispersion - 1. The Range
The range is defined to be the largest observation minus the
smallest observation.
Since the range is only based on 2 observations it conveys little
information and is sensitive to extreme values (errors).
45 / 69
2
2. The sample variance sn−1
The sample variance is a measure of the average square distance
from the mean.
2
The formula for the sample variance sn−1
is given by
n
2
sn−1
1 X
=
(xi − x)2 .
n−1
i=1
2
2
sn−1
≥ 0 and sn−1
= 0 if and only if all the observations are equal
to each other.
46 / 69
3. The sample standard deviation s
The sample standard deviation is given by the square root of the
variance.
It (and hence the sample variance) can be calculated on a scientific
calculator by using the σn−1 or sn−1 function as appropriate.
In simple terms, the standard deviation is a measure of the average
distance of an observation from the mean. It cannot be greater
than the maximum deviation from the mean.
47 / 69
4. The interquartile range
The i-th quartile, Qi , is taken to be the value such that i quarters
of the observations are less than Qi . Thus, Q2 is the sample
median.
If
n+1
4
is an integer, then the lower quartile Q1 is given by
Q1 = x( n+1 )
4
Otherwise, if a is the integer part of n+1
4 [this is obtained by simply
removing everything after the decimal point], then
Q1 = 0.5[x(a) + x(a+1) ]
48 / 69
The interquartile range
If
3n+3
4
is an integer, then the upper quartile Q3 is given by
Q3 = x( 3n+3 )
4
Otherwise, if b is the integer part of
3n+3
4 ,
then
Q3 = 0.5[x(b) + x(b+1) ]
The interquartile range (IQR) is the difference between the upper
and lower quartiles
IQR = Q3 − Q1
49 / 69
Choice of the measure of dispersion
The units of all the measures used so far (except for the variance)
are the same units as those used for the measurement of
observations. The units of variance are the square of the units of
measurement.
For example, if we observe velocity in metres per second, the
variance is measured in metres squared per second squared. For
this reason the standard deviation is generally preferred to the
variance as a measure of dispersion.
If a distribution is skewed then the interquartile range is a more
reliable measure of the dispersion of a random variable than the
standard deviation.
50 / 69
Comparison of the dispersion of two variables
Sometimes we wish to compare the dispersion of two positive
variables.
In cases where different units are used to measure the two variables
or the means of two variables are very different, it may be useful to
use a measure of dispersion which does not depend on the units in
which it is measured.
The coefficient of variation C .V . does not depend on the units of
measurement. It is the standard deviation divided by the sample
mean
sn−1
.
C .V . =
x
51 / 69
Example 1.2 - The sample mean
Calculate the measures of centrality and dispersion defined above
for the following data.
6, 9, 12, 9, 8, 10
There are 6 items of data hence,
P6
x=
i=1 xi
6
=
6 + 9 + 12 + 9 + 8 + 10
=9
6
52 / 69
Example 1.2 - The sample median
In order to calculate the median, we first order the data. If an
observation occurs k times, then it must appear k times in the list
of ordered data.
The ordered list of data is 6, 8, 9, 9, 10, 12.
Since there is an even number of data (n = 6), the median is the
average of the two observations in the middle of this ordered list.
Hence,
Q2 = 0.5[x(n/2) + x(1+ n2 ) ] = 0.5[x(3) + x(4) ] =
9+9
2
53 / 69
Example 1.2 - The range
The range is the difference between the largest and the smallest
observations
Range = 12 − 6 = 6.
54 / 69
Example 1.2 - The variance and standard deviation
The variance is given by
n
2
sn−1
=
1 X
(xi − x)2
n−1
i=1
(6 − 9)2 + (9 − 9)2 + (12 − 9)2 + (9 − 9)2 + (8 − 9)2 + (10 − 9)2
=
5
=4
q
2
The standard deviation is given by sn−1 = sn−1
= 2.
55 / 69
Example 1.2 - The interquartile range
In order to calculate the interquartile range, we first calculate the
lower and upper quartiles. n = 6, hence n+1
4 = 1.75. The integer
part of this number is 1. Hence, the lower quartile is
Q1 = 0.5[x(1) + x(2) ] = 0.5(6 + 8) = 7
Similarly, 3n+3
4 = 5.25. The integer part of this number is 5.
Hence, the upper quartile is
Q3 = 0.5[x(5) + x(6) ] = 0.5(10 + 12) = 11.
Hence,
IQR = 11 − 7 = 4.
56 / 69
Example 1.2 - The coefficient of variation
C .V . =
2
sn−1
= .
x
9
A coefficient of variation above 1 is accepted to be very large (such
variation may occur in the case of wages when wage inequality is
high).
With regard to physical traits in a species, values for the coefficient
of variation of around 0.1 to 0.3 are common (in humans the
coefficient of variation of height is around 0.1, the coefficient of
variation for weight is somewhat bigger).
57 / 69
1.5 Measures of Location and Dispersion for Grouped Data
- a) Discrete Random Variables
A die was rolled 100 times and the following data were obtained
Result
1
2
3
4
5
6
No. of observations
15
18
20
14
15
18
58 / 69
Grouped discrete data
Suppose the possible results are {x1 , x2 , . . . , xk } and the result xi
occurs fi times.
The total number of observations is
n=
k
X
fi .
i=1
The sum of the observations is given by
k
X
xi fi .
i=1
59 / 69
Grouped discrete data
It follows that the sample mean is given by
Pk
x=
i=1 fi xi
n
The variance of the observations is given by
k
2
sn−1
1 X
=
fi (xi − x)2
n−1
i=1
60 / 69
Grouped discrete data
The following table is useful in calculating the sample mean
xi
1
2
3
4
5
6
P
fi
15
18
20
14
15
18
100
Hence, the sample mean is x =
350
100
fi x i
15
36
60
56
75
108
350
= 3.5.
61 / 69
Grouped discrete data
Once the mean has been calculated, we can add two columns for
(xi − x)2 and fi (xi − x)2 :
xi
1
2
3
4
5
6
P
x
fi
15
18
20
14
15
18
100
350
100 = 3.5
fi xi
15
36
60
56
75
108
350
(xi − x)2
2.52
1.52
0.52
0.52
1.52
2.52
2
sn−1
fi (xi − x)2
15 × 2.52 = 93.75
18 × 1.52 = 40.5
20 × 0.52 = 5
14 × 0.52 = 3.5
15 × 1.52 = 33.75
18 × 2.52 = 112.5
289
289
99 = 2.92
62 / 69
Grouped discrete data
The sample variance is given by
k
2
sn−1
1 X
289
=
= 2.92.
fi (xi − x)2 =
n−1
99
i=1
63 / 69
Calculation of the sample median for grouped discrete data
In this case we know the exact values of the observations and hence
we can order the data. In this way we can calculate the median.
Since there are 100 observations, the median is
Q2 = 0.5[x(50) + x(51) ]
64 / 69
Calculation of the sample median for grouped discrete data
The 15 smallest observations are equal to 1 i.e.
x(1) = x(2) = . . . = x(15) = 1.
The next 18 smallest observations are equal to 2 i.e.
x(16) = x(17) = . . . = x(33) = 2.
The next 20 smallest observations are all equal to 3 i.e.
x(34) = x(35) = . . . = x(53) = 3.
65 / 69
Calculation of the sample median for grouped discrete data
It follows that
x(50) = x(51) = 3.
Hence,
Q2 = 0.5[x(50) + x(51) ] = 3.
66 / 69
1.5 Measures of Location and Dispersion for Grouped Data
- b) Continuous Random Variables
In such cases we have data grouped into intervals. Let xi be the
centre of the i-th interval and fi the number of observations in the
i-th interval.
The approach to calculating the sample mean and variance is the
same as in the case of discrete data. In order to carry out the
calculations, we assume that each observation is in the middle of
the appropriate interval.
67 / 69
Example 1.4
Consider the grouped data from Example 1.2
Height (x)
150 ≤ x ≤ 160
160 < x ≤ 170
170 < x ≤ 180
180 < x ≤ 190
190 < x ≤ 200
P
Thus, the sample mean is x =
xi
155
165
175
185
195
-
3480
20
fi
2
5
7
5
1
20
fi x i
310
825
1225
925
195
3480
= 174.
68 / 69
Example 1.4
Now we can add the remaining 2 columns of the table.
xi
155
165
175
185
195
P
x
fi
2
5
7
5
1
20
3480
20 = 174
fi x i
310
825
1225
925
195
3480
(xi − x)2
192
92
1
112
212
2
sn−1
fi (xi − x)2
2×192 = 722
5×92 = 405
7
2
5 × 11 = 605
212 = 441
2180
2180
19 = 114.74
2
The variance is sn−1
= 2180
19 ≈ 114.74.
√
The standard deviation is sn−1 ≈ 114.74 ≈ 10.71.
69 / 69