Download Document

Data description
Peter Shaw
One variable or many?
In your research you will almost certainly end up measuring many
different things:
‘Survey the plants’ means collecting 10-50 columns of data
‘Analyse the soil’ means 5-10 variables
‘take body measurements’ means 5-30 variables.
This lecture is essentially about how to explore each of those variables,
one by one, to tell a reader about the range or distribution of values it
contains. This tells a reader about how important the variable is and
what sort of tests may be run on it (P or N-P?).
But this does not treat your dataset as a unified object.
There is a powerful branch of data description called Ordination, which
is essentially asking for a description of ALL variables at the same time.
Things to do with
data:

This is an infinite morass of statistical techniques, but one
fundamental division is paramount and must be understood.


DESCRIPTIVE <---------------------> INFERENTIAL


Descriptive statistics aim to condense out the useful/important
essence of a (usually large) body of data.

Calculate an average, plot a graph showing the range of values etc.


Inferential statistics requires that the user sets up a formal
hypothesis, then invokes a procedure which ends up with a
probability value by which the hypothesis may be judged.
Why bother with data
descriptions?
Standard format:
Abstract
Introduction
Methods
Results
Discussion
References
I have lost count of the number of
times that students have got this
far then dived straight into the
fancier analyses – Correlations or
Anovas usually, without bothering
to tell the reader anything about
the data they are analysing.
Standard format:
Abstract
Introduction
Methods
Results
Discussion
References
1: Describe your data: units,
indications of typical values +
variability.
2: Analyse relationships within
your data
Pb, ppm in white
General ground rules:
Paint exposed
on a nursery door

16207

14833

29524

18436
26236
1: What do the data mean,
What are the units?
2: Eyeball it!
3: Summon up the formal
procedures, by PC or calculator
Graphing data

This is a huge topic, entire
books have been written.
One unifying point:

A good graph is the best
way to present data.
I am going to show you several
histograms today.
These show the distribution of
values within a dataset.
Number of observations
Size of value
What could you want to
know about a dataset?





In order of decreasing likelihood:
What magnitude of numbers are
you dealing with?
What sort of spread have you got?
What is the nature of the
distribution of results?
?other? (your turn!)
Magnitude summaries



In plain English, what sort of values am I
dealing with?
In statspeak, you require Measures of
Central Tendency
There are 3 such measures you need to
learn, of which 2 are actually useful!



Mean
Median
Mode
Mode

Pb, ppm in white
Paint exposed
on a nursery door
16207
14833
29524
18436
26236

This is simply the commonest
occurrence in the data. Most real
datasets don’t have a mode, as all
values are different.
As such, the Mode is easily the
least useful technique for data
description, but is always
mentioned in the books so you
may as well learn it!
Median
Pb, ppm in white
Paint exposed

on a nursery door
14833
16207
18436 median

This is the middle of the dataset, defined
as the point below which half the data
points lie, and above which half the data
lie.
How to find it:

26236
29524


Sort data into ascending order 1..N
If N is odd, median is the (N+1)/2th value
If N is even, median is half way between
(N/2)th and ((N/2)+1)th value
Median, contd



The median is an under-rated tool, often
preferable to the more widely used
mean, because it gives a sensible
answer whatever the shape of data
distribution
It is a special case of a more general
descriptive technique known as centiles.
The median is the 50th centile of a
dataset, meaning that 50% of the data
points lie below it.
The Mean





The ‘Mean’ is the name given by
statisticians to what everyone else calls
the ‘average’! Often given symbol μ.
Easy to calculate: add up the numbers
and divide by N
μ=Σx/N
Your calculator should have this built in
as a stats function
It is often NOT the middle of the data.
This happens when data are
asymmetrically distributed
Number of observations
A symmetrical distribution
Size of value
Mean and median
about the same
Mean
Size of value
Median
An asymmetrical distribution.
Note that the mean is
misleading here
Number of observations
Two data sets.
Distribution A
Size of value
Number of observations
Distribution B
Size of value
In which one are you more
likely to guess the next
value correctly?
This leads onto..
Measures of dispersion


These are indicators of how tightly
clumped data are.
There is a proliferation of such
indices, but they divide into 2
families:
Non-parametric, based on centiles
 Parametric, based on variance and
giving rise to standard deviations
etc.

UNSORTED
More paint Pb
Data, ppm
2734
3404
5000
4641
16207
14833
1515
1667
29524
18436
26236
7255
5800
10588
9462
6368
5122
6585
6846
4143
Centiles



SORTED
Paint Pb Data, ppm
1: 1515
2: 1667
2734
1: sort data into ascending 3:
4: 3404
order (this is a PC job for big 5: 4143
***25th centile here
datasets)
6: 4641
2: To get the 25th centile, find 7: 5000
8: 5122
the number below which
9: 5800
10: 6368
25% of the data lie
*** 50th centile =
median here
3: To get the 75th centile, find 11: 6585
12: 6846
the number below which
13: 7255
75% of the data lie, etc
14: 9462
15: 10588
***75th centile here
16: 14833
17: 16207
18: 18436
19: 26236
20: 29524
The inter-quartile range


Number of observations
25th
50th
75th
Is the difference between the 25%
and 75% centiles of a distribution.
This means that is is the range
covered by the middle half of the
data.
Size of value
Interquartile range
Boxplots

These are underrated, but extremely
helpful tools for
examining the
distribution of data.
100
Highest value
75th centile
50
median
25th centile
Lowest value
0
Standard deviations and
all that.. The parametric family of measures of

1000
X1
500
X3
Mean


dispersion have messy-looking formulae,
but luckily are easily obtained from
calculators or PCs
They are based on a measure
misleadingly called the sum of squares
of the data (SS).
The origin of SS is as follows:

For each data point Xi calculate
(Xi - mean)*(Xi - mean) [This square >=0 ]

Add up all these squares = SS

Formula: SS = Σi(xi - μ)2

X2
0

Luckily there is a simpler(?) formula

SS = Σi(xi2) – (Σixi * Σixi) /N
Variance etc


Having got the Sum of Squares
Variance is the mean value of SS


Variance = SS/N
(an alternative formula also used:



Geographers tend to prefer


Variance = SS / (N-1)
This estimates the variance of the whole
population, while /N gives variance just for
the sample taken.
Variance = SS/N
Biologists tend to prefer

Variance = SS/(N-1)
Standard deviation




Is the square root of variance
This has the useful property that sd has the same
units as the raw data and will be commensurate
with the interquartile range. (Roughly, for typical
data, the IQR= 2* sd)
Because there are 2 ways to calculate variance,
there are 2 s.d.s
Sd = (SS/N)1/2. This is labelled σ on many calculators
or
Sd = (SS/(N-1))1/2. This is labelled s on many calculators
How to use your
calculator

Why is he telling me this – I
already know?!
OK then, what’s this hierarchy?
+- < */ < Yx < ()

Use this to calculate





123*456+789*112
(109*256+103*876)/(22*44+89*78)
The solution to the ‘grains of rice on a
chessboard’ problem is 264-1

(ie 2*2*2………..….*2 –1), which is?
Stats mode
on your
calculator

If you have buttons saying N, Σx, sd then
your machine has stats functions

This means it has special registers called N, Σx and
Σx2, which keep running totals as you enter data.







Put into stat mode
Enter the number 7 by hitting the Σ or M+ or Xi
button
Optional, but for your education find the contents of
the special registers (Kout or recall)
N =1, Σx = 7, Σx2 = 49
Enter the number 2
Now you find that N =2, Σx = 9, Σx2 = 53
Now the Mean button will give you the mean 4.5, the
sd buttons the sds (2.5 sd/n, 3.53.. /n-1). Easy!














Water
content of
heathland
soils, %
8.53
17.53
39.14
32.00
20.53
21.07
26.20
23.80
12.53
20.80
31.33
28.87
14.00
Your turn!
For the numbers listed here
Find mean, median, mode,
and interquartile range
Find both standard deviations, by your
calculator’s inbuilt functions or by the
formulae:
SS = Σi(xi2) – (Σixi * Σixi) /N
Then
Sd = (SS/N)1/2.

or
Sd = (SS/(N-1))1/2
Distribution shape
20
10
Often real data don’t follow the
Normal curve but are skewed – here
organic content in heath soils
Std. De v = 27.97
Mean = 29.3
N = 69. 00
0
5.0
15.0
10.0
25.0
20.0
35.0
30.0
45.0
40.0
55.0
50.0
65.0
60.0
75.0
70.0
85.0
80.0
90.0
LOI
12
Try log-transforming the data.
Here the same data after
calculating log of the numbers –
not perfect, but clearly more
symmetrical
10
8
6
4
2
Std. Dev = .44
Mean = 1.26
N = 69.00
0
.63
.88
.75
LOGLOI
1.13
1.00
1.38
1.25
1.63
1.50
1.88
1.75
Normal P-P Plot of LOI
1.00
How to decide abou
normality?
.75
.50

.25
0.00
0.00
.25
.50
.75
1.00

Observed Cum Prob
Normal P-P Plot of LOGLOI
1.00

.75
.50
.25
0.00
0.00
.25
.50
.75
1.00
Inspect histogram + fitted
normal curve.
Inspect a cumulative “P-P
curve” with predicted norma
distribution
Run the KolgomorovSmirnov test
The Kolmogorov-Smirnov test
examines whether data can be
assumed to come from a chosen
distribution – here the normal.

One-Sample Kolmogorov-Smirnov Test
LOI
N
Normal Parametersa, b
Mos t Extreme
Differences
Mean
Std. Deviation
Abs olute
Pos itive
Negative
Kolmogorov-Smirnov Z
Asymp. Sig. (2-tailed)
69
29.2806
27.9695
.217
.217
-.183
1.804
.003
LOGLOI
69
1.2603
.4409
.086
.080
-.086
.716
.685
a. Tes t dis tribution is Normal.
b. Calculated from data.
LOI is almost certainly NOT
normally distributed
LogLOI may or may not
be normal, but the test
tells us that its deviations
from normality would
occur 7 times in 10 in
randomly chosen normal
data
Kolmogorov test in
SPSS
Typical SPSS – does the same test in 2 ways in different bits of menu
structure and uses different algorithms to assess significance. I use the
basic version
Analyse – non parametric stats – 1 sample KS
But it also hides under
Analyses – descriptive statistics – explore – plots then click the box
labelled “Normality plots with tests”. This well-hidden version uses a
modified significance test (Lilliefor’s correction), which really threw me
the first time I met it!