Download Psychometrics

DEPARTMENT OF INDUSTRIAL
PSYCHOLOGY
IPS 232
PSYCHOMETRICS
2019
W E E K 3 - BA S I C M E A S U R E M E N T A N D S C A L I N G C O N C E P T S
CHAPTER 3
Dr. Sanet van der Westhuizen
Ms Amirah Davids
RECAPPING LAST WEEK
“A psychological test is essentially an objective and standardized measure of a
sample of behavior.” [Anastasi & Urbina, 1997, p. 4]
“Psychological tests are evaluation and assessment procedures that have the
specific purpose of determining people’s characteristics [] in the fields of
mental ability, aptitude, interests, personality composition and personality
functioning. They comprise a collection of tasks or questions or items that are
aimed at eliciting a specific type of behaviour under standard circumstances,
on the basis of which scores with acceptable psychometric characteristics such
as satisfactory validity and reliability coefficients can be deduced according
to prescribed procedures.” [Owen & Taljaard, 1988, p. 446]
OPERATIONALIZATION
▪Most of the construct/variables are latent – methodology to assign number to
the characteristics
▪Measurement is possible when and because there is a certain degree of
isomorphism between the attribute to be measured and numbers; i.e. because
there is a similarity between the characteristics of numbers/the numerical
system on the one hand, and the characteristics of the attribute to which these
numbers are assigned, on the other hand. [see the Krantz et al. definition of
measurement].
▪The numerical system/numbers/figures can thus be used as a model to
describe and replace the attribute to be measured
QUOTATION FROM GUILFORD AND FRUCHTER [1978, P. 98]
Although it should be seen in a somewhat broader context, the following quotation
from Guilford and Fruchter [1978, p. 98] summarizes the above argument very well :
Mathematics exists entirely in the realm of ideas. It is a logic-based system of elements
and relationships, all of which are precisely defined. It is a completely logical language
that can be applied to the description of nature because the events and objects have
properties that provide a sufficient parallel to mathematical ideas. There is isomorphism
[similarity of form] between mathematical ideas and phenomena of nature. Even if the
description of nature in mathematical terms is never completely exact, there is enough
agreement between the forms of nature and the forms of mathematical expression to
make the description acceptable. Once we have applied the mathematical description,
we can follow where the mathematical logic leads and come out with deductions that
also apply to nature.
THREE PROPERTIES OF MEASUREMENT SCALES
Magnitude – the property of “moreness”.
Equal intervals – the differences between all points on the scale are uniform
and continuous data is generated. Where the difference between points on a
scale is not the same, categorical data is generated.
Absolute zero – there is nothing present of the attribute being measured.
With many human attributes it is difficult, if not impossible, to define an
absolute zero point.
BASIC PRINCIPLES OF MEASUREMENT
Equal Intervals
 A scale possesses the property of equal intervals if the difference between all
points on the scale is uniform.
 Eg. Difference between 6 & 8 centimetres on a ruler is the same as 10 & 12
centimetres . Difference is exactly 2.
 Example A represents an equal interval rating scale. Interval between scores
equal 1. Generates continuous data.
 Example B numbers are assigned different categories & are therefore not
equal. Generates categorical data.
6
BASIC PRINCIPLES OF MEASUREMENT
Absolute Zero
 Is obtained when there is absolutely nothing of the attribute being measured. Complete absence
of what is measured.
 Eg. Length – zero centimetres means there is no distance.
 Length therefore possesses property of absolute zero.
 With human attributes it is extremely difficult if not impossible to define an absolute zero point.
 Eg. Zero Personality. No such thing as zero personality
7
CATEGORIES OF MEASUREMENT LEVELS
Language
Two broad levels of measurement are found namely:
Afrikaans
1
English
2
IsiNdebele
3
IsiXhosa
4
 Nominal – numbers are assigned to attribute to describe or name it, rather than
IsiZulu
5
to indicate its rank or magnitude.
 Eg. In SA, Eleven official languages coded 1 -11,
Sepedi
6
Sesotho
7
Setswana
8
SiSwati
9
Tshivenda
10
Xitsonga
11
 Categorical measures – produce data in the form of discrete/distinct categories (eg.
Males & females or Black, Coloured, Indian, White).
 These can be divided into nominal & ordinal measurement scales.
 Ordinal – numbers are assigned to reflect some sort of sequential ordering or
amounts of an attribute.
 Eg. Athletes ranked in order of how they finished in a race. Example B would
represent an ordinal level of measurement.
8
CONTINUOUS DATA
Produces data which have been measured on a continuum which can be broken down into
smaller units.
❖Interval – equal numerical differences can be interpreted as corresponding to equal
differences in the characteristic being measured
❖Ratio – not only can equal difference be interpreted as reflecting equal differences in the
characteristic being measured, but
- There is a true (absolute) zero which indicates complete absence of what is being measured.
- The inclusion of a true zero allows for the meaningful interpretation of numerical ratios.
MEASUREMENT ERRORS
Two broad types of errors may occur:
Random sampling errors - These can be defined as: “a statistical fluctuation that
occurs because of chance variations in the elements selected for a sample” (Zikmund,
Babin, Carr, & Griffin, 2010, p. 188).
Systematic errors - Systematic errors or non-sampling errors are defined as: “error
resulting from some imperfect aspect of the research design that causes respondent
error or from a mistake in the execution of the research” (Zikmund et al., 2010, p.
189).
MEASUREMENT SCALES
Scaling Options
1. Category scales
Psychometrics
2. Likert-type scales
- Deals with measurement of abstract/
intangible constructs.
3. Semantic differential scales
- Need to measure in ways they manifest
themselves.
5. Constant-sum scales
4. Intensity scales
6. Graphic rating scales
7. Forced-choice scales
8. Guttman scales
SEMANTIC DIFFERENTIAL SCALE
CONSTANT SUM SCALES
GUTTMAN SCALE
CONSIDERATIONS IN DESIGNING A MEASUREMENT SCALE:
1. Will the scale measure one dimension or many (multi) dimensions or facets
of a construct?
2. Will the item be formulated as a question or a statement?
3. Will all the response labels have descriptors or only the extreme
categories?
4. Will the scale measure one construct or many constructs?
5. How many response categories are needed?
6. Will the scale lead to ipsative or normative comparisons?
BASIC STATISTICAL CONCEPTS
Assessment measures & survey instruments often produce data in the form of numbers.
We need to make sense of these numbers.
BSC help to establish and interpret norm scores.
1. Displaying data - Data can be depicted graphically.
2. Measures of central tendency - The centre of the distribution can be determined by
calculating the mean, median, and mode.
3. Measures of variability - The distribution of the scores around the mean can be computed
(range and standard deviation)
4. Measures of association - Two sets of scores can be correlated and performance on the one
can be predicted from the other.
SUMMATION NOTATION ( )
Summation notation means to sum or add up. So whenever you see the
summation notation symbol, you need to add up.
For all the computations you need to do later on, you will only substitute
the values in the summation table in the different formulas. So make
sure that you work very accurately and that all your calculations are
correct.
X = SCORE ON ASSIGNMENT QUESTION
Y = SCORE ON EXAMINATION QUESTION
Square the x scores from column one, square the y scores from column 2 and multiply the
x and y scores in these last three columns.
Student
A
B
C
D
E
F
G
H
I
J
N=
X
1
2
2
2
3
4
5
7
7
8
ΣX =
Y
5
4
5
6
8
7
8
7
8
8
ΣY =
X
2
4
9
16
25
49
49
64
ΣX 2 =
Y
2
25
16
25
64
49
64
64
ΣY 2 =
Add up all these columns as the summation
notation indicates
XY
5
8
10
12
24
28
ΣXY =
THE COMPLETED SUMMATION NOTATION OF
TABLE 1
Student
A
B
C
D
E
F
G
H
I
J
N = 10
X
1
2
2
2
3
4
5
7
7
8
X = 41
Y
5
4
5
6
8
7
8
7
8
8
Y = 66
X²
1
4
4
4
9
16
25
49
49
64
X² = 225
Y²
XY
25
5
16
8
25
10
36
12
64
24
49
28
64
40
49
49
64
56
64
64
Y² = 456 XY = 296
MAKE SURE THAT YOU KNOW HOW TO CALCULATE THE FOLLOWING
NOTATIONS AS WELL:
(ΣX) 2 =
(ΣY) 2 =
(ΣX)(ΣY) =
These would be:
(X)² = (41)² = 1 681
(Y)² = (66)² = 4 356
(X)(Y) = (41)(66) = 2 706
MEASURES OF CENTRAL TENDENCY
The first way in which we can describe a dataset is by means of central tendency.
This is where most of the scores are ‘centered’.
We will look at three measures of central tendency, namely mode, median and mean.
We will be working with the first dataset of assignment scores that was given earlier.
MEASURES OF CENTRAL TENDENCY:
MODE
The symbol for the mode is:
Mo =
The formula for the mode is:
Mo = Most frequently occurring score
So this would be the score with the highest frequency in a dataset. For the
assignment scores it would be:
1
2
2
 Mo = 2
2
3
4
5
7
7
8
MEASURES OF CENTRAL TENDENCY: MEDIAN
In order to find the median, we first need to determine the
median location.
The formula for the median location is:
Median location = N + 1
2
= 11
2
= 5,5
MEDIAN
Next we need to arrange all of the data points in numerical order. Then
count to the median location, at which the median would be.
So for the assignment scores again:
1
2
2
2
3
4 5 7

5,5 position/posisie
7
8
When the median location is between two scores, add up the scores and
divide it by 2. In this case: 3+4/2 =
 Median = 3,5
MEASURES OF CENTRAL TENDENCY:
MEAN
The symbol for the mean is:
The formula for the mean is:
X=
X

X =
N
MEASURES OF CENTRAL TENDENCY:
MEAN EXAMPLE
The calculation for the mean is:
(Use the table to substitute the correct values in the formula)
X

X =
N
41
X =
10
= 4,1
MEASURES OF CENTRAL TENDENCY: MEAN
PRACTICE
CALCULATE THE MEAN EXAMINATION SCORE (Y)
Student
A
B
C
D
E
F
G
H
I
J
N = 10
X
1
2
2
2
3
4
5
7
7
8
X = 41
Y
5
4
5
6
8
7
8
7
8
8
Y = 66
X²
1
4
4
4
9
16
25
49
49
64
X² = 225
Y²
XY
25
5
16
8
25
10
36
12
64
24
49
28
64
40
49
49
64
56
64
64
Y² = 456 XY = 296
MEASURES OF CENTRAL TENDENCY: MEAN
PRACTICE
Average Examination score = ∑Y/N
= 66/10
= 6,6
MEASURES OF VARIABILITY
Apart from using measures of central tendency, one can also look at
measures of variability.
Three datasets can have similar means, but look significantly different in
terms of their variability (how widely scores are distributed around the
mean).
VARIANCE
If one is interested in the deviation of individual scores around the mean- the most logical
thing to do is to work out the deviation of scores from the mean;
(X − X )
Deviations from the mean are both positive and negative and ultimately cancel each other out
– thus deviations collectively equate to zero
To remedy the problem- square deviations (s2)
Definitional formula:
( X − X ) 2
s =
N −1
2
Example
 See next slide
VARIANCE EXAMPLE – FORMULA IN THE TEXTBOOK IS WRONG!!!!!

Calculation
X
2 4 5 8 7 4
3 2 -1
0
( X − X )2 9 1 0 9 4 1
24
X − X.
-3 -1 0
30
( X − X )
24
s =
=
= 4.80
N −1
5
2
2
MEASURES OF VARIABILITY: VARIANCE
THE FORMULA FOR THE VARIANCE IS:
s
2
( x )
x −

2
N
sx =
N −1
YOU CAN AGAIN JUST SUBSTITUTE ALL THE
VALUES FROM THE TABLE:
2
2
MEASURES OF VARIABILITY: VARIANCE
PRACTICE
CALCULATE THE VARIANCE OF THE ASSIGNMENT SCORES (X)
Student
A
B
C
D
E
F
G
H
I
J
N = 10
X
1
2
2
2
3
4
5
7
7
8
X = 41
Y
5
4
5
6
8
7
8
7
8
8
Y = 66
X²
1
4
4
4
9
16
25
49
49
64
X² = 225
Y²
XY
25
5
16
8
25
10
36
12
64
24
49
28
64
40
49
49
64
56
64
64
Y² = 456 XY = 296
VARIANCE OF ASSIGNMENT SCORES
Student
A
B
C
D
E
F
G
H
I
J
N = 10
X
1
2
2
2
3
4
5
7
7
8
X = 41
Y
5
4
5
6
8
7
8
7
8
8
Y = 66
X²
1
4
4
4
9
16
25
49
49
64
X² = 225
Y²
XY
25
5
16
8
25
10
36
12
64
24
49
28
64
40
49
49
64
56
64
64
Y² = 456 XY = 296
2
(
x
)

2
x
−

2
N
sx =
N −1
( 41) 2
225 −
10
s x2 =
10 − 1
= 6,32
THE VARIANCE OF A SINGLE ITEM
▪We want each item in a test to contribute information about the
individuals we test (i.e. discriminate between individuals with different
inherent standings on the to-be-measured construct)
▪Items to which everybody responds the same contributes no
information
▪Generally, items with larger variances are more desirable
▪The standard deviation of an item is the square root of the variance
36
STANDARD DEVIATION
Standard deviation is the square root of variance - average deviation of each score from the mean
Definitional formula:
( X − X )
s= s =
N −1
2
2
▪Normal distribution- 66.6% of scores are distributed one standard deviation above and below the mean
▪95% if scores are distributed between 1.96 standard deviations above and below the mean
EXAMPLE
2

X
)
2 (
2
2
2
2
2
2 30
X −
2 + 4 +5 +8 +7 + 4 −
N =
6 = 4.80
s2 =
N −1
5
2
s=
X
2
X )
(
−
N
N −1
2
= 4.80 = 2.19
STANDARD DEVIATION
PRACTICE
Calculate the standard deviation of the
Assignment scores.
STANDARD DEVIATION OF ASSIGNMENT
SCORES
The formula for the standard deviation is:
s=
=
=
s2x
6,32
2,51
CORRELATION
Relationship between two variables
Deduction about nature or direction (positive or negative)
Deduction about strength (-1 to 1)
Symbol for correlation is r.
Formula is:
r=
NXY − XY
[ NX 2 − (X ) 2 ][ NY 2 − (Y ) 2 ]
CORRELATION FORMULA
CORRELATION FORMULA IN WORDS
Correlation = sum of (∑) odd and even items multiplied – (1/items) multiplied by
(sum of odd) multiplied by (sum of even)
Divided by square root of:
(sum of squared odd items) minus (1/items) multiplied by (sum of odd items squared)
MULTIPLIED by
(sum of squared even items) minus (1/items) multiplied by (sum of even items squared)
10 because we work with the
total score of odd and even
numbers
CORRELATION BETWEEN
ODD AND EVEN ITEMS
𝑥=
5*6
5+6
62 + 65
20
6.35
Applicant
Personality Measure Test X scores
Score for
odd items
(x1)
Sam
Bongi
Cathy
Bilqees
Eli
Funeka
John
Harry
Ingrid
Joe
5
8
7
8
7
6
5
8
6
2
X12
Score for
even items
(x2)
X22
Product of
items x1x2
25
64
49
64
49
36
25
64
36
4
416
6
10
7
9
6
8
4
7
5
3
65
36
100
49
81
36
64
16
49
25
9
465
30
80
49
72
42
48
20
56
30
6
433
Cognitive Ability Measure Test Z scores
Total score
(X)
Score for
odd items
(z1)
11
18
14
17
13
14
9
15
11
5
127
4
4
6
5
2
4
3
7
2
4
41
Product XZ
Z12
Score for
even items
(z2)
Z22
Product of
items z1z2
Total score
(Z)
16
16
36
25
4
16
9
49
4
16
191
3
4
4
3
4
5
3
2
6
2
36
9
16
16
9
16
25
9
4
36
4
144
12
16
24
15
8
20
9
14
12
8
138
7
8
10
8
6
9
6
9
8
6
77
See next slide
for calculation
38
72
70
67
38
64
27
70
42
14
502
62
4030
1476
127*77
9779
44
CORRELATION
0.81
45
𝑛
𝑗 =1
CORRELATION BETWEEN X AND Z
𝑛
2
𝑗 =1 𝑋𝐽
416 +
465
Applicant
Personality Measure Test X scores
Score for
odd i tems
(x 1 )
Sam
Bongi
Cathy
Bilqees
Eli
Funeka
John
Harry
Ingrid
Joe
5
8
7
8
7
6
5
8
6
2
=
𝑋𝐽 𝑍𝐽 −
− 1/𝑛[
1
[
𝑛
𝑛
𝑗 =1
𝑛
𝑗 =1
𝑋𝐽 ]2 ][
𝑋𝐽 ] [
𝑛
𝑗 =1
𝑛
𝑗 =1
𝑍𝐽 ]
𝑍𝐽2 − 1/𝑛[
1
502 − 20 [127][77]
1
1
[881 − 20 1272 [335 − 20 772 ]
Product
XZ
Cognitive Ability Measure Test Z scores
X1 2
Score for
even i tems
(x 2 )
X2 2
Product of
i tems x 1 x 2
Total
s core (X)
Score for
odd i tems
(z1 )
Z1 2
Score for
even i tems
(z2 )
Z2 2
Product of
i tems z1 z2
Total
s core (Z)
25
64
49
64
49
36
25
64
36
4
416
6
10
7
9
6
8
4
7
5
3
65
36
100
49
81
36
64
16
49
25
9
465
30
80
49
72
42
48
20
56
30
6
433
11
18
14
17
13
14
9
15
11
5
127
4
4
6
5
2
4
3
7
2
4
41
16
16
36
25
4
16
9
49
4
16
191
3
4
4
3
4
5
3
2
6
2
36
9
16
16
9
16
25
9
4
36
4
144
12
16
24
15
8
20
9
14
12
8
138
7
8
10
8
6
9
6
9
8
6
77
38
72
70
67
38
64
27
70
42
14
502
62
4030
=
1476
9779
502 − 488.95
[881 − 806.45 [335 − 296.45 ]
= .24
𝑛
2
𝑗 =1 𝑍𝐽 ]
46
𝑛
1
𝑋𝐽 𝑍𝐽 − 𝑛 [
𝑗=1
𝑛
2
𝑗=1 𝑋𝐽
ITEM NUMBER
− 1Τ𝑛 [
𝑛
𝑋
𝑗=1 𝐽
]
𝑛
𝑗=1 𝑋𝐽 ]
2
][
[
𝑛
2
𝑗=1 𝑍𝐽
X Odd and Even Numbers
How to calculate
Product of X and Z:
X*Z
Example:
5*4 = 20
𝑛
𝑍]
𝑗=1 𝐽
− 1Τ𝑛 [
x²
𝑛
𝑗=1 𝑍𝐽 ]
2
Z Odd and Even Numbers
z²
Product of X and Z
Sam Odd Items
5
25
4
16
20
Bongi Odd Items
8
64
4
16
32
Cathy Odd Items
7
49
6
36
42
Bilqees Odd Items
8
64
5
25
40
Eli Odd Items
7
49
2
4
14
Funeka Odd Items
6
36
4
16
24
John Odd Items
5
25
3
9
15
Harry Odd Items
8
64
7
49
56
Ingrid Odd Items
6
36
2
4
12
Joe Odd Items
2
4
4
16
8
Sam Even Items
6
36
3
9
18
Bongi Even Items
10
100
4
16
40
Cathy Even Items
7
49
4
16
28
Bilqees Even Items
9
81
3
9
27
Eli Even Items
6
36
4
16
24
Funeka Even Items
8
64
5
25
40
John Even Items
4
16
3
9
12
Harry Even Items
7
49
2
4
14
Ingrid Even Items
5
25
6
36
30
Joe Even Items
3
9
2
4
6
Total: 127
Total: 881
Total: 77
Total: 335
Total: 502
PRACTICE
CALCULATE THE CORRELATION COEFFICIENT BETWEEN X AND Y
Student
A
B
C
D
E
F
G
H
I
J
N = 10
X
1
2
2
2
3
4
5
7
7
8
X = 41
r=
Y
5
4
5
6
8
7
8
7
8
8
Y = 66
X²
1
4
4
4
9
16
25
49
49
64
X² = 225
Y²
XY
25
5
16
8
25
10
36
12
64
24
49
28
64
40
49
49
64
56
64
64
Y² = 456 XY = 296
NXY − XY
[ NX 2 − (X ) 2 ][ NY 2 − (Y ) 2 ]
IS THERE A RELATIONSHIP BETWEEN HOW WELL STUDENTS DO
IN THEIR ASSIGNMENTS AND HOW WELL THEY DO IN THEIR
EXAMINATIONS?
r=
r=
NXY − XY
[ NX 2 − (X ) 2 ][ NY 2 − (Y ) 2 ]
10(296) − (41)(66)
[10(225) − (41) 2 ][10(456) − (66) 2 ]
r = 0,75
NORMS
What is a norm?
Standard normal distribution
How are norms created?
What is co-norming? Practical benefits of co-norming?
Different types of test norms:
Interrelationships among different types of norm scores.
How to set or determine cut-off scores to compare performance to an external criterion or
standard.
WHAT IS A NORM?
 A norm can … be defined as a measurement against which an individual's
raw score is evaluated so that the individual's position relative to that of the
normative sample can be determined.
STANDARD NORMAL DISTRIBUTION
 Characteristics measured in psychology that are assumed to be normally
distributed.
STANDARD NORMAL DISTRIBUTION
ESTABLISHING NORM GROUPS
1. Choose a norm group and identify the sub-groups from the norm group that must
be represented.
2. Administer the test to a representative sample from the norm group.
3. Standardised or normal scores are then calculated.
4. Test-taker’s score is then compared to the standardised or normal scores.
✓ Norm groups – consist of 2 subgroups; applicant pool and incumbent population
✓ Choice of norm group – representative of both sub groups.
CO-NORMING
• Co-norming entails the process where two or more related, but different
measures are administered and standardised as a unit on the same norm
group.
•E.g. Consensus Cognitive Battery – National Institute of Mental Health
initiative.
•Goal is to enhance interventions for schizophrenia
TYPES OF TEST NORMS
1. Developmental scales
i.
ii.
Mental Age scales – basal age
Grade Equivalents
2. Percentiles – percentage of people in a normative standardized sample who fall below a given
raw score. 50th percentile correspond to the median.
3. Standard Scores
I.
II.
III.
Z-scores – normalized standard scores. Raw score equal to mean= z-score of zero
Linearly transformed scores (to compensate for limited range of scores and negative scores),
Normalised standard scores:
I.
MC Calls T-score (Mean of 50; sd of 10)
II.
Stanine scale (range of 1 to 9; mean of 5; sd of 1.96)
III.
Sten scale (range of 10 scale units; mean of 5.5; sd of 2)
IV. Deviation IQ scale
DIFFERENT NORMS
STANDARDS AND CUT-OFF SCORES
• Instead of finding out how an individual has performed in relation to
others, you can compare performance to an external criterion (standard).
• Referred to as criterion referenced norms
• Eg., The psychometrics board exam passing standard is 70 percent.
Someone who obtains a mark of 60 percent may have performed better than a classmate who
obtained 50 per cent (norm-referenced comparison).
But would fail the examination, as his/her percentage is below that of the 70 per cent standard
(cut-off) that has been set.