Download Week 4

Week 4
11/23/07
Five General Item-Writing
Commandments
· 1. Thou shall not
provide opaque
directions to students
regarding how to
respond to your
assessment
instruments.
Five General Item-Writing
Commandments
· 2. Thou shall not
employ ambiguous
statements in your
assessment items.
· Sentences in which
pronouns are used can
frequently be unclear
about the individual or
individuals to whom the
pronouns refers.
Five General Item-Writing
Commandments
·
·
·
·
True or False:
Leaders of developing
nations have tended to
distrust leaders of developed
nations due to their
imperialistic tendencies.
The word their could refer to
the leaders of developing or
developed nations.
Five General Item-Writing
Commandments
·
3. Thou shall not provide
students with unintentional
clues regarding appropriate
responses.
·
Thus, we want you to
intentionally avoid
unintentional clues.
·
Five General Item-Writing
Commandments
·
Inexperienced item writers often
tend to make the correct answer
to a multiple-choice items twice
as long as the incorrect answers.
·
Absolute qualifiers such as never
and always are sometimes used
for the false items in a true false
test.
·
Even uninformed students know
that there are few absolutes in
this world.
·
Five General Item-Writing
Commandments
·
Here is an unintended clue:
·
·
·
·
·
The bird in the story was an…
A. Falcon
B. Hawk
C. Robin
D. Owl
·
Can you guess the correct
answer, even though you did not
read the story?
·
Five General Item-Writing
Commandments
·
4. Thou shall not employ
complex syntax in your
assessment items.
·
Avoid complicated sentence
constructions and, instead, use
very simple sentences.
·
Five General Item-Writing
Commandments
·
5. Thou shall not use vocabulary
that is more advanced than
required.
·
You should eschew obfuscative
verbiage.
·
Binary-Choice Items
· Two options from which to select.
· (True-False, Yes-No, Correct-Incorrect,
Fact-Opinion)
· Good things…so terse, that students can
answer many items in a short time. You
can cover a lot of content.
· Bad thing…50-50 chance.
Item-Writing Guidelines for
Binary-Choice Items
· 1. Phrase items so
that a superficial
analysis by the
student suggests a
wrong answer.
· (Phrase the items to
elicit thoughtfulness)
Item-Writing Guidelines for
Binary-Choice Items
·
2. Rarely use negative
statements, and never use
double negatives.
·
·
Example:
The League of Nations was
not formed immediately after
the conclusion of World Ware
II.
Or
The League of Nations was
formed immediately after the
conclusion of World War II.
·
·
Item-Writing Guidelines for
Binary-Choice Items
·
·
·
·
Example:
The League of Nations was
not formed immediately after
the conclusion of World Ware
II. True
Or
The League of Nations was
formed immediately after the
conclusion of World War II.
False
Item-Writing Guidelines for
Binary-Choice Items
·
3. Include only one concept
in each statement.
·
This is common sense…one
concept at a time please!
Item-Writing Guidelines for
Binary-Choice Items
·
4. Have an approximate
equal number of of items
representing the two
categories being tested.
Item-Writing Guidelines for
Binary-Choice Items
·
5. Keep item length similar
for both categories being
tested.
·
Avoid systematic patterns
wherein long statements are
true and short statements are
false.
Multiple Binary-Choice Items
· A multiple binarychoice item is one in
which a cluster of
items is presented to
students requiring a
binary response to
each of the items in
the cluster.
Multiple Binary-Choice Items
· Rules:
· 1. Separate item clusters
vividly from one cluster
to another.
· 2. Make certain that
each item meshes well
with the cluster’s
stimulus material.
Mode, Median and Mean
· Mode: The mode is the most frequent
piece of data in a frequency distribution.
Mode, Median and Mean
· Median
The median measures the middle position of a
frequency distribution for a group of data. In
order to calculate the median we need to order
our group of data from the lowest (the 1 st
score in our order) to the highest (the last score
in our order). How we calculate the median
depends on whether there is an even or odd
number in your group of data.
Mode, Median and Mean
· Median:
· Even numbers: If the group of data
contains an even number of scores; in
other words, if your group of data
contains 100 scores, for example, then
we calculate the median by adding the
two middle scores together and dividing
them by two.
Mode, Median, Mean
· Median:
· As a result, we must take the middle two
values, namely the 50th and 51st scores,
add them together (for example…58 + 59
= 117) and divide them by 2 (117 ÷ 2 =
58.5). Hence, the median for our group
of data is 58.5 out of 100.
Mode, Median, Mean
· Median:
· Odd numbers: If the group of data
contains an odd number of scores; in
other words, if your group of data
contains 9 scores, for example, then the
median will simply be the middle score,
the score of the 5th student.
Mode, Median, Mean
· The median is considered to be a good
measure of central tendency because it
describes the middle position of a
frequency distribution for a group of
data. However, it has one main
weakness. Like the mode, the median
fails to take into consideration all the
values in the group of data.
Mode, Median, Mean
· Mean
The mean, otherwise called the
arithmetic mean or average, is the most
commonly used measure of central
tendency. In order to calculate the mean,
we simply add up all the values in the
group of data and divide them by the
number of values.
Mode, Median, Mean
· As a result, the mean is much more
effective at measuring the central
position in a frequency distribution
because it takes into account not only the
total number of scores but also their
distance from the central position.
Range
· In descriptive statistics, the range is the length
of the smallest interval which contains all the
data. It is calculated by subtracting the smallest
observations from the greatest and provides an
indication of statistical dispersion.
· It is measured in the same units as the data.
Since it only depends on two of the
observations, it is a poor and weak measure of
dispersion except when the sample size is
large.
Multiple Binary-Choice Items
·
·
·
… Suppose that a dozen of your students
completed a 10 item multiple-choice test that
earned the following number of correct
scores:
5,6,7,7,7,7,8,8,8,8,9,10
9. The median of your students’ scores is 7.5 True or False
10. The mode for the set of scores is 8.0 True or False
11. The range of the students’ scores is 5.0 True or False
12. The median is different than the mean. True or False
Multiple-Choice Items
·
The strength of multiple-choice questions is they can
contain several answers that differ in their relative
correctness, thus, the student can be called upon to
make subtle distinctions among answer options.
·
A weakness is that students need only recognize the
correct answer not generate the correct answer.
Multiple-Choice Items
·
The first-part of a multiple-choice item is referred to as the
item’s stem.
·
The potential answer options are described as item alternatives.
·
Incorrect alternatives are typically referred to as the item’s
distracters.
Item stems can be a direct question or an incomplete statement.
With younger students the direct question approach is preferable. Using
these stems, you can ask the student to select the best answer or the
best answer.
Multiple-Choice Items
· 1. The stem should
consist of a selfcontained question
or problem.
· It should be loaded
with as much content
as possible so that
the student knows
exactly what they are
to consider.
Multiple-Choice Items
· 2. Avoid negatively
stated stems. If you
use the word not in a
multiple-choice
stem, be sure to
highlight it so that
the student sees and
pays attention to this
negative.
Multiple-Choice Items
Which one of the the
following cities is
located west of the
Mississippi? San Diego,
Pittsburgh, Boston or
Atlanta
Which one of the following
cities is not located in a
state east of the
Mississippi? San Diego,
Pittsburgh, Boston or
Atlanta
Multiple-Choice Items
3. Do not let the length of
alternatives supply
unintended clues.
4. Keep it short and sweet
or make all
lengthy…you make the
call.
Multiple-Choice Items
4. Randomly assign
correct answers to
alternate positions.
5. As a rule, if you have
four choices, make
sure the correct answer
is assigned to each
position (a,b,c,or d,)
25% of the time.
Multiple-Choice Items
5. Never use all-ofthe-above
alternative but do
use none of the
above alternatives
to increase item
difficulty.
Matching Items
A matching item consists of two
parallel lists of words or
phrases that require the
student to match entries on
one list with appropriate
entries on the second list.
Entries in the list for which a
match is sought are referred
to as premises.
Entries in the list from which
selections are made are
referred to as responses.
Matching Items
Advantage:
Easily scored
Takes up little space and
taps a good deal of
information efficiently
Disadvantage:
Encourages memorization
of low level factual
information.
Matching Items
· Matching Item-Writing Guidelines
· 1. Employ homogeneous lists
· 2. Use relatively brief lists, placing the shorter
words or phrases at the right.
· 3. Employ more responses than premises.
· 4. Order the responses logically
· 5. Describe the basis for matching and the
number of times responses may be used
Constructed-Response Items
Short answer items and essays
Constructed-Response Items
· Short-Answer Items:
· Students are asked to supply a word, a phrase,
or a sentence in response to either a direct
question or an incomplete statement.
· The major advantage is that the student needs
to produce a correct response answer, not
merely recognize it from a set of selectedresponse options.
· The major disadvantage is that student
responses are difficult to score. The longer the
responses sought, the tougher it is to score
them.
Five Guidelines for Writing
Short Answer Items
· 1. Usually employ direct questions rather than
incomplete statements, particularly for younger
students.
· 2. Structure the item so that a response should be
concise.
· 3. Place blanks in the margin for direct questions or
near the end of incomplete statements.
· 4. For incomplete statements, use only one or, at most,
two blanks.
· 5. Make sure blanks for all items are equal in length.
Essay Items: Development
·
·
Essay items are particularly useful in gauging a student’s ability to
synthesize, evaluate and compose.
For assessing certain types of complex learning outcomes, the
essay item is hands-down the winner.
·
·
Disadvantages:
Essay items are more difficult to write properly--than is generally
thought.
·
Scoring essays can be difficult also.
·
·
Essay Items: writing
guidelines
·
·
·
·
·
·
·
1. Convey to students a clear idea regarding the extensiveness of
the response desired
2. Construct items so that the student’s task is explicitly
described.
3. Provide students with the approximate time to be expended on
each item as well as each item's value
4. Do not employ optional items
5. Precursively judge an item’s quality by composing, mentally or
in writing, a possible response.