Download Cognitive Walkthrough Script

Cognitive Walkthrough Script
Testing
Materials
Needed:
Opening
Statement:
Handout
Directions
for testing
session
1. A copy of the procedure and accompanying
illustrations
2. Engineering blueprints and any other supplemental
materials
3. Stop watch to record the time each participant
takes to evaluate the procedure
4. Pen and writing pad
5. Consent form (If you are including a
video/computer simulation model during the
test, be sure to include that information in the
Consent Form.)
6. Follow-up question sheet for recording responses
7. Satisfaction questionnaire
8. Background questionnaire
9. (Optional) Any video or computer simulation model
which has been developed to aid unfamiliar
participants
(Meet the participant and escort into the room.)
“Before we begin I would like to thank you for agreeing
to help. My name is _________________________
and I will be facilitating this session. Your participation
in this study is very important. Your comments and
insights will help _____________ develop a more
usable maintenance manual. Before we begin I would
like you to read over and sign a consent form. (Allow
participants to keep a copy of the consent form if
requested) Also, can you please fill out this
background questionnaire? This information will help
us better understand who you are.”
1. Consent form
“Before we begin, I would like to remind you of your
right to withdraw from participation at any time.
Should you choose to exercise this right; no
repercussions or questions will be asked.”
“Your participation is also confidential. Any answers or
discussions from this interview will not be associated
with your name. We are collecting data from several
participants and will evaluate the results collectively.”
(If you are going to have the participant watch a
video, etc., this should be done before giving the
participant any further explanation regarding the
actual Cognitive Walkthrough.)
“During this session you will be given a maintenance
task to read. I will not be able to answer any questions
regarding task; however I can clarify wording. I would
like to emphasize this is not a test -- we are evaluating
the procedure, not you. Any questions so far?”
“As you read through the task, pretend you are at
work and try to imagine completing the task name as
it is written. As you “think” through the task name,
please note directly on the procedure areas in which
you would question the step as written. Try to write as
much information as possible as to why you identified
this as a problem. There is additional paper to draw or
write your information if you wish.”
“Do you have any questions before we begin?”
Hand out
1. Hand out
Follow-up
questions
(Record the participant’s “begin” time on their
Follow-up question form.)
Give the participant the procedure / illustration(s).
Have all supplemental materials available on the desk,
including engineering blueprints, writing pad, and pen.
(When the participant is finished reviewing the
procedure, record their “end” time on the Followup question form.
2. Satisfaction Questionnaire
(To avoid any bias from comments by the
evaluator, satisfaction questionnaires should be
given before any discussion of the Cognitive
Walkthrough evaluation.)
(Go through their copy of the procedure with
them to make sure you understand their
notations. Write any clarifying comments in a
different color ink on the procedure. Then sit
with the participant and ask them a set of
prepared questions, recording their responses on
the sheet. As an example …)







Handout
Closing
Statement
What strategies did you use to walk through the
tasks?
How do you visualize the aircraft?
What is the most critical information you look at?
What supplemental information (if any) did you use
and where?
What other resources would have been helpful?
How confident are you that you found all of the
problem areas? (use a 1 (not that confident)
through 5 (very confident) scale)
How easy was it thinking through this task on paper
versus viewing the actual aircraft?
Others…?
2. Background Questionnaire
Your comments and insights were very valuable to us.
Once again I would like to thank you for your time and
participation in our study. (Escort the participant
out)