Download AMASS 2006 - workshop part 1

Psychological Science on the Internet:
Designing Web-Based Experiments
From the Ground Up
R. Chris Fraley
University of Illinois at Urbana-Champaign
Introduction
•
During the last few years an increasing number of
psychologists have begun to use the Internet to collect
empirical data.
•
The Internet has the potential to be useful for two
reasons:
•
Integrated data management
•
Open laboratory
– Whether you collect data from undergraduate subject
pools, community samples, or elsewhere, you can use the
same platform for research design, collecting data,
providing feedback, and managing data.
– Participants can take part in your research any time of the
day.
– If desired, you can open your lab doors to people from
across the world.
Our plan
•
My objective today is to illustrate some of the research
techniques that can be implemented using the Internet.
•
Outline:
1. What kinds of research can be done using the Internet?
2. How to obtain access to a web server.
3. How to create a basic HTML web page that can be used to
collect data.
4. How to create Perl/CGI scripts to process and save those
data.
5. How to do advanced things, such as randomize
stimuli/questions, randomly assign people to conditions,
and present items/stimuli on separate pages.
6. A discussion on ethics, sampling concerns, and informed
consent.
Part 1
Basic concepts and what can be done
How the Net Works
• Most web pages are written in a language called
HTML (hypertext markup language).
• HTML files exist on a web server. When a person
types in the URL for a specific file in his or her
web browser, the person is essentially retrieving
that pre-existing file from the web server.
How the Net Works
web user
types a
URL into
the browser
web server
web user
the browser
renders the HTML
file as a web page
locates
requested
HTML file and
sends it to
user’s
computer
Static vs. Dynamic Web Pages
• The obvious limitation of using HTML pages in
this manner is that the user’s web experience is
static—he or she will see the same page each
time he or she goes to that address.
• It is possible, however, to program the server to
deliver dynamic content—content that varies
from one situation to the next or from one
person to the next.
• This kind of interactivity is necessary in order to
produce the kinds of web pages that allow for
creative, complex research designs.
Dynamic Web Programming and CGI
• This kind of interactivity can be accomplished by
writing programs, called CGI scripts, that run on
the web server.
• CGI scripts are often written in Perl, PHP, or
other languages. (I’ll expand on some technical
details later.) What is important to note right
now is that those languages are available for free
and, with a bit of practice, are relatively easy to
use.
Dynamic Web Programming and CGI
web user
types a
URL into
the browser
web server
• locates requested
CGI script.
• Performs the
commands and
creates a new HTML
file.
web user
the browser
renders the HTML
file as a web page
• Sends HTML file to
user’s computer
What Can Be Done with CGI Scripts?
• By using CGI scripts, it is possible to accomplish
a number of feats that are impossible using
standard web pages.
–
–
–
–
–
–
–
•
save data
randomize the order of questions
random assignment to conditions
track participants over time
implement complex branching patterns
customize text and images
create customized feedback for users
Let’s discuss some of these points in more detail.
Automatically Save Responses to a Data
File
• One of most time-consuming aspects of
conducting paper-and-pencil research is data
entry.
• It is easy to write CGI scripts that save the data
automatically.
– save as a comma-delimited text file (that can be
easily imported into SPSS or MS Excel)
– save directly into a database (such as MySQL)
snapshot of a comma-delimited text file
Provide Participants with Customized
Feedback
• One of the most valuable features of conducting
research over the Internet is that it gives you a
means to provide instant feedback to the
research subject.
– plot the person’s scores
– create bar charts or tables that summarize the data
(e.g., means, SDs, correlations) from the broader
sample automatically
Substitution
• One of the perks of using dynamic programming
is that it allows for substitution.
• In other words, you can take information that
has been provided by the user (or computed by
some other part of the script) and substitute that
into the web page that is sent to the user.
My wife’s name has been
substituted into the web
page, making the questions
tailored to me.
Randomization of Stimuli Order
• In most questionnaire-based research, it is not
unusual for researchers to present questions in
the same order for each research participant.
• In fact, many researchers who have placed their
questionnaires on-line have followed this
practice.
Randomization of Stimuli Order
• There are obvious limitations of using the same
presentation order for stimuli.
• Most importantly, there may be systematic order
effects that impact responding.
• By randomizing the order in which items/stimuli
are presented, it is possible to minimize these
kinds of problems.
• It is possible to randomize the order of stimuli
quite easily using a CGI script.
Random Assignment to Conditions
• It is also possible to use these same
randomization techniques to randomly assign
participants to experimental conditions.
• Importantly, it is easy to implement this in a way
that precludes research subjects from knowing
that they are in but one of many possible cells of
a research design.
• Example:
• http://www.web-research-design.net/cgibin/APS2006/experiment1.pl
Measure Response Times
• It is also possible to write CGI scripts that assess
the amount of time that a user spends on a web
page.
Measure Response Times
• It is not advisable to assess response times for
decisions that can be made within a few seconds
(e.g., lexical decision tasks) because it can take
up to three seconds for a simple HTML page to
fully load.
• However, by using multimedia applications, such
as Macromedia’s Flash, it is possible to assess
response times at a more microscopic level.
Measure Response Times
• Example lexical decision application
• http://www.web-researchdesign.net/APS2006/lexicaldecision.swf
Use Complex Branching Patterns
• One limitation of paper-and-pencil
questionnaires is that it is difficult for
participants to follow complex branching
operations.
– There may be certain questions that should be skipped if
the participant is not married, for example.
– If you want to tailor the stimuli to a specific user (e.g., a
person’s ideographically unwanted traits), it is difficult to
do so with a paper-based test.
• Again, CGI scripts can be used to implement
these conditional patterns.
Track Participants Over Time
• It is often desirable to follow a group of people
across time to study continuity and change in
attitudes, moods, behavior, and traits.
• The Internet provides a useful way to track data
across time because people can
– (a) log-on at their convenience
– (b) all submissions can be time-stamped
automatically by the web server
– (c) the person’s progress can be easily tracked
across multiple sessions
Learning How to Conduct Research over the
Internet
• How do you do this stuff?!
Learning How to Conduct Research over the
Internet
• The first thing to note is that these techniques
are relatively easy to learn—even if you have
very little experience in programming.
Learning How to Conduct Research over the
Internet
• What you need:
– a computer with an Internet connection (dial-up
modem or high-speed)
– a web server
• Professional hosting sites (e.g. , Netfirms)
• Create your own (e.g., spare PC, Apache
software)
– software that is freely available over the Internet
• text editor (e.g., EditPlus)
Learning How to Conduct Research over the
Internet
• There are a number of web sites available that
provide tutorials on designing HTML and CGI
files.
– HTML
• http://www.w3schools.com/html/
– CGI
• http://www.cgi101.com/class/
• http://hotwired.lycos.com/webmonkey/98/47/index2a.html
– Both HTML and CGI
• http://www.web-research-design.net/
• [Note: You can download a ton of free researchrelevant scripts at this site]
Learning How to Conduct Research over the
Internet
• There are also some useful books.
Guilford Press
Prentice Hall
Learning How to Conduct Research over the
Internet
• Finally, there are a number of professional
services for hire that can design web studies for
you.
– Knowledge Networks
• http://www.knowledgenetworks.com/
– Survey Monkey
• http://www.surveymonkey.com/