Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
CCT 333: Imagining the Audience in a Wired World Class 7: Quantitative Research Methods Quantitative Methods • Unlike qualitative, involves metrics reduced to numbers • Why helpful? • Why problematic? Surveys/Questionnaires • Common method of obtaining information from broad cross-section of people • Quality of information directly depends on quality and purpose of questions and who is surveyed • Online tools help - e.g., http://www.surveymonkey.com Sampling • • • • Who is included in sample? How are they reached? Response rate issues Responders vs. non-responders - are they qualitatively different groups? • Directly impacts quality of results - e.g., 1936 presidential poll General Questionnaire Guidelines • • • • • Understandable Unambiguous Collects data that is actually valuable Can be easily analyzed I’d add - limited in scope - take respondent’s attention span and willingness to help into account! Question Guidelines • Specific better than general • Open/closed-ended questions - benefits and challenges • Opening and closing preamble and instructions important - especially if you’re not there to supervise • Test it before you use it Likert Scale Q • 1-5, 1-7, 1-9 scales • Midpoint - what does it mean? If no opinion, give that option • Take care in too many consecutive items with same polarity of options leads to patterned responses • Semantic differential can be effective Scales • Set of related questions measuring attitudes, beliefs, orientations etc. • Ex: multiple intelligences (others?) • Scales must actually measure what they claim, not be redundant • Verify authenticity (esp. in web searches) - many scales are meaningless Experiments • Controlled specific measurement of phenomenon • Often used to determine causation - not just X related to Y but X causes Y • Inferential vs. descriptive statistics - not simply 68% do X, but that this leads to something else Benefits and Limitations • Benefits: Controlled environment, measured responses, quantitative data that can lead to causal links • Limitations: Must simplify environment to minimize other potential explanatory variables, creating rather fake environment and tasks Data Mining • Observation without being there quantitative artifacts - e.g., Web access logs, click regions, eye tracking • PeopleMeter example • Records consequences of actual action - but potential privacy and collection issues (e.g., social networking helmet) Imagining the Audience in a Wired World • Hey, the course title means something! • Hierarchical task analysis and GOMS descriptions of cause and effect at functional level • Definitely important for planning computing systems (and often used - e.g. flowcharts, UML) • Why? Computers are not all that bright. GOMS • • • • • Goals Operators Methods Selection Rules Very possible that human and system GOMS differ - which causes problems Choosing Tools • Methods are like tools in a toolbox - all are useful for something - but you don’t hammer a nail with a screwdriver • Goals of research should primarily influence choice of tools • What else does? Other factors influencing method choice • Type of data needed - qual vs. quant, descriptive vs. inferential • Cost and time to collect data • Cost and time to analyze data • Triangulation needs • Contextual requirements Physical research example • 2002 racecar seat • Partially materials selection, stress calculations etc. - but mostly ergonomic • Quantitative measures of 5-95% percentile team members, and everyone in between • Lots of individual testing though too Org. research example • Social network questionnaire - who trusted whom in six domains • Correlated with three scales interdependence, independence and proactivity • Correlational study - what relations existed between scales and position in network? • Relations verified by respondent reflection and personal experience • Redesign implications Next week • A look at how simple user interaction gets complex when you add a few more humans or technologies • Think ahead Q: What technologies get more complicated to use when more people are involved?