Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download ppt - CSE Labs User Home Pages
Transcript
Evaluating User Interfaces Walkthrough Analysis Joseph A. Konstan [email protected] October 10 CSci 5115 Introduction to Evaluation Cognitive Walkthrough Other Evaluation Methods 2 Interface Development Methodology Prototype and Iterate keep iterating until it is good enough evaluate along the way to assess What is Good? What is Good Enough? set usability goals should relate to tasks CSci 5115 3 Casual Iteration Find major usability problems missing features user confusion poor interaction Try interface with specific tasks first use designers, then move towards users observe overall usage CSci 5115 4 Casual Iteration Remember the goal don’t defend the interface don’t bias the tests towards the interface If possible, allow user exploration may even lead to capturing new tasks CSci 5115 Consider alternative ways to fix a problem 5 Limits of Casual Iteration CSci 5115 Does not indicate when to stop Financial trade-offs Justification of delay 6 Usability Goals and Measures Concrete, quantitative measures of usability learning time use time for specific tasks and users error rates measures of user satisfaction Comparative usability goals compare with prior versions or competitors CSci 5115 7 Things to Watch Goals should be realistic 100% is never realistic Many goals go beyond the application UI training, manuals Testing goals should help improve the UI detail--not CSci 5115 just good/bad 8 Exercise: Setting Usability Goals In project groups, come up with 2 usability goals for your project discuss the feasibility of testing these goals what CSci 5115 is needed for the test when in the process can they be tested? how much effort, user preparation/training, etc.? what would you learn from the test? 9 Interface Evaluation Goals of interface evaluation find problems find opportunity for improvement determine if interface is “good enough” CSci 5115 10 With or Without Users Users are expensive and inconsistent usability studies require several users some users provide great information, others little Users are users cannot CSci 5115 be simulated perfectly Best choice--Both 11 Evaluation Without Users Quantitative Methods GOMS/keystroke analysis back-of-the-envelope action analysis Qualitative Methods expert evaluation cognitive walkthrough heuristic evaluation CSci 5115 12 Walkthrough Analysis Economical interface evaluation low-fidelity prototype development team users optional Effective, if goal is improvement, not defense some team members skilled proper motivation CSci 5115 13 Cognitive Walkthrough Goals imagine user’s experience evaluate choice-points in the interface detect confusing labels or options detect likely user navigation errors Start with a complete TCUID scenario never try to “wing it” on a walkthrough CSci 5115 14 Tell a Believable Story CSci 5115 How does the user accomplish the task Action-by-action Based on user knowledge and system interface 15 Best Approach Work as a group don’t partition the task Be highly skeptical remember CSci 5115 the goal! Every gap is an interface problem 16 Who Should Do the Walkthrough Designers, as an early check Team of designers & users remember: goal is to find problems avoid making it a show CSci 5115 Skilled UI people may be valuable team members 17 How Far Along Basic requirements description or prototype of interface know who users are (and their experience) a task description a list of actions to complete the task (scenario) DO CSci 5115 NOT try to create the action list on the fly! Viable once the scenario and interface sketch are completed 18 How to Proceed For each action in the sequence tell the story of why the user will do it ask critical questions will the user be trying to produce the effect? will the user see the correct control? will the user see that the control produces the desired effect? will the user select a different control instead? will the user understand the feedback to proceed correctly? CSci 5115 19 Walkthroughs are not Perfect They won’t find every problem limited by nature new users who know what task they need to accomplish biased towards correct action sequence limited hard CSci 5115 in implementation to shed the expertise of evaluators A useful tool in conjunction with others 20 Exercise: Cognitive Walkthrough Analysis In non-project groups of 3-5 Users and Task to be announced Scenario developed jointly Perform walkthrough identify problems estimate error probabilities (25% intervals) CSci 5115 Remember who your users are! 21 GOMS/Keystroke Analysis Formal action analysis accurately predict task completion time for skilled users Break task into tiny steps keystroke, mouse movement, refocus gaze retrieve item from long-term memory Look up average step times tables CSci 5115 from large experiments 22 GOMS/Keystroke Analysis Primary utility: repetitive tasks e.g., telephone operators benefit: can be very accurate (within 20%) may identify bottlenecks Difficulties challenging CSci 5115 to decompose accurately long/laborious process not useful with non-experts 23 Back-of-the-Envelope Action Analysis Coarse-grain list basic actions (select menu item) each action is at least 2-3 seconds what must be learned/remembered? what can be done easily? documentation/training? CSci 5115 Goal is to find major problems Example: 1950’s 35mm camera 24 Expert Evaluation Usability specialists are very valuable double-specialists CSci 5115 are even better An inexpensive way to get a lot of feedback Be sure the expert is qualified in your area 25 Looking Ahead Next week: Heuristic Evaluation Walkthroughs Due “raw” notes • notes from each step of walkthrough • copy of prototype used, markups • copy of scenarios used (note changes or fixes) processed results • 1-2 pages of issues identified, solutions not needed CSci 5115 26
