Download Reactive Information Displays to Augment Problem Solving

Reactive Information Displays
N. Hari Narayanan
Overview
 Research Vision
 Recent Work
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
2
Information Comprehension
 Presenting information about complex
domains in a comprehensible fashion - an
old problem
 Accurate comprehension is crucial to
successful problem solving:
 Explaining, operating, troubleshooting,
predicting, planning, decision making...
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
3
Characteristics of Complex
Domains
 Components distributed in space
 Behaviors evolve over time
 11, 1many, many1 and
manymany cause-effect influences
These generate chains of events
Event chains branch and merge in
spatial and temporal dimensions
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
4
Examples of Complex
Domains
 Algorithms/Software
 Disaster Response
Machines/Mechanics
Meteorology
 Military Planning
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
5
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
6
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
7
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
8
Complex Event Chains
Integration of two
first-order variables
required here
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
9
Interactive Info Displays
Widespread use of interactive information
displays (IIDs) that employ multimedia to
convey complex information
Supposition: such displays allow people to:
comprehend more information faster...
and perform better…
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
10
Interactive Info Displays
 Converging evidence that such
suppositions may not be true
 Emerging theories of cognitively
based design guidance on multimedia
design to enhance comprehension and
learning
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
11
IID’s for Learning
 Cognitive model of comprehension
 Designing and evaluating IID’s
conform to the model
 Six design principles
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
12
Design Principles






Decomposition
Prior-knowledge
Co-reference
Lines-of-action
Mental animation
Basic laws
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
13
From Learning to
Performance




Limited time
Limited display space
Higher information density
Real-time response needs
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
14
Reactive Information
Displays
Display reacts to a user’s attention
shifts with a variety of behaviors:
elide, highlight, animate, zoom,
inform…
assists the problem solver by guiding
attention and offloading cognitive and
visual processing
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
15
Reactive Information
Displays
Displays that automatically augment
information in local regions, based on:
 a model of problem solving
 knowledge about task
 trajectory of the user’s attention
Present the right information at the right
time and in the right place
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
16
Example
Progressive Revealing
(Decomposition Principle)
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
17
Where is it raining in the US and what is its path?
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
18
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
19
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
20
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
21
Example
Systematic Attention Guiding
(Lines-of-action principle)
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
22
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
23
Overview
 Research Vision
 Recent Work
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
24
Experiment 1
 What separates successful &
unsuccessful problem solvers?
9 subjects
 8 mechanical problems
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
25
Problems
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
26
Data Collected
Accuracy of answers
Response time
Eye movements
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
27
Comparative Measures
 Response time
 Number of focus shifts
 Duration of gaze on critical
components
 Two aspects of “systematicity”:
 Coverage
 Order
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
28
Results
No significant difference in response time,
number of focus shifts and coverage
between successful and unsuccessful
subjects
Successful subjects considered
significantly more causal connections and
longer lines of action than unsuccessful
subjects.
Successful subjects had significantly
longer durations of visual attention on
critical components
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
29
Experiment 2
 90 subjects
Compared
a static display (n=15)
Machine-guided highlighting RID (n=15)
User-guided highlighting RID (n=20)
Machine-guided animating RID (n=20)
User-guided animating RID (n=20)
 Dependent measures




ONR-PIM@RPI 6.03
Accuracy
Response time
Coverage
Order
 Hari Narayanan All rights reserved
30
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
31
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
32
accur acy
1 00%
90%
80%
70%
60%
50%
40%
30%
20%
1 0%
0%
S
ONR-PIM@RPI 6.03
MH
UH
 Hari Narayanan All rights reserved
MA
UA
33
Results
 The machine-guided animating display
produced a significant improvement in
accuracy over the static display.
It induced a marginally significant increase
in mean response time compared to the
static display.
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
34
Next Steps
 Experimental evaluation of other, more
sophisticated, reactive strategies.
 Inferring & predicting locus of cognitive
focus from other data sources.
Developing a software architecture and
toolkit for RIDs that employ effective
strategies.
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
35
Summary
Research theme: Cognitive model-based
design of information displays
Past success: Design of interactive
information displays that significantly
improve learning
Current work: Design of reactive
information displays to improve problem
solving performance
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
36
Questions?
For more information and references:
see the short paper in the binder,
visit
http://www.eng.auburn.edu/~narayan
or email [email protected]
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
37
Results
S
A
40%
RT
76
C
55.5%
O
80.2
ONR-PIM@RPI 6.03
MH
UH
MA
UA
60%
60%
85%
60%
112
100
114
86
59.1%
63.4%
63.1%
56.4%
83.2
107.8
106.7
89.5
 Hari Narayanan All rights reserved
38
Example
Showing local behaviors
(mental animation principle)
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
39
ONR-PIM@RPI 6.03
 Hari Narayanan All rights reserved
40