Download Applications of Universal Computing

Collaboration via the Info. Utility
John Canny
Endeavour Mini-retreat
1/20/2000
Consequences of Ubiquitous
Computing and Calm Technology:
• As computing becomes ubiquitous and invisible, it also
ceases to be the focus of activity.
• The focus shifts instead to people and the tasks they
are working on.
• This implies a Human-Centered Computing approach
(HCC) to designing applications.
• Our work involves collaborations with several faculty
outside CS who are expert in aspects of human
behavior, and especially group behavior, via a HCC
center.
Three takes on collaboration
• Design media for natural (physical)
interaction: Desks, PRoPs, Bearables.
• Model behavior: Activity-Based Computing
(ABC):
– uses tacit data to infer structure in group
members’ activities.
• Learning as collaboration:
– Large classes create opportunity via group
cooperation and student perspectives to help
other learners.
Three takes on collaboration
• Design media for natural (physical)
interaction: Desks, PRoPs, Bearables.
• Model behavior: Activity-Based Computing
(ABC):
– uses tacit data to infer structure in group
members’ activities.
• Learning as collaboration:
– Large classes create opportunity via group
cooperation and student perspectives to help
other learners.
Designing Media:
Distributed Cognition Theory
• People use artifacts in the world to share their
thoughts (Norman, Hutchins, Pea).
• Computer-enhanced artifacts have richer
behaviors and be shared across distance.
A desk that loses entropy...
• The UPM (Universal Planar Manipulator) is a desk
that can move many objects placed on it. It is a
tangible input/output device.
Applications of the UPM
• Teaching Chemistry: A tangible simulation of planar
organic chemistry. Colored disks for atoms,
simulation of bond forces and reactions, movement
of atoms by the desk.
• Urban planning. Phycons of buildings moved by
initially by hand, then aligned by the desk according
to a set of rules.
• Other layout problems: web site design (moving
postit-like notes).
UPM principle of operation
PRoPs: Proxys for remote collab.
• There is still a huge gap
between face-to-face and
videoconferencing.
• First difference: Physical cues:
gaze, posture, proxemics... are
missing or distorted.
• Much collaboration involves
“being in” a remote space. You
need to have a body there...
PRoPs: Proxys for remote collab.
• PRoPs are internet-controlled robot avatars that
allow you to have a second presence somewhere.
• The user sits at a computer console watching
video from the PROP, and controlling it with
mouse and joystick.
Theoretical underpinnings
• Right now, much of the theory
driving PROP design comes from
outside CS:
• Non-verbal communication (D.
Keltner psychology).
• Theory of interpersonal
psychology (G. Mendelsohn).
• Spatial reasoning and spatial
memory.
Bearable computers
• The laptop can serve as a “bearable” computer, linked
wirelessly to its owner.
Heads-up virtual image
Double-mirror element
in glasses
Pocket-worn portable
LCD TV
Wireless chordal keyboard or
Palmpilot in pocket for input.
Laptop in briefcase
with wireless TV
and keyboard TXs.
Advantages of Bearables vs. Wearables
• Short term:
– Leverage higher performance of laptops and miniTV displays compared to wearable components.
• Long term:
–
–
–
–
No wires between components.
No power source to the head.
The laptop display is preferable sometimes.
Some optical benefits for augmented reality.
Three takes on collaboration
• Design media for natural (physical)
interaction: Desks, PRoPs, Bearables.
• Model behavior: Activity-Based Computing
(ABC):
– uses tacit data to infer structure in group
members’ activities.
• Learning as collaboration:
– Large classes create opportunity via group
cooperation and student perspectives to help
other learners.
ABC: Activity-Based Computing
• How do you build a UI when the computer
is invisible, e.g. a smart space?
• What is context and how do you use it?
• How does ABC help collaboration?
ABC: Activity-Based Computing
• How do you build a UI when the computer
is invisible, e.g. a smart space?
• Build a conceptual model instead.
• What is context and how do you use it?
• How does ABC help collaboration?
ABC: Activity-Based Computing
• How do you build a UI when the computer
is invisible, e.g. a smart space?
• Build a conceptual model instead.
• What is context and how do you use it?
• Use Activities as the context.
• How does ABC help collaboration?
ABC: Activity-Based Computing
• How do you build a UI when the computer
is invisible, e.g. a smart space?
• Build a conceptual model instead.
• What is context and how do you use it?
• Use Activities as the context.
• How does ABC help collaboration?
• Pro-active document sharing;
Understanding roles; Document context;
ABC: Activity-Based Computing
• Activities are clusters
with:
– Users
– Documents
– Tools
• The model can be
implemented as a GUI,
or with speech, etc.
• Nearness encodes
awareness.
ABC: Motivation
• Based on “activity theory”, a rich psychological
theory of human behavior, especially collaboration.
• AT Emphasizes on tools (such as computers) as
mediators of activity.
• Includes the role of symbol systems (language,
documents) as mediators.
• Ties together development (learning) and everyday
activity.
• Rich models of context.
ABC: Uses
•
•
•
•
•
•
•
•
•
•
Context (disambiguation) of person, object, tool.
Pro-active sharing of information.
Skill mining (tracking user expertise).
Document visibility and access rights.
Understanding individual’s roles in activities.
Managing work relationships.
Inferring quality of documents/tools.
Sensemaking for documents.
Perspectives on a document or meeting.
Attention management/awareness.
ABC: Under development now
•
•
•
•
•
•
•
•
•
•
Context (disambiguation) of person, document, tool.
Pro-active sharing of information.
Skill mining (tracking user expertise).
Document visibility and access rights.
Understanding an individual’s roles in activities.
Managing work relationships.
Inferring quality of documents/tools.
Sensemaking for documents.
Perspectives on a document or meeting.
Attention management/awareness.
ABC: Tacit data sources
•
•
•
•
Direct communication: 1-1 email, phone, F2F.
Topical discussions, forums, F2F meetings.
Document access, reading, doc. markup.
Document search and browsing, who, when.
Knowledgescapes: Info Retrieval
with tacit data
Query
(terms)
Search
Engine
Document1
Document2
Information
Need
Document3
Document4
Rankings from reading time.
Treat as probability of interest
or E(interest)
Info Needs and tacit rankings
Info
need1
P(D1|N1)
Document1
Document2
Info
need2
Document3
Document4
Info
need3
Document5
P(D5|N3)
First use: Most popular hits
• Start with a query string and use all the information
need nodes recorded for that query.
• Return highest weighted docs.
Document1
Info
need2
Document2
Document3
Query
Info
need3
Document4
Document5
Tacit knowledge in user queries
• The IN is usually more specific than the query, and
this is encoded in the user’s tacit rankings.
US to France
Document1
Info
need1
Document2
“Airlines”
Info
need2
Document3
Document4
Info
need3
Document5
West to East Coast
Tacit knowledge in user queries
• Tacit user ranking while browsing boosts most
probable info needs, improving recall.
US to France
Document1
Info
need1
Document2
“Airlines”
Info
need2
Document3
Document4
Info
need3
Document5
West to East Coast
Tacit knowledge in user queries
• Tacit user ranking while browsing boosts most
probable info needs, improving recall.
US to France
Document1
Info
need1
Document2
“Airlines”
Info
need2
Document3
Document4
Info
need3
Document5
West to East Coast
Tacit knowledge in user queries
• Tacit user ranking while browsing boosts most
probable info needs, improving recall.
US to France
Document1
Info
need1
Document2
“Airlines”
Info
need2
Document3
Document4
Info
need3
Document5
West to East Coast
Current testbeds:
• Berkeley DigLibs (environmental) collections.
Knowledgescapes on Dlib is running at:
http://indios.cs.berkeley.edu/knowledgescapes.html
• IMDB
• Philip Stark’s statistics course.
• NEEDs: the ME courseware database.
Future:
• My groups web and campus library access.
Three takes on collaboration
• Design media for natural (physical)
interaction: Desks, PRoPs, Bearables.
• Model behavior: Activity-Based Computing
(ABC):
– uses tacit data to infer structure in group
members’ activities.
• Learning as collaboration:
– Large classes create opportunity via group
cooperation and student perspectives to help
other learners.
Education as collaboration:
Opportunities of scale.
• In lecture format classrooms, attention is the
critical resource (Norman, Papert...).
• Attention is best gained by interaction: with
artifacts (the LOGO model) or with other
students (the Stanford TVI model).
An augmented reality classroom
Virtual image with notes,
questions, private chat space
Glasses to rotate
the laptop image
Physical space
Livenotes in the classroom
• Students work in groups of 4; communicate silently via
pen or keyboard chat.
• Each group has one main note-taker; others add their
own comments or questions to the transcript.
• Students can mark up a group transcript, the
lecturer’s notes, or a non-archived window.
• One student per group works as facilitator or TA,
posing questions to the others.
Using Livenotes
• The group transcripts include notes, plus student
comments and questions.
• Remote live participants should be equally engaged in
the note-taking.
Initial feedback
• 4 Students used Livenotes in a grad course in F99 on
IBM laptops running Netmeeting on a wireless net.
• Reactions:
– Overlay touch screens were bad, everyone used keyboard
chat.
– Difficulty in listening and chatting simultaneously only in
first lecture.
– After that, attention level higher. No chance of falling
asleep.
– Many notes: two parallel threads, the note-taker and the
group chat. Group chat periodically comes back to lecture
content as new notes appear from the note-taker.
Next Step
• Move to Vadem Clios with wireless:
– Support both keyboard and pen note-taking.
– Cheaper, lighter, batteries last all day.
• Develop custom software based on feedback:
– Support conversation threads.
– Include hyperlinks (or hyper-ink).
– Include timestamping to allow synchronization with MM
transcript of the lecture.
– In-class “lecture rewind” was proposed as a useful feature.
Further along...
• Offline students lose interaction, but gain through
access to live student perspectives (their notes).
• By organizing live student notes according to
expertise and a few dimensions of perspective, we
can provide offline students with an ideal
“scaffolding” perspective.
The End..
PROPs and Bearables are complementary
• A user with a bearable display
can see a video overlay of the
PROPs pilot.
PROPs and Bearables are complementary
• A user with a bearable display
can see a video overlay of the
PROPs pilot.
• This display can cover a much
larger area than the PROPs
built-in display, and provide
body language cues.
PROPs and Bearables are complementary
• A user with a bearable display
can see a video overlay of the
PROPs pilot.
• This display can cover a much
larger area than the PROPs
built-in display, and provide
body language cues.
• In this display, the pilot is
standing up, implying he is
controlling the PROP with a
wearable computer.