Download Lecture 17

Cognitive models

Today we will cover
 goal and task hierarchies
 Linguistic
 physical and device
 architectural

What is cognitive science?
Cognitive science is
the science of mind and behavior.







Cognitive Science Theme
“Cognitive”
Of or pertaining to cognition, or to the action or process of knowing.
Understanding knowledge acquisition and use is the key to understanding the mind.
So what is cognitive science?
Cognitive science is a scientific study of the mind with special emphasis on the use and
acquisition of knowledge and information.
Implications
 An inter-disciplinary approach – Many scientific disciplines contribute to cognitive
science.
 A computational approach – Explain information processing in terms of neural
computations.






Definition 1
"the study of intelligence and intelligent systems, with particular reference to intelligent
behavior as computation" (Simon & Kaplan, 1989)
Simon, H. A. & C. A. Kaplan, "Foundations of cognitive science", in Posner, M.I. (ed.)
1989, Foundations of Cognitive Science, MIT Press, Cambridge MA.
Definition 2
Cognitive science refers to the interdisciplinary study of the acquisition and use of
knowledge. It includes as contributing disciplines: artificial intelligence, psychology,
linguistics, philosophy, anthropology, neuroscience, and education.
Cognitive science grew out of three developments: the invention of computers and the
attempts to design programs that could do the kinds of tasks that humans do; the
development of information processing psychology where the goal was to specify the
internal processing involved in perception, language, memory, and thought; and the
development of the theory of generative grammar and related offshoots in linguistics


















Definition 2 (con.)
Cognitive science was a synthesis concerned with the kinds of knowledge that underlie
human cognition, the details of human cognitive processing, and the computational
modeling of those processes.
There are five major topic areas in cognitive science: knowledge representation, language,
learning, thinking, and perception.
Eysenck, M.W. ed. (1990). The Blackwell Dictionary of Cognitive Psychology.
Cambridge, Massachusetts: Basil Blackwell Ltd.
Definition 3
Generally stated, this is the study of intelligence and intelligence systems.
It is a relatively new science that combines knowledge gained from a number of
disciplines. These include: computer science, neuroscience, cognitive psychology,
philosophy, and linguistics.
As a result of the collaborative effort between these disciplines, there have been, and will
continue to be, huge advancements in our understanding of human cognition.
Definition in wiki
Cognitive science may be broadly defined as the multidisciplinary study of mind and
behavior. It draws on multiple empirical disciplines, including psychology, philosophy,
neuroscience, linguistics, anthropology, computer science, sociology and biology.
Definition in Plato encyclopedia
Cognitive science is the interdisciplinary study of mind and intelligence, embracing
philosophy, psychology, artificial intelligence, neuroscience, linguistics, and anthropology.
Cognition
Cognition – from Latin base cognitio – “know together”
The collection of mental processes and activities used in perceiving, learning,
remembering, thinking, and understanding, and the act of using those processes
Information processing everywhere
Perception
 acquiring real-time information about the surrounding environment.
Language use






making use of information about syntax, semantics and phonology.
Reasoning
 combining different sources of information, deriving new information, testing
consistency of information, etc.
Action
 making use of information in action planning and guidance.
Memory
 storing and retrieving information
Cognitive Model for HCI
We need to model some aspect of the user’s understanding, knowledge, intentions or
processing.
 The level of representation differs from technique to technique – from models of
high-level
goals
and
the
results
of
problem-solving activities, to descriptions of motor-level activity, such as keystrokes
and mouse clicks.




Cognitive models
They model aspects of user:
 understanding
 knowledge
 intentions
 processing
Common categorisation:
 Competence vs. Performance
 Computational flavour
 No clear divide
Competence Vs Performance
“Competence models tend to be ones that can predict legal behavior sequences but
generally do this without reference to whether they could actually be executed by users.
In contrast, performance models not only describe what the necessary behavior
sequences are but usually describe both what the user needs to know and how this is
employed in actual task execution.”





Competence Vs Performance
Competence models, therefore, represent the kinds of behavior expected of a user, but they
provide little help in analyzing that behavior to determine its demands on the user.
Performance models provide analytical power mainly by focusing on routine behavior in
very limited applications.
Classification of Cognitive Models
Cognitive models for HCI are mainly classified into,
 hierarchical representation of the user’s task and goal structure
 linguistic and grammatical models
 physical and device-level models.

Goal and task hierarchies
Solve goals by solving subgoals
- Mental processing as “divide-and-conquer”




Example: sales report
produce report
gather data
. find book names
. . do keywords search of names database
. . . … further sub-goals
. . sift through names and abstracts by hand
. . . … further sub-goals
. search sales database - further sub-goals
layout tables and histograms - further sub-goals
write description - further sub-goals
goals vs. tasks
goals – intentions
what you would like to be true
tasks – actions
how to achieve it

GOMS – goals are internal

HTA
– actions external
– tasks are abstractions



Issues for goal hierarchies
Granularity
 Where do we start?
 Where do we stop – how far to subdivide?
 Get down to a routine learned behavior, not problem solving
Conflict
 More than one way to achieve a goal



Techniques

Cognitive Complexity Theory (CCT)







Treatment of error
Goals, Operators, Methods and Selection (GOMS)
Hierarchical Task Analysis (HTA) -(Covered in Later Lectures)
GOMS: Overview
Formal representation of routine cognitive skill.
A description of knowledge required by an expert user to perform a specific task.
Provides a description of what the user must learn.
GOMS: Classification
Provides a predictive, descriptive and prescriptive model





- the unit task
Predictive
 Predicts the time it will take user to perform the tasks under analysis
Descriptive
 Represents the way a user performs tasks on a system
Prescriptive
 Guides the development of training programs and help systems
GOMS: Definition
GOMS models user’s behavior in terms of:

Goals





What the user wants to do.
Operators
 Specific steps a user is able to take and assigned a specific execution time.
Methods
 Well-learned sequences of subgoals and operators that can accomplish a goal.
Selection Rules
 Guidelines for deciding between multiple methods.
GOMS example
GOAL: CLOSE-WINDOW
. [select GOAL: USE-MENU-METHOD
. MOVE-MOUSE-TO-FILE-MENU
. PULL-DOWN-FILE-MENU
. CLICK-OVER-CLOSE-OPTION
GOAL: USE-CTRL-W-METHOD
. PRESS-CONTROL-W-KEYS]
For a particular user:
Rule 1: Select USE-MENU-METHOD unless another
rule applies
Rule 2: If the application is GAME,
select CTRL-W-METHOD
 Cognitive Complexity Theory - CCT
-
basic premises of goal decomposition
provides more predictive power
Two parallel descriptions:



User - production rules of the form:
if condition then action
Device - generalized transition networks
covered under dialogue models
Example: editing with vi
Production rules are in long-term memory
- 4 rules in the text on page 425
User sees a mistake - Model contents of working memory as attribute-value mapping
(GOAL perform unit task
(TEXT task is insert space)
(TEXT task is at 5 23)
(CURSOR 8 7)












Four rules to model inserting a space
Notes on CCT
Rules don’t fire in order written, may repeat
Parallel model – rules can fire simultaneously
Novice versus expert style rules
Error behavior can be represented
Measures



Depth of goal structure
Number of rules (more means interface more difficult to learn)
Comparison with device description
Problems with goal hierarchies
description can be enormous
a post hoc technique – risk is that it is defined by the computer dialog and not user
expert versus novice
Simple extensions possible
 goal closure (makes sure subgoal satisfied)





eg. ATM example
Linguistic notations
User’s interaction with a computer is often viewed in terms of a language.
Understanding the user's behaviour and cognitive difficulty based on analysis of language
between user and system.
Similar in emphasis to dialogue models
 Backus–Naur Form (BNF)

 Task–Action Grammar (TAG)
Backus-Naur Form (BNF)

Very common notation from computer science

A purely syntactic view of the dialogue







Terminals
 lowest level of user behaviour
 e.g. CLICK-MOUSE, MOVE-MOUSE
Nonterminals
 ordering of terminals
 higher level of abstraction
 e.g. select-menu, position-mouse
Example of BNF
Basic syntax:
 nonterminal ::= expression
An expression
 contains terminals and nonterminals
 combined in sequence (+) or as alternatives (|)
draw line
select line
choose points
choose one
last point
pos mouse
::=
::=
::=
::=
::=
::=
select line + choose points + last point
pos mouse + CLICK MOUSE
choose one | choose one + choose points
pos mouse + CLICK MOUSE
pos mouse + DBL CLICK MOUSE
NULL | MOVE MOUSE+ pos mouse
Measurements with BNF
Number of rules (not so good)

Number of + and | operators

Complications



same syntax for different semantics
no reflection of user's perception
minimal consistency checking


Task Action Grammar (TAG)
Making consistency more explicit

Encoding user's world knowledge

Parameterised grammar rules



Nonterminals are modified to include additional semantic features
Consistency in TAG
In BNF, three UNIX commands would be described as:
copy ::= cp + filename + filename | cp + filenames + directory
move ::= mv + filename + filename | mv + filenames + directory
link ::= ln + filename + filename | ln + filenames + directory

No BNF measure could distinguish between this and a less consistent grammar in which
link








::= ln + filename + filename | ln + directory + filenames
Consistency in TAG
In TAG, this consistency of argument order can be made explicit using a parameter, or
semantic feature for file operations.
Consistency in TAG (cont'd)
Feature Possible values
Op = copy; move; link
Rules
file-op[Op] ::=
command[Op] + filename + filename
| command[Op] + filenames + directory
command[Op = copy] ::= cp
command[Op = move] ::= mv
command[Op = link] ::= ln
Other uses of TAG
User’s existing knowledge
Congruence between features and commands













These are modelled as derived rules
Physical and device models
The Keystroke Level Model (KLM)
Buxton's 3-state model
Based on empirical knowledge of human motor system
User's task: acquisition then execution.
 these only address execution
Complementary with goal hierarchies
Keystroke Level Model (KLM)
lowest level of (original) GOMS
six execution phase operators
 Physical motor: K - keystroking
P - pointing
H - homing
D - drawing

Mental
M - mental preparation

System
R - response
times are empirically determined.
Texecute = TK + TP + TH + TD + TM + TR
KLM example
GOAL: ICONISE-WINDOW
[select
GOAL: USE-CLOSE-METHOD
. MOVE-MOUSE-TO- FILE-MENU
. PULL-DOWN-FILE-MENU
. CLICK-OVER-CLOSE-OPTION
GOAL: USE-CTRL-W-METHOD
PRESS-CONTROL-W-KEY]
compare alternatives:
•
•

USE-CTRL-W-METHOD vs.
USE-CLOSE-METHOD
assume hand starts on mouse

Architectural models
All of cognitive models make assumptions
about the architecture of the human mind.


Problem Space Model
Rational behavior is characterized as behavior that is intended to achieve a specific goal.
 This element of rationality is often used to distinguish between intelligent and
machine-like behavior.



In the field of artificial intelligence (AI), a system exhibiting rational behavior is referred to
as
a
knowledge-level
system.
Problem Space Model
A knowledge-level system contains an agent behaving in an environment.
 The
agent
has
itself and its environment, including its own goals.
knowledge
 It
can
perform
certain
and sense information about its changing environment.
about
actions
 The agent behaves in its environment, it changes the environment and its own
knowledge.


Problem Space Model
We can view the overall behavior of the knowledge-level system as a sequence of
environment and agent states as they progress in time.
 The goal of the agent is characterized as a preference over all possible sequences of
agent/environment states.


Problem Space Model
Contrast this rational behavior with another general computational model for a machine,
which is not rational.
 For example, it is common to describe a problem as the search through a set of
possible states, from some initial state to a desired state.

Problem Space Model
 The search proceeds by moving from one state to another possible state by means of
operations or actions, the ultimate goal of which is to arrive at one of the desired
states.
 Once a programmer has identified a problem and a means of arriving at the solution
to the problem (the algorithm), the programmer then represents the problem and
algorithm in a programming language, which can be executed on a machine to reach
the desired state.

Problem Space Model
 The architecture of the machine only allows the definition of the search or problem
space and the actions that can occur to traverse that space.
 Termination is also assumed to happen once the desired state is reached.
 The machine does not have the ability to formulate the problem space and its
solution, mainly because it has no idea of the goal.



Problem Space Model
It is the job of the programmer to understand the goal and so define the machine to achieve
it.
We can adapt the state-based computational model of a machine in order to realize the
architecture of a knowledge-level system.
 The
new
the problem space model.



computational
model
is
Problem Space Model
Thus,
a
problem
space
consists
of
states and a set of operations that can be performed on the states.
a
set
of
Behavior in a problem space is a two-step process.
 First,
the
current
operator
is
chosen
based
on
the current state and then it is applied to the current state to achieve the new state




Summary
Cognitive models attempt to represent users as they interact with the system.
Most cognitive models do not deal with user observation and perception.
Some techniques have been extended to handle system output, but problems persist.


Summary
Issues:
 Level of granularity
 Exploratory interaction versus planning