Download Dimensions of Scalability in Cognitive Models

Dimensions of Scalability
in Cognitive Models
Research Team:
Carnegie Mellon University - Psychology Department
Dr. Christian Lebiere
Dr. David Reitter
Dr. Jerry Vinokurov
Michael Furlong
Jasmeet Ajmani
Overview
• Goal: Scaling up high-fidelity cognitive models by
– Composing models
– Abstracting models
– Running large networks of models
• ACT-UP: a toolkit view of cognitive architectures
– Same validated functionality, different form
• Lemonade game: Reusing and integrating models
• Language learning: Scaling up to network cognition
• The Geo-Game: Bringing it all together
– Platform for experimentation and integration
Dimensions of Scaling
ACT-R Cognitive Architecture
• Computational
implementation of
unified theory of
cognition
• Commitment to taskinvariant mechanisms
• Modular organization
• Parallelism but strong
attentional limitations
• Hybrid symbolic/
statistical processes
Issues with Cognitive Modeling
• High-fidelity cognitive models provide very
accurate models of all observable dimensions of
cognition (time, accuracy, gaze, neural) but
• They are computationally intensive as they
simulate all cognitive processes in full detail
• They are labor intensive to specify all aspects of
cognitive performance (knowledge, strategies)
• They are specialized to a given task in a way
that makes them difficult to compose and reuse
• They usually focus on single-agent cognition
Scaling Up Cognitive Modeling
• Enable the implementation of more complex
cognitive models in a more efficient manner
• Scale up the application of cognitive models
to simulate learning and adaptation in
communities (e.g., 1,000 models in parallel)
• Enable reuse and composition of cognitive
models similar to software engineering view
• Facilitate integration of cognitive models with
other modeling and simulation platforms
• Improve maintenance, update and validation
The Approach
• Difficulties: ACT-R is heavily constrained already, and
models are difficult to develop, reuse and exchange
• Constraints: Architectural advances require further
constraints, e.g. more representational constraints
• Scaling it up: Complex tasks, broad coverage of
behavior, multi-agent cognition and predictive modeling
may motivate further architectural changes
• Solution: produce models at a higher abstraction level
• Retain and emphasize key cognitive mechanisms
• Abstract purely mechanistic model aspects
• Precisely specify model claims, underspecify/fit rest
• Benefits of abstraction in efficiency, scalability,
reuse
Cognitive Strategy
Symbolic
deterministic
Subsymbolic
(Learning /
Adaptation)
non-deterministic
explains empirical
variance
Underspecified Models
underspecify:
deterministic
specify:
non-deterministic
explains empirical
variance
(Lisp Functions)
ACT-UP vs ACT-R 6
• Declarative memory: chunks as objects
– Explicit context specification; all activation
computations
• Procedural memory: productions as functions
– Explicit conflict set groups; utility
reinforcement learning
• ACT-UP is synchronous with serial execution
– Parallelism in process of being implemented
• Perceptual-motor modules being planned
Validation
• Against canonical ACT-R tutorial models data
Efficiency
• Sentence production (syntactic priming)
model
– 30 productions in ACT-R, 720 lines of code
– 82 lines of code in ACT-UP (3 work-days)
– ACT-R 6: 14 sentences/second
– ACT-UP: 380 sentences/second
Scalability
• Language evolution model
– Simulates domain vocabulary emergence
(ICCM 2009, JCSR 1010)
– 40 production rules in ACT-R
– Complex execution paths: could not prototype
– 8 participants interacting in communities
• In larger community networks:
– 1000 agents
– 84M interactions (about 1 min. sim. Each)
– 37 CPU hours
Related Work
• Douglass (2009; 2010) on large declarative memories
– Implementation through Erlang threads
– Focus on scalability
• Salvucci (2010) work on supermodels
– Integrating and validating independent models
– Focus on instruction interpretation for generality
• Stewart and West (2007) work on Python-ACT-R
– Similar deconstructive view of architecture
– Integration with neural constructs
Future Work
• Complete validation against canonical model set; currently in
beta testing; full release planned for spring 2011
• Possible collaboration with AFRL Mesa on implementation
of finite-state-based systems
• Potential use in other projects (Minds Eye, Robotics CTA)
• Allow optional parallelism where needed and desired
• Implement perceptual and motor modules
• Potential implementation in other languages (C++, Java) to
facilitate code-level integration with common frameworks
Reitter, D., & Lebiere, C. (2010). Accountable Modeling in ACT-UP, a Scalable, Rapid-Prototyping ACT-R Implementation. In
Proceedings of the 2010 International Conference on Cognitive Modeling. Philadelphia, PA.
Lebiere, C., & Reitter, D. (2010). ACT-UP: A Cognitive Modeling Toolkit for Composition, Reuse and Integration. In
Proceedings of the 2010 MODSIM conference. Hampton, VA.
Lebiere, C., Stocco, A., Reitter, D., & Juvina, I. (2010). High-fidelity cognitive modeling to real-world applications. In
Proceedings of the NATO Workshop on Human Modeling for Military Application, Amsterdam, NL, 2010.
Cognitive principles in cooperative and
adversarial games:
Metacognition transfers via ACT-UP
Networks (Distributed Knowledge)
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Complex Tasks,
Broad-Coverage Models
Controlled Tasks,
High-Fidelity Models
ACT-UP: Rapid prototyping/Reuse
• Dynamic Stocks & Flows ACT-UP model
– Winning modeling competition entry
– Model written in < 1 person-month
– Free parameters (timing) estimated from
example data
– Model generalized to novel conditions
• Reuse of Metacognitive Strategy in the
Lemonade Stand Game (BRIMS 2010)
Kevin A. Gluck, Clayton T. Stanley, Jr. L. Richard Moore, David Reitter, and Marc Halbrügge. Exploration for understanding in
model comparisons. Journal of Artificial General Intelligence (to appear), 2010.
David Reitter. Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General
Intelligence (to appear), 2010.
David Reitter, Ion Juvina, Andrea Stocco, and Christian Lebiere. Resistance is futile: Winning lemonade market share through
metacognitive reasoning in a three-agent cooperative game. In Proceedings of the 19th Behavior Representation in
Modeling & Simulation (BRIMS), Charleston, SC, 2010.
Multi-agent Games
• 2x2 games such as the Prisoner’s Dilemma
– Evolution of cooperation vs. competition
– Memory-based expectations (Lebiere et al, 2001)
• Adversarial games (Paper Rock Scissors, Baseball)
– Zero-sum competition where predictability is fatal
– Sequence-based expectations (Lebiere et al, 1998; 2003)
• Lemonade game (3 players)
– Simultaneous cooperation and competition
– Predictability can be desirable for cooperation
The Lemonade Stand Game
Zinkevich (2010, unpublished)
• In each iteration, each of three players
chooses a location 1..12
• Payoff is proportional to the
distance to left and right neighbors.
• Hidden moves (blind choice)
• 1 game: 100 iterations, then reset
(no state across games)
Basic Strategies
• Random (unpredictable): choose random loc.
• Sticky (predictable): choose same location
– Roll, SquareRoot
• Tournament with those four agents
– Equal performance
Strategy Elements
• Offer Cooperation: Be predictable
• Predict: Learn patterns of opponents
• Maximize Utility: Choose highest expected
payoff
• Cooperate: Pick “friendly” opponent whose
payoff is also maximized
• Monitoring: analyzing own/others
performance, keep history
Strategies
offer Coop
Sticky
+
StickySmart
+
Predict
maxPayoff
Coop
MonitorSelf MonitorOpp
+
+
StickySharp
+
CopyCat
+
+
Statistician
Cooperator
+
Strategist
+
+
+
+
+
Metacognition
• Facility to constantly monitor performance,
and to adapt behavior accordingly
• Choose the best-performing strategy out of a
set of strategies (Flavell 1979, Brown 1987)
• Strategy-shifting assumed in Dynamic Stocks
& Flows data (DSF Challenge)
General Metacognition
• Prediction of each opponent’s next move
– Learn from agent’s history in this game
– Multiple possible representations and
pattern-matching
• Action: Making a move
– Optimize Utility
– Suggest cooperation
– Cooperate
– Hurt the worst adversaries
Evaluating Strategies
• Prediction and Action strategies are learned as
episodes (instances):
– Each prediction strategy per iteration, per
opponent
– one action strategy per iteration
• Instance-based learning (Gonzales&Lebiere 2003)
• Objective: Prediction quality/Action payoff
• Blending: weighted mean (recency, frequency,
objective as above)
Metacognition in Prediction
as in Reitter (2010) - DSF model
• Each prediction strategy suggests a next
location for each opponent
• All past predictions are stored throughout the
game: <t,l,p> (time, actual location, predicted
probability of that location)
Expected success
of strategy s and
agent a
Episode in memory:
time t, actual chosen location l of agent a,
predicted probability p for l,a by strategy s
ACT-R Activation
(recency, frequency)
Metacognition for Actions is similar
Evaluation
• Outcome of each
strategy depends on
configuration of
players
– Some strategies will
cooperate
• Metacognitive
strategy is flexible,
achieves consistently
high results
• Bigger circle: higher
winnings. Darker
circle: consistent
results.
Tournament
Adaptive Multi-Agent Behavior
• Offering cooperation and cooperating with
the right opponent are crucial to doing well
• Metacognitive layer allows an agent to trump
all others through generality and adaptivity
• Research questions:
– Human performance in cooperative games:
issues of trust, social and cultural biases
– Memory activation and rational retrieval
expectations as proxy for weighing past
strategy success – limits of metacognition
Future Work
• ACT-R/ACT-UP’s learning vs. more basic Bayesian models:
is cognitive learning more robust through open-endedness?
• Break down current limits of cognitive models generality
– Are canonical architectural parameters optimal through
coevolution for empirical clustering factors and degrees?
• Key part of environment is social interactions
– Automatic acquisition of rules, strategies, structural
representations rather than modeler specification
– Metacognition: accumulation of micro-strategies library
into reusable, general-purpose metacognitive layer
– Combination of above provide way of breaking out of taskspecific models and their assumptions: beyond taskspecific parameters, representation, strategies
Scaling Up Cognitive Models from
Individuals to Large Networks
The case of communication in human communities
Dr. Christian Lebiere
Dr. David Reitter
Carnegie Mellon University
Networks (Distributed Knowledge)
Communities (Teamwork)
David Reitter and Christian Lebiere.
Towards explaining the evolution of
domain languages with cognitive
simulation. Cognitive Systems
Research (in press), 2010.
Complex Tasks,
Broad-Coverage Models
Dyads (Dialogue)
Individuals
Controlled Tasks,
High-Fidelity Models
Interactive Alignment
• Garrod & Pickering 2004:
Syntactic
Representation
from: Garrod &Pickering, BBS 2004
Syntactic
Representation
Adaptation in Language
• Rapid decay
within 8-10 seconds
experimentally, for selected
constructions: Levelt & Kelter (1982),
Branigan et al. (2000)
• Long-term
adaptation effects,
which do not decay,
have also been observed
(Comprehension: Mitchell et al. 1995.
Production: Bock&Griffin 2000)
• ACT-R’s declarative
memory decay
explains the repetition probability decay
(Switchboard corpus)
Reitter (2008)
Interactive Alignment
Syntactic and Lexical Adaptation
Predict Task Success!
(Reitter & Moore 2007)
Lexical
Representation
from: Pickering&Garrod, BBS 2004
Lexical
Representation
Domain Language Experiment
• Vocabulary: Signs as meaning-signifier combination
Simple Communication System: Lewis 1989, Hurford 1989, Oliphant&Batali 1996
• Naming game: an idealized transaction between two players
– Pictionary: a director draws a given target concept using
elementary drawings; a matcher has to guess the concept.
– 20 target concepts, repeated
– Director/Matcher receive no explicit feedback
“Brad Pitt”
Fay et al., Cognitive Science 34(3), 2010. Kirby et al., PNAS 2008; Fay et al. PhilTransRoySoc-B 200
Pictionary Performance
partner switch
(communities)
partner switch
(communities)
(empirical)
partner switch
(communities)
Community
Isolated
Pairs
ID accuracy:
proportion of
signs retrieved
From data by
Fay et al. 2010
Broad Questions
• How does the architecture of human cognition interact with
social structure?
• Have the human mind and large-scale social structures coevolved?
• Can modeling predict the kinds of team structures that will
yield optimal communication and collaboration?
Pictionary Model in ACT-UP
• Ontology shared between
director and matcher
– abstract target concepts
– concrete drawings
– link weight distribution
acquired from Wall Street Journal
collocations
• Director chooses three
related drawings to
convey a target concept
• Choice is conventionalized
• Decision-making and memory
retention modeled with ACT-UP
Ontology
Pictionary
and Networks
partner switch
(communities)
Community
partner switch
(communities)
partner switch
(communities)
(ACT-UP model)
Isolated
Pairs
ID accuracy:
proportion of
signs retrieved
100 rep.
Reitter&Lebiere
Journal of Cognitive
Systems Research, in
press
Scaling up to Networks
Dr. Christian Lebiere
Dr. David Reitter
Carnegie Mellon
University
Reitter, D., & Lebiere, C. (2010). Did social networks
shape language evolution? A multi-agent cognitive
simulation. In Proc. Cognitive Modeling and
Computational Linguistics Workshop (CMCL 2010),
Uppsala, Sweden.
Networks (Distributed Knowledge)
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Complex Tasks,
Broad-Coverage Models
Controlled Tasks,
High-Fidelity Models
Research Questions
• Does network structure affect convergence towards a
common community vocabulary?
– Or: Is declarative memory robust w.r.t. a variety of network
structures?
• The small-scale, empirical and modeling data suggest that
extreme networks (fully vs. disconnected) arrive at similar
performance, but converge differently. How? Why?
• Larger communities that differ in their connectivity are
needed to answer these questions.
Network Types
• In a network, only network neighbors play the naming game
• Social: Small-World network
(low path length, high clustering coefficient,
assortatively mixed by degree)
• Grid (torus)
• Random Graph
• Organizational: Trees
•
•
Controlled: mean degree (except trees),
number of nodes
Here: 512 nodes, mean deg. 6., 50 rep. per condition
Network type: ***
random<grid<smallworld<tree
MCMC on LMER log(IDacc) ~ center(round) + cond + (1|sequence)
preliminary results
ID Accuracy: Neighbors
ID Accuracy: Random Pairs
Indication of convergence towards common vocabulary across
network (measured after round 35)
Small
World
Random
Grid
preliminary results
Tree
Tree vs. others: n.s.
(p=0.14, MCMC on LMER log(IDacc) ~ cond + (1|sequence))
Summary
• Online Linguistic Adaptation is a known phenomenon
– syntactic, lexical. Between two and more participants.
– Nodes can adapt to their immediate surroundings
• Tree hierarchies function very well when stable, but are
not robust to structural change
– Tree hierarchies represent contemporary organizational
hierarchies and generalize typical command structures
– Small-World structures are more robust to change.
Future Work
• Which advantages do non-tree network organizational
structures have in situations where environment/ground
truth changes, where adversarial elements are present?
• How can temporal dynamics in network structure (gradual
ramp-up in connectivity) support information convergence
(domain vocabulary acquisition)?
• Do cognitive models require explicit information processing
policies in non-tree hierarchies, such that accountability and
reliability are preserved?
• Integration of communication with planning, control and
decision-making in complex dynamic domains.
Information Exchange in Networks
• Simulation at cognitive level: Language Evolution Model
• Simulation with Bayesian Learners
Wang et al. (CMU Robotics), for a Bayesian Belief Update network
• Empirical validation is rare
– Real-time communication networks are rarely studied
– Most empirical datasets contain asynchronously produced
communication, lacking control over exchanged information
(e.g., Enron or Twitter corpora)
Human Networks:
Empirical Experiments with the Geo
Game
Dr. Christian Lebiere
Dr. David Reitter
Psychology,
Carnegie Mellon
University
Dr. Katia Sycara
Antonio Juarez
Dr. Paul Scerri
Dr. Robin Glinton
Robotics Institute,
Carnegie Mellon
University
Dr. Michael Lewis
University
of Pittsburgh
Complex
Tasks,
Broad-Coverage Models
Networks (Distributed Knowledge)
Communities (Teamwork)
Dyads (Dialogue)
Individuals
Controlled Tasks,
High-Fidelity Models
MURI Team: The Geo Game
Cornell
Scaling of cognitive
performance and
workload
Task allocation
among
humans/agents
Probabilistic models
of human decisionmaking in network
situations
Level 12.5
Level 1,2
GMU
Pitt
CMU
Robotics
CMU
Psychology
Level 1,3
Level 1,2
Level 1
Level 2
Level 1-3
Level 1
Level 1,3
Level 12.5
Level 3
Level 13
Decentralized control
search and planning
Information fusion
MIT
Level 1,2
Level 1,2
Level 1,2
Level 1-3
Level 1,3, 4
Level 2
Level 2
Network
performance as a
function of topology
Level 4
Level 2
Communication,
evolution, language
Level 3
Level 2, 3
Adaptive automation
Level 1,2
Level 1
The Geo Game
• Information exchange in human networks
– On-line (real-time) communication
– Medium to large networks (15 to 1,000 nodes)
– Defined information needed to execute given task
– Information is spread throughout the network
• Natural language as a means to exchange communication
– Often task-specific, controlled language
(e.g. radio communication)
• Trade-off: communication vs. task execution
The Geo Game platform is being developed by CMU Psychology and CMU Robotics
Geo Game: Participant’s Task
Geo Game as Platform
• Subjects are organized in a graph
– vertices define communication channels:
subjects can only communicate with network neighbors
– Currently: small-world network
• The Geo Game is a platform
– Current game: foraging task
– Other variants: trading agents, varied information types
(stochastic, graded, discrete, etc.), other networks (trees,
adversarial networks)
– Server&web-browser based system; remotely deployable
Geo Game: Push vs. Pull
• Basic manipulation: push vs. pull of information
– Relevant to practical domain and theoretical cognitive issues
• Push condition:
– Post all relevant information - items in cities; path efficiency
– Maximize information at cost of overloading attention/memory
• Pull condition:
– Specify needs and only answer/forward relevant information
– Minimize overload at cost of opportunities
Push vs. Pull: Scalability
• If communication aids task success, does this effect
scale?
One group of 15 participants
September 2010
Geo Game: Time-to-Response
• Question-Answer Pairs: time (Q to A) is power-law
distributed
cf. Barabasi (2010)
Publications
•
•
•
•
•
•
•
•
•
•
•
•
•
Reitter, D., & Lebiere, C. (2009). A subsymbolic and visual model of spatial path planning. In Proceedings of the Behavior
Representation in Modeling and Simulation (BRIMS 2009). Best paper award BRIMS 2009.
Reitter, D., & Lebiere, C. (2009). Towards Explaining the Evolution of Domain Languages with Cognitive Simulation. In
Proceedings of the 9th International Conference on Cognitive Modeling. Manchester, England.
Reitter, D., Lebiere, C., Lewis, M., Wang, H., & Ma, Z. (2009). A Cognitive Model of Perceptual Path Planning in a Multi-Robot
Control System. In Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics. San Antonio,
Texas.
Reitter, D., Juvina, I., Stocco, A., & Lebiere, C. (2010). Resistance is Futile: Winning Lemonade Market Share through
Metacognitive Reasoning in a Three-Agent Cooperative Game. In Proceedings of the Behavior Representation In Modeling and
Simulations (BRIMS 2010) Conference. Charleston, SC
Reitter, D., & Lebiere, C. (2010). Did social networks shape language evolution? a multi-agent cognitive simulation. In Proc.
Cognitive Modeling and Computational Linguistics Workshop (CMCL 2010, at Association for Computational Linguistics: ACL
2010), Uppsala, Sweden.
Reitter, D., & Lebiere, C. (2010). Accountable Modeling in ACT-UP, a Scalable, Rapid-Prototyping ACT-R Implementation. In
Proceedings of the 2010 International Conference on Cognitive Modeling. Philadelphia, PA.
Reitter, D., & Lebiere, C. (2010). On the influence of network structure on language evolution. In R. Sun, editor, Proc. Workshop on
Cognitive Social Sciences: Grounding the Social Sciences in the Cognitive Sciences (at Cognitive Science: CogSci 2010),
Portland, Oregon.
Lebiere, C., & Reitter, D. (2010). ACT-UP: A Cognitive Modeling Toolkit for Composition, Reuse and Integration. In Proceedings of
the 2010 MODSIM conference. Hampton, VA.
Lebiere, C., Stocco, A., Reitter, D., & Juvina, I. (2010). High-fidelity cognitive modeling to real-world applications. In Proceedings of
the NATO Workshop on Human Modeling for Military Application, Amsterdam, NL, 2010.
Reitter, D. & Lebiere, C. (in press). Towards explaining the evolution of domain languages with cognitive simulation. Journal of
Cognitive Systems Research.
Reitter, D. & Lebiere, C. (in press). A cognitive model of spatial path planning. Journal of Computational and Mathematical
Organization Theory.
Reitter, D. (to appear). Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General
Intelligence.
Gluck, K. A., Stanley, C. T., Moore, L. R., Reitter, D., & Halbrügge, M. (in press) . Exploration for understanding in model
comparisons. Journal of Artificial General Intelligence.
Future Work
• The Geo Game can be exploited as an experimental platform
for years to come
– Communication: Alignment of communication standards
(e.g., vocabulary)
– Knowledge: Information type and acquisition and its
influence on its distribution across the network (shared
knowledge is not copied knowledge)
– Network structure: influence on team performance
– Trust and strategy: Adversarial networks
• Mixed human/model/agent networks
– Bootstrapping methodology for model validation
– Information filtering for humans