* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Dimensions of Scalability in Cognitive Models
Michael Tomasello wikipedia , lookup
William Clancey wikipedia , lookup
Recurrent neural network wikipedia , lookup
Bioecological model wikipedia , lookup
Situated cognition wikipedia , lookup
Junction Grammar wikipedia , lookup
George Armitage Miller wikipedia , lookup
Social network (sociolinguistics) wikipedia , lookup
Conceptual model wikipedia , lookup
Stephen Grossberg wikipedia , lookup
Play (activity) wikipedia , lookup
Mathematical model wikipedia , lookup
Nervous system network models wikipedia , lookup
Neural modeling fields wikipedia , lookup
Agent-based model wikipedia , lookup
Neo-Piagetian theories of cognitive development wikipedia , lookup
Neurophilosophy wikipedia , lookup
Background music wikipedia , lookup
Cognitive flexibility wikipedia , lookup
Cognitive neuroscience wikipedia , lookup
Cognitive semantics wikipedia , lookup
Ecological interface design wikipedia , lookup
Cognitive interview wikipedia , lookup
Cognitive psychology wikipedia , lookup
Embodied cognitive science wikipedia , lookup
Dimensions of Scalability in Cognitive Models Research Team: Carnegie Mellon University - Psychology Department Dr. Christian Lebiere Dr. David Reitter Dr. Jerry Vinokurov Michael Furlong Jasmeet Ajmani Overview • Goal: Scaling up high-fidelity cognitive models by – Composing models – Abstracting models – Running large networks of models • ACT-UP: a toolkit view of cognitive architectures – Same validated functionality, different form • Lemonade game: Reusing and integrating models • Language learning: Scaling up to network cognition • The Geo-Game: Bringing it all together – Platform for experimentation and integration Dimensions of Scaling ACT-R Cognitive Architecture • Computational implementation of unified theory of cognition • Commitment to taskinvariant mechanisms • Modular organization • Parallelism but strong attentional limitations • Hybrid symbolic/ statistical processes Issues with Cognitive Modeling • High-fidelity cognitive models provide very accurate models of all observable dimensions of cognition (time, accuracy, gaze, neural) but • They are computationally intensive as they simulate all cognitive processes in full detail • They are labor intensive to specify all aspects of cognitive performance (knowledge, strategies) • They are specialized to a given task in a way that makes them difficult to compose and reuse • They usually focus on single-agent cognition Scaling Up Cognitive Modeling • Enable the implementation of more complex cognitive models in a more efficient manner • Scale up the application of cognitive models to simulate learning and adaptation in communities (e.g., 1,000 models in parallel) • Enable reuse and composition of cognitive models similar to software engineering view • Facilitate integration of cognitive models with other modeling and simulation platforms • Improve maintenance, update and validation The Approach • Difficulties: ACT-R is heavily constrained already, and models are difficult to develop, reuse and exchange • Constraints: Architectural advances require further constraints, e.g. more representational constraints • Scaling it up: Complex tasks, broad coverage of behavior, multi-agent cognition and predictive modeling may motivate further architectural changes • Solution: produce models at a higher abstraction level • Retain and emphasize key cognitive mechanisms • Abstract purely mechanistic model aspects • Precisely specify model claims, underspecify/fit rest • Benefits of abstraction in efficiency, scalability, reuse Cognitive Strategy Symbolic deterministic Subsymbolic (Learning / Adaptation) non-deterministic explains empirical variance Underspecified Models underspecify: deterministic specify: non-deterministic explains empirical variance (Lisp Functions) ACT-UP vs ACT-R 6 • Declarative memory: chunks as objects – Explicit context specification; all activation computations • Procedural memory: productions as functions – Explicit conflict set groups; utility reinforcement learning • ACT-UP is synchronous with serial execution – Parallelism in process of being implemented • Perceptual-motor modules being planned Validation • Against canonical ACT-R tutorial models data Efficiency • Sentence production (syntactic priming) model – 30 productions in ACT-R, 720 lines of code – 82 lines of code in ACT-UP (3 work-days) – ACT-R 6: 14 sentences/second – ACT-UP: 380 sentences/second Scalability • Language evolution model – Simulates domain vocabulary emergence (ICCM 2009, JCSR 1010) – 40 production rules in ACT-R – Complex execution paths: could not prototype – 8 participants interacting in communities • In larger community networks: – 1000 agents – 84M interactions (about 1 min. sim. Each) – 37 CPU hours Related Work • Douglass (2009; 2010) on large declarative memories – Implementation through Erlang threads – Focus on scalability • Salvucci (2010) work on supermodels – Integrating and validating independent models – Focus on instruction interpretation for generality • Stewart and West (2007) work on Python-ACT-R – Similar deconstructive view of architecture – Integration with neural constructs Future Work • Complete validation against canonical model set; currently in beta testing; full release planned for spring 2011 • Possible collaboration with AFRL Mesa on implementation of finite-state-based systems • Potential use in other projects (Minds Eye, Robotics CTA) • Allow optional parallelism where needed and desired • Implement perceptual and motor modules • Potential implementation in other languages (C++, Java) to facilitate code-level integration with common frameworks Reitter, D., & Lebiere, C. (2010). Accountable Modeling in ACT-UP, a Scalable, Rapid-Prototyping ACT-R Implementation. In Proceedings of the 2010 International Conference on Cognitive Modeling. Philadelphia, PA. Lebiere, C., & Reitter, D. (2010). ACT-UP: A Cognitive Modeling Toolkit for Composition, Reuse and Integration. In Proceedings of the 2010 MODSIM conference. Hampton, VA. Lebiere, C., Stocco, A., Reitter, D., & Juvina, I. (2010). High-fidelity cognitive modeling to real-world applications. In Proceedings of the NATO Workshop on Human Modeling for Military Application, Amsterdam, NL, 2010. Cognitive principles in cooperative and adversarial games: Metacognition transfers via ACT-UP Networks (Distributed Knowledge) Communities (Teamwork) Dyads (Dialogue) Individuals Complex Tasks, Broad-Coverage Models Controlled Tasks, High-Fidelity Models ACT-UP: Rapid prototyping/Reuse • Dynamic Stocks & Flows ACT-UP model – Winning modeling competition entry – Model written in < 1 person-month – Free parameters (timing) estimated from example data – Model generalized to novel conditions • Reuse of Metacognitive Strategy in the Lemonade Stand Game (BRIMS 2010) Kevin A. Gluck, Clayton T. Stanley, Jr. L. Richard Moore, David Reitter, and Marc Halbrügge. Exploration for understanding in model comparisons. Journal of Artificial General Intelligence (to appear), 2010. David Reitter. Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General Intelligence (to appear), 2010. David Reitter, Ion Juvina, Andrea Stocco, and Christian Lebiere. Resistance is futile: Winning lemonade market share through metacognitive reasoning in a three-agent cooperative game. In Proceedings of the 19th Behavior Representation in Modeling & Simulation (BRIMS), Charleston, SC, 2010. Multi-agent Games • 2x2 games such as the Prisoner’s Dilemma – Evolution of cooperation vs. competition – Memory-based expectations (Lebiere et al, 2001) • Adversarial games (Paper Rock Scissors, Baseball) – Zero-sum competition where predictability is fatal – Sequence-based expectations (Lebiere et al, 1998; 2003) • Lemonade game (3 players) – Simultaneous cooperation and competition – Predictability can be desirable for cooperation The Lemonade Stand Game Zinkevich (2010, unpublished) • In each iteration, each of three players chooses a location 1..12 • Payoff is proportional to the distance to left and right neighbors. • Hidden moves (blind choice) • 1 game: 100 iterations, then reset (no state across games) Basic Strategies • Random (unpredictable): choose random loc. • Sticky (predictable): choose same location – Roll, SquareRoot • Tournament with those four agents – Equal performance Strategy Elements • Offer Cooperation: Be predictable • Predict: Learn patterns of opponents • Maximize Utility: Choose highest expected payoff • Cooperate: Pick “friendly” opponent whose payoff is also maximized • Monitoring: analyzing own/others performance, keep history Strategies offer Coop Sticky + StickySmart + Predict maxPayoff Coop MonitorSelf MonitorOpp + + StickySharp + CopyCat + + Statistician Cooperator + Strategist + + + + + Metacognition • Facility to constantly monitor performance, and to adapt behavior accordingly • Choose the best-performing strategy out of a set of strategies (Flavell 1979, Brown 1987) • Strategy-shifting assumed in Dynamic Stocks & Flows data (DSF Challenge) General Metacognition • Prediction of each opponent’s next move – Learn from agent’s history in this game – Multiple possible representations and pattern-matching • Action: Making a move – Optimize Utility – Suggest cooperation – Cooperate – Hurt the worst adversaries Evaluating Strategies • Prediction and Action strategies are learned as episodes (instances): – Each prediction strategy per iteration, per opponent – one action strategy per iteration • Instance-based learning (Gonzales&Lebiere 2003) • Objective: Prediction quality/Action payoff • Blending: weighted mean (recency, frequency, objective as above) Metacognition in Prediction as in Reitter (2010) - DSF model • Each prediction strategy suggests a next location for each opponent • All past predictions are stored throughout the game: <t,l,p> (time, actual location, predicted probability of that location) Expected success of strategy s and agent a Episode in memory: time t, actual chosen location l of agent a, predicted probability p for l,a by strategy s ACT-R Activation (recency, frequency) Metacognition for Actions is similar Evaluation • Outcome of each strategy depends on configuration of players – Some strategies will cooperate • Metacognitive strategy is flexible, achieves consistently high results • Bigger circle: higher winnings. Darker circle: consistent results. Tournament Adaptive Multi-Agent Behavior • Offering cooperation and cooperating with the right opponent are crucial to doing well • Metacognitive layer allows an agent to trump all others through generality and adaptivity • Research questions: – Human performance in cooperative games: issues of trust, social and cultural biases – Memory activation and rational retrieval expectations as proxy for weighing past strategy success – limits of metacognition Future Work • ACT-R/ACT-UP’s learning vs. more basic Bayesian models: is cognitive learning more robust through open-endedness? • Break down current limits of cognitive models generality – Are canonical architectural parameters optimal through coevolution for empirical clustering factors and degrees? • Key part of environment is social interactions – Automatic acquisition of rules, strategies, structural representations rather than modeler specification – Metacognition: accumulation of micro-strategies library into reusable, general-purpose metacognitive layer – Combination of above provide way of breaking out of taskspecific models and their assumptions: beyond taskspecific parameters, representation, strategies Scaling Up Cognitive Models from Individuals to Large Networks The case of communication in human communities Dr. Christian Lebiere Dr. David Reitter Carnegie Mellon University Networks (Distributed Knowledge) Communities (Teamwork) David Reitter and Christian Lebiere. Towards explaining the evolution of domain languages with cognitive simulation. Cognitive Systems Research (in press), 2010. Complex Tasks, Broad-Coverage Models Dyads (Dialogue) Individuals Controlled Tasks, High-Fidelity Models Interactive Alignment • Garrod & Pickering 2004: Syntactic Representation from: Garrod &Pickering, BBS 2004 Syntactic Representation Adaptation in Language • Rapid decay within 8-10 seconds experimentally, for selected constructions: Levelt & Kelter (1982), Branigan et al. (2000) • Long-term adaptation effects, which do not decay, have also been observed (Comprehension: Mitchell et al. 1995. Production: Bock&Griffin 2000) • ACT-R’s declarative memory decay explains the repetition probability decay (Switchboard corpus) Reitter (2008) Interactive Alignment Syntactic and Lexical Adaptation Predict Task Success! (Reitter & Moore 2007) Lexical Representation from: Pickering&Garrod, BBS 2004 Lexical Representation Domain Language Experiment • Vocabulary: Signs as meaning-signifier combination Simple Communication System: Lewis 1989, Hurford 1989, Oliphant&Batali 1996 • Naming game: an idealized transaction between two players – Pictionary: a director draws a given target concept using elementary drawings; a matcher has to guess the concept. – 20 target concepts, repeated – Director/Matcher receive no explicit feedback “Brad Pitt” Fay et al., Cognitive Science 34(3), 2010. Kirby et al., PNAS 2008; Fay et al. PhilTransRoySoc-B 200 Pictionary Performance partner switch (communities) partner switch (communities) (empirical) partner switch (communities) Community Isolated Pairs ID accuracy: proportion of signs retrieved From data by Fay et al. 2010 Broad Questions • How does the architecture of human cognition interact with social structure? • Have the human mind and large-scale social structures coevolved? • Can modeling predict the kinds of team structures that will yield optimal communication and collaboration? Pictionary Model in ACT-UP • Ontology shared between director and matcher – abstract target concepts – concrete drawings – link weight distribution acquired from Wall Street Journal collocations • Director chooses three related drawings to convey a target concept • Choice is conventionalized • Decision-making and memory retention modeled with ACT-UP Ontology Pictionary and Networks partner switch (communities) Community partner switch (communities) partner switch (communities) (ACT-UP model) Isolated Pairs ID accuracy: proportion of signs retrieved 100 rep. Reitter&Lebiere Journal of Cognitive Systems Research, in press Scaling up to Networks Dr. Christian Lebiere Dr. David Reitter Carnegie Mellon University Reitter, D., & Lebiere, C. (2010). Did social networks shape language evolution? A multi-agent cognitive simulation. In Proc. Cognitive Modeling and Computational Linguistics Workshop (CMCL 2010), Uppsala, Sweden. Networks (Distributed Knowledge) Communities (Teamwork) Dyads (Dialogue) Individuals Complex Tasks, Broad-Coverage Models Controlled Tasks, High-Fidelity Models Research Questions • Does network structure affect convergence towards a common community vocabulary? – Or: Is declarative memory robust w.r.t. a variety of network structures? • The small-scale, empirical and modeling data suggest that extreme networks (fully vs. disconnected) arrive at similar performance, but converge differently. How? Why? • Larger communities that differ in their connectivity are needed to answer these questions. Network Types • In a network, only network neighbors play the naming game • Social: Small-World network (low path length, high clustering coefficient, assortatively mixed by degree) • Grid (torus) • Random Graph • Organizational: Trees • • Controlled: mean degree (except trees), number of nodes Here: 512 nodes, mean deg. 6., 50 rep. per condition Network type: *** random<grid<smallworld<tree MCMC on LMER log(IDacc) ~ center(round) + cond + (1|sequence) preliminary results ID Accuracy: Neighbors ID Accuracy: Random Pairs Indication of convergence towards common vocabulary across network (measured after round 35) Small World Random Grid preliminary results Tree Tree vs. others: n.s. (p=0.14, MCMC on LMER log(IDacc) ~ cond + (1|sequence)) Summary • Online Linguistic Adaptation is a known phenomenon – syntactic, lexical. Between two and more participants. – Nodes can adapt to their immediate surroundings • Tree hierarchies function very well when stable, but are not robust to structural change – Tree hierarchies represent contemporary organizational hierarchies and generalize typical command structures – Small-World structures are more robust to change. Future Work • Which advantages do non-tree network organizational structures have in situations where environment/ground truth changes, where adversarial elements are present? • How can temporal dynamics in network structure (gradual ramp-up in connectivity) support information convergence (domain vocabulary acquisition)? • Do cognitive models require explicit information processing policies in non-tree hierarchies, such that accountability and reliability are preserved? • Integration of communication with planning, control and decision-making in complex dynamic domains. Information Exchange in Networks • Simulation at cognitive level: Language Evolution Model • Simulation with Bayesian Learners Wang et al. (CMU Robotics), for a Bayesian Belief Update network • Empirical validation is rare – Real-time communication networks are rarely studied – Most empirical datasets contain asynchronously produced communication, lacking control over exchanged information (e.g., Enron or Twitter corpora) Human Networks: Empirical Experiments with the Geo Game Dr. Christian Lebiere Dr. David Reitter Psychology, Carnegie Mellon University Dr. Katia Sycara Antonio Juarez Dr. Paul Scerri Dr. Robin Glinton Robotics Institute, Carnegie Mellon University Dr. Michael Lewis University of Pittsburgh Complex Tasks, Broad-Coverage Models Networks (Distributed Knowledge) Communities (Teamwork) Dyads (Dialogue) Individuals Controlled Tasks, High-Fidelity Models MURI Team: The Geo Game Cornell Scaling of cognitive performance and workload Task allocation among humans/agents Probabilistic models of human decisionmaking in network situations Level 12.5 Level 1,2 GMU Pitt CMU Robotics CMU Psychology Level 1,3 Level 1,2 Level 1 Level 2 Level 1-3 Level 1 Level 1,3 Level 12.5 Level 3 Level 13 Decentralized control search and planning Information fusion MIT Level 1,2 Level 1,2 Level 1,2 Level 1-3 Level 1,3, 4 Level 2 Level 2 Network performance as a function of topology Level 4 Level 2 Communication, evolution, language Level 3 Level 2, 3 Adaptive automation Level 1,2 Level 1 The Geo Game • Information exchange in human networks – On-line (real-time) communication – Medium to large networks (15 to 1,000 nodes) – Defined information needed to execute given task – Information is spread throughout the network • Natural language as a means to exchange communication – Often task-specific, controlled language (e.g. radio communication) • Trade-off: communication vs. task execution The Geo Game platform is being developed by CMU Psychology and CMU Robotics Geo Game: Participant’s Task Geo Game as Platform • Subjects are organized in a graph – vertices define communication channels: subjects can only communicate with network neighbors – Currently: small-world network • The Geo Game is a platform – Current game: foraging task – Other variants: trading agents, varied information types (stochastic, graded, discrete, etc.), other networks (trees, adversarial networks) – Server&web-browser based system; remotely deployable Geo Game: Push vs. Pull • Basic manipulation: push vs. pull of information – Relevant to practical domain and theoretical cognitive issues • Push condition: – Post all relevant information - items in cities; path efficiency – Maximize information at cost of overloading attention/memory • Pull condition: – Specify needs and only answer/forward relevant information – Minimize overload at cost of opportunities Push vs. Pull: Scalability • If communication aids task success, does this effect scale? One group of 15 participants September 2010 Geo Game: Time-to-Response • Question-Answer Pairs: time (Q to A) is power-law distributed cf. Barabasi (2010) Publications • • • • • • • • • • • • • Reitter, D., & Lebiere, C. (2009). A subsymbolic and visual model of spatial path planning. In Proceedings of the Behavior Representation in Modeling and Simulation (BRIMS 2009). Best paper award BRIMS 2009. Reitter, D., & Lebiere, C. (2009). Towards Explaining the Evolution of Domain Languages with Cognitive Simulation. In Proceedings of the 9th International Conference on Cognitive Modeling. Manchester, England. Reitter, D., Lebiere, C., Lewis, M., Wang, H., & Ma, Z. (2009). A Cognitive Model of Perceptual Path Planning in a Multi-Robot Control System. In Proceedings of the 2009 IEEE International Conference on Systems, Man, and Cybernetics. San Antonio, Texas. Reitter, D., Juvina, I., Stocco, A., & Lebiere, C. (2010). Resistance is Futile: Winning Lemonade Market Share through Metacognitive Reasoning in a Three-Agent Cooperative Game. In Proceedings of the Behavior Representation In Modeling and Simulations (BRIMS 2010) Conference. Charleston, SC Reitter, D., & Lebiere, C. (2010). Did social networks shape language evolution? a multi-agent cognitive simulation. In Proc. Cognitive Modeling and Computational Linguistics Workshop (CMCL 2010, at Association for Computational Linguistics: ACL 2010), Uppsala, Sweden. Reitter, D., & Lebiere, C. (2010). Accountable Modeling in ACT-UP, a Scalable, Rapid-Prototyping ACT-R Implementation. In Proceedings of the 2010 International Conference on Cognitive Modeling. Philadelphia, PA. Reitter, D., & Lebiere, C. (2010). On the influence of network structure on language evolution. In R. Sun, editor, Proc. Workshop on Cognitive Social Sciences: Grounding the Social Sciences in the Cognitive Sciences (at Cognitive Science: CogSci 2010), Portland, Oregon. Lebiere, C., & Reitter, D. (2010). ACT-UP: A Cognitive Modeling Toolkit for Composition, Reuse and Integration. In Proceedings of the 2010 MODSIM conference. Hampton, VA. Lebiere, C., Stocco, A., Reitter, D., & Juvina, I. (2010). High-fidelity cognitive modeling to real-world applications. In Proceedings of the NATO Workshop on Human Modeling for Military Application, Amsterdam, NL, 2010. Reitter, D. & Lebiere, C. (in press). Towards explaining the evolution of domain languages with cognitive simulation. Journal of Cognitive Systems Research. Reitter, D. & Lebiere, C. (in press). A cognitive model of spatial path planning. Journal of Computational and Mathematical Organization Theory. Reitter, D. (to appear). Metacognition and multiple strategies in a cognitive model of online control. Journal of Artificial General Intelligence. Gluck, K. A., Stanley, C. T., Moore, L. R., Reitter, D., & Halbrügge, M. (in press) . Exploration for understanding in model comparisons. Journal of Artificial General Intelligence. Future Work • The Geo Game can be exploited as an experimental platform for years to come – Communication: Alignment of communication standards (e.g., vocabulary) – Knowledge: Information type and acquisition and its influence on its distribution across the network (shared knowledge is not copied knowledge) – Network structure: influence on team performance – Trust and strategy: Adversarial networks • Mixed human/model/agent networks – Bootstrapping methodology for model validation – Information filtering for humans