Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
How to Motivate Machines to Learn and Help Humans in Making Water Decisions? Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University, USA www.ent.ohiou.edu/~starzyk EE141 Outline Embodied Intelligence (EI) Embodiment of Mind EI Interaction with Environment How to Motivate a Machine Goal Creation Hierarchy Goal Creation Experiment Promises of EI To economy To society EE141 Intelligence AI’s holy grail From Pattie Maes MIT Media Lab “…Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember..” from Principles of Neural Science by E. R. Kandel et al. E. R. Kandel won Nobel Price in 2000 for his work on physiological basis of memory storage in neurons. “…The question of intelligence is the last great terrestrial frontier of science...” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted to brain research EE141 Traditional AI Abstract intelligence Embodied Intelligence attempt to simulate “highest” human faculties: knowledge is implicit in the fact that we have a body – language, discursive reason, mathematics, abstract problem solving Environment model Condition for problem solving in abstract way “brain in a vat” EE141 Embodiment – embodiment is a foundation for brain development Intelligence develops through interaction with environment Situated in a specific environment Environment is its best model Design principles of intelligent systems from Rolf Pfeifer “Understanding of Intelligence”, 1999 EE141 Interaction with complex environment cheap design ecological balance redundancy principle parallel, loosely coupled processes asynchronous sensory-motor coordination value principle Agent Drawing by Ciarán O’Leary- Dublin Institute of Technology Embodied Intelligence Definition Embodied Intelligence (EI) is a mechanism that learns how to survive in a hostile environment – Mechanism: biological, mechanical or virtual agent with embodied sensors and actuators – EI acts on environment and perceives its actions – Environment hostility is persistent and stimulates EI to act – Hostility: direct aggression, pain, scarce resources, etc – EI learns so it must have associative self-organizing memory – Knowledge is acquired by EI EE141 Embodiment of a Mind Embodiment contains intelligence core and sensory motor interfaces under its control to interact with environment Necessary for development of intelligence Not necessarily constant or in the form of a physical body Boundary transforms modifying brain’s selfdetermination EE141 Embodiment Sensors channel Environment Intelligence core Actuators channel Embodiment of a Mind Brain learns own body’s dynamic Self-awareness is a result of identification with own embodiment Embodiment can be extended by using tools and machines Successful operation is a function of correct perception of environment and own embodiment EE141 EI Interaction with Environment Agent Architecture Reason Short-term Memory Perceive Act RETRIEVAL LEARNING Long-term Memory INPUT OUTPUT Task Environment Simulation or Real-World System EE141 From Randolph M. Jones, P : www.soartech.com How to Motivate a Machine ? The fundamental question is how to motivate a machine to do anything, in particular to increase its “brain” complexity? How to motivate it to explore the environment and learn how to effectively work in this environment? Can a machine that only implements externally given goals be intelligent? If not how these goals can be created? EE141 How to Motivate a Machine ? I suggest that hostility of environment motivates us. It is the pain that moves us. Our intelligence that tries to minimize this pain motivates our actions, learning and development We need both the environment hostility and the mechanism that learns how to reduce inflicted by the environment pain I propose based on the pain mechanism that motivates the machine to act, learn and develop. So the pain is good. Without the pain there will be no intelligence. Without the pain there will be no motivation to develop. EE141 Pain-center and Goal Creation Dual pain level Pain increase Sensor (-) Simple Mechanism Creates hierarchy of values Leads to formulation of complex goals Reinforcement : • Pain increase • Pain decrease Forces exploration + (+) Environment (+) (-) Pain level Wall-E’s goal is to keep his plants from dying EE141 (-) - (+) Motor Pain decrease Excitation Primitive Goal Creation faucet refill garbage w. can sit on water tank Dual pain Dry soil EE141 + Pain Primitive level open Abstract Goal Creation The goal is to reduce the primitive pain level Abstract goals are created to reduce abstract pains in order to satisfy the primitive goals Abstract pain center Sensory pathway Motor pathway (perception, sense) (action, reaction) faucet “water can” – sensory input to abstract pain w. can center Activation Stimulation Inhibition Reinforcement Echo Need Expectation EE141 open - Dry soil + Abstract pain water Dual pain Level II Level I + Pain Primitive Level Abstract Goal Hierarchy Sensory pathway (perception, sense) A hierarchy of abstract goals is created - they satisfy the lower level goals Motor pathway (action, reaction) tank refill - + faucet open - Activation Stimulation Inhibition Reinforcement Echo Need Expectation Dry soil EE141 Level II + w. can water - Level III Level I + Primitive Level GCS vs. Reinforcement Learning States Policy Desired action &state Pain Critic States Value Function action GCS Sensory pathway Action decision Motor pathway reward Environment Gate control Environment Action Actor-critic design Goal creation system Case study: “How can Wall-E water his plants if the water resources are limited and hard to find?” EE141 Goal Creation Experiment SENSORY MOTOR INCREASES DECREASES 1 water can water the plant moisture water in can 8 faucet open water in can water in tank 15 tank refill water in tank reservoir water 22 pipe open reservoir water lake water 29 rain fall lake water - PAIR # Sensory-motor pairs and their effect on the environment EE141 Results from GCS scheme Dry soil pain 4 2 0 0 100 200 400 500 600 300 400 500 600 300 400 500 600 300 400 500 600 300 400 500 600 300 No water in can pain 2 1 0 0 100 200 100 200 100 200 100 200 pain 2 1 0 0 pain 1 No water in reservoir 0.5 0 0 pain 4 No water in lake 2 0 0 EE141 No water in tank GCS vs. Reinforcement Learning Averaged performance over 10 trials: GCS: Primitive pain pain 1 0.5 0 0 100 200 400 500 600 400 500 600 400 500 600 Lack of food RL: 1 pain 300 30 0.5 20 0 0 100 200 300 10 Lack of money 0.4 0 pain 0 100 200 300 0.2 Machine using GCS learns to control all abstract pains and 0 maintains the primitive pain 0 100 200 300 signal on400a low level 500 in Lack of bank savingsconditions. demanding environment 0.4 EE141 600 Goal Creation Experiment Goal Scatter Plot 40 35 30 Goal ID 25 20 15 10 5 0 0 100 200 300 400 Discrete time 500 600 Action scatters in 5 CGS simulations EE141 Goal Creation Experiment Pain Pain Pain Pain Pain Primitive pain – dry soil 0.5 0 0.2 0.1 0 0.2 0.1 0 0.2 0.1 0 0.1 0.05 0 0 100 200 300 400 Lack of water in can 500 600 0 100 200 300 400 Lack of water in tank 500 600 0 100 200 300 400 Lack of water in reservoir 500 600 0 100 200 300 400 Lack of water in lake 500 600 0 100 200 300 Discrete time 500 600 400 The average pain signals in 100 CGS simulations EE141 Promises of embodied intelligence To society Advanced use of technology – Robots – Tutors – Intelligent gadgets Intelligence age follows – Industrial age – Technological age – Information age Society of minds – Superhuman intelligence – Progress in science – Solution to societies’ ills To industry Technological development New markets Economical growth EE141 ISAC, a Two-Armed Humanoid Robot Vanderbilt University Biomimetics and Bio-inspired Systems Mission Complexity Impact on Space Transportation, Space Science and Earth Science 2002 2010 2020 2030 Embryonics Self Assembled Array Space Transportation Memristors Biologically inspired aero-space systems Sensor Web Extremophiles Mars in situ life detector Brain-like computing Skin and Bone Self healing structure and thermal protection systems EE141 Biological nanopore low resolution Artificial nanopore high resolution DNA Computing Biological Mimicking Sounds like science fiction EE141 If you’re trying to look far ahead, and what you see seems like science fiction, it might be wrong. But if it doesn’t seem like science fiction, it’s definitely wrong. From presentation by Feresight Institute Questions? EE141