Download uw-cogsci - MindRACES

Anticipatory Behavior for Object
Recognition and Robot Arm
Control
Modular and Hierarchical Systems, &
Anticipatory Behavior and Control
Department of Cognitive Psychology
University of Würzburg, Germany
Martin V. Butz, Oliver Herbort, Joachim Hoffmann, Andrea Kiesel
MindRACES, First Review
Meeting, Lund, 11/01/2006
1
Related Publications
Date
Journal/con
ference
Title
Author
Date
Journal/con
ference
Title
Author
09/2005
CogWiss
2005
Towards the
Advantages of
Hierarchical
Anticipatory
Behavioral Control
Oliver Herbort,
Martin V. Butz, &
Joachim
Hoffmann
11/2005
IEEE
Transactions
on
Evolutionary
Computation
Gradient Descent Methods
in Learning Classifier
Systems: Improving XCS
Performance in Multistep
Problems
Martin V. Butz,
David E.
Goldberg, & Pier
Luca Lanzi
11/2005
AAAI Fall
Symposium
Towards an Adaptive
Hierarchical
Anticipatory
Behavioral Control
System
Oliver Herbort,
Martin V. Butz, &
Joachim
Hoffmann
07/2005
GECCO
2005 (best
paper
nomination)
Extracted Global Structure
Makes Local Building
Block Processing Effective
in XCS
Martin V. Butz,
Martin Pelikan,
Xavier Llora,
David E.
Goldberg
09/2005
In Book:
Foundations
of Learning
Classifier
Systems
Computational
Complexity of the
XCS Classifier
System
Matin V. Butz,
David E.
Goldberg, & Pier
Luca Lanzi
07/2005
GECCO
2005 (best
paper
nomination)
Kernel-based, Ellipsoidal
Conditions in the RealValued XCS Classifier
System
Martin V. Butz
(in press)
Evolutionary
Computation
Journal
(ECJ)
Automated Global
Structure Extraction
For Effective Local
Building Block
Processing in XCS
Matin V. Butz,
Martin Pelikan,
Xavier Llorà, &
David E.
Goldberg
11/2005
Book
Rule-based Evolutionary
Online
Learning Systems:
A Principled Approach to
LCS Analysis and Design
Martin V. Butz
2
MindRACES, First Review Meeting, Lund, 11/01/2006
Overview
•
•
•
•
Anticipatory Behavioral Control
Scenario involvement
Modular systems
Targeted system integrations
 Learning of environment dynamics
 Object recognition, symbol grounding
 Hierarchical anticipatory arm control
3
MindRACES, First Review Meeting, Lund, 11/01/2006
Anticipatory Behavior Control
(Hoffmann, 1993, 2003)
effect A
action
situation
effect B
Goal
effect C
• Actions are selected, initiated and controlled by
anticipating the desired sensory effects.
4
MindRACES, First Review Meeting, Lund, 11/01/2006
The Big Challenge
Motivations
Epsitemic variables
Class
Global features
Local features
Perception
Joints
Musclecontrol
5
MindRACES, First Review Meeting, Lund, 11/01/2006
Scenario Involvement
• Watching a scene, learning existence and behavior of
objects (Scenario 2)
 Continuous movement
 Blocking of movement
 Object permanence
• Control and manipulation of objects (Scenario 1)
 Cognitive, anticipatory arm control
 Interactive object manipulation
• Finding objects (Scenario 1)
 Search of particular objects (with certain properties)
 Search in room or house
• Behavior triggered by motivations (and possibly
emotions) (Scenario 1)
6
MindRACES, First Review Meeting, Lund, 11/01/2006
Simple Object Recognition
• Scenario 2:
 Watching a scene
 Predicting object behavior /
movement
 Tracking multiple objects
 Learning object
permanence
• Scenario 1:
 Manipulating objects (with
robot arm or directly)
 Anticipatory control with
inverse models (IM)
7
MindRACES, First Review Meeting, Lund, 11/01/2006
Multiple Objects
• Scenario 1:
 Searching objects
 Searching objects of
certain properties
 Partial observability
(fovea, multiple rooms)
 Multiple motivations
for multiple objects
W
F
F
F
F
F
F
W
W
W
W
W
F
8
MindRACES, First Review Meeting, Lund, 11/01/2006
Learning Modules
• XCS predictive modules
 State prediction
 RL prediction
• The ALCS framework
 ACS2 & XACS
 Predictive module
 RL module
• AIS for rule-linkage (OFAI)
• Neural network modules
 Hebbian-learning
 LSTM units (IDSIA)
 Rao-Ballard networks
• Kalman filtering techniques
• Context processing (LUCS)
9
MindRACES, First Review Meeting, Lund, 11/01/2006
Integration of Modules
• Learning environment dynamics
 AIS-based sequences (OFAI)
 Context information for sequences (LUCS)
 Top-down, bottom-up (Kalman filtering-based)
combination of information
• Combination with LSTM-based mechanisms
(IDSIA)
 For object permanence
 Object location out of sight (fovea region)
10
MindRACES, First Review Meeting, Lund, 11/01/2006
A Hierarchical Control Model
desired effects
hand coordinates
IM
IM
processing
(visual, …)
joint angles
IM
muscle length
muscle tension
IM
IM
IM
IM
descending signals
exteroception
interneurons
motorsignals
proprioception
Body / Environment
11
MindRACES, First Review Meeting, Lund, 11/01/2006
Current Cognitive Arm Model
hand coordinates
IM
arm configuration
IM
IM
joint angle
IM
IM
2 , 2 x, y, x, y , elbow
motor torque
1 , 1
q2
q1
MindRACES, First Review Meeting, Lund, 11/01/2006
12
Results: Arm
IM
IM
IM
IM
IM
2 , 2 x, y, x, y , elbow
1 , 1
q2
q1
13
MindRACES, First Review Meeting, Lund, 11/01/2006
Summary
•
Simple simulations
 For object recognition
 Object manipulation
 Development of interactive control structures
•
Modular system combinations





•
LSTM integration into XCS / ACS
Context processing integration into XCS / ACS
Integration of Kalman filtering techniques
Rule-linkage with AIS principles
Hierarchical combinations
Anticipatory, developmental arm control models
 Learning to control an arm
 Learning the existence of objects
 Object recognition
 Object behavior
 Object persistence
14
MindRACES, First Review Meeting, Lund, 11/01/2006