Download Learning to Control Robotic Systems Presented at

Learning to Control Visually Guided
Robotic Systems
Laxmidhar Behera
Professor
Department of Electrical Engineering
Indian Institute of Technology Kanpur
Uttar Pradesh, India – 208016
Topics
• Kohonen SOM, Fuzzy SOM, and TS Fuzzy
Model based Visual Servoing
• Adaptive Critic based Visual Servoing
• Visual Tracking of Humans
• Imitation Learning
• Quantum Learning – Application to Brain wave
Controlled Robotics
• Evolutionary Controllers
A SELF-REPLICATING MACHINE
 A self-replicating machine is a construct that is capable
of autonomously manufacturing a copy of itself using
raw materials taken from its environment, thus
exhibiting self-replication in a way analogous to that
found in nature.
 A detailed conceptual proposal for a physical nonbiological self-replicating system was first put forward
by mathematician John von Neumann in lectures
delivered in 1948 and 1949, when he proposed
a kinematic self-reproducing automaton model as
a thought experiment.
The Beginning
Neural Computing
Neuron
• McCulloh Pitts Neuron
• Hodgkin-Huxley
• Spiking Neuron
• Quantum Neuron
Networks
• Feedforward Network
–
–
–
–
–
MLN
RBFN
SVM
FNN
QNN
• Feedback Network
– Hopfield
– DNN
– RQNN
Approximate Dynamic
Programming
Artificial Neural Networks
Dynamic programming prescribes a search tracking backwards from
the final step, rejecting all suboptimal paths from any given point to
the finish, but retaining all other possible trajectories in memory until
the starting point is reached. ADP learns this optimal policy in realtime.
Necessity of Intelligent Controllers for
Nonlinear Systems
Model uncertainties and unmodeled
dynamics necessitate alternative
approaches for controlling the
nonlinear systems
The most successful control
systems on earth are biological
ones
The effective way is to make the
controller intelligent , ie., with
minimum information about the
system, the controller learns the
system dynamics as well as the control
actuation.
Variable Gain Controller using TS Fuzzy model – An
example
System Dynamics:
The design model is chosen as TS fuzzy system
represented with local dynamics as,
Control Strategy:
If A is asymptotically stable and P is a positive
definite matrix satisfying,
Suppose that,
Then the controller
The global nonlinear dynamics is given by,
The design model is represented as a linear system
with nonlinear disturbance
Patchaikani, Kar and Behera , IEEE SMC-B 2006, Kar et al Fuzzy
Sets and Systems, 2012
asymptotically stabilizes the system
Controller gain for a Two link Manipulator
Control Theory in Neural Network
Parameter Optimization
 NN parameter optimization problem is formulated as a control problem
 Weight update laws are derived using Lyapunov Function Approach
 Remarkable Contribution: Adaptive Learning Rate
Cost function:
Parameter update law:
Behera, Kumar and Patnaik, IEEE Tr. On NN 2006
Visual Control Set Up and the Problem
The Visual servo problem is to coordinate joint positions of the arm
through velocity actuation such that the actual visual feature
converges to desired visual features
Visual Manipulation in a Learning Enviroment
•Model based visual servoing requires instantaneous updates
of Image Jacobian (L)
•L depends on visual features, Cartesian depth information of
the object and end-effector and camera transformation matrix
Behera and Nandagopal, IEEE control systems 1999, Patchaikani and Behera Elsevier-RA 2010, Kumar , Patchaikani, Dutta and Behera
Robotica 2008, Kumar, Patel and Behera , Neural Processing Letters 2008, Kar and Behera , Intelligent Service Robotics 2010
SOM Based Control: Redundant
Solutions
Swagat Kumar, Premkumar P., Ashish Dutta and Laxmidhar Behera, Visual Motor Control of a 7DOF Redundant
Manipulator using Redundancy Preserving Learning Network, Robotica, Vol 28, No 6, 2010
Inverse-Forward Adaptive Scheme with SOM based Hint Generator
Effective obstacle Avoidance is demonstrated
with inverse-forward adaptive scheme
•A feed-forward network is used to learn
the approximate forward kinematics
•Inverse kinematic solution is obtained
through a network inversion algorithm
with SOM-SC based hint generator
•Inverse-forward adaptive scheme does
not require accurate forward kinematic
model and hence reduces the number of
training data
•Forward kinematic model can be
updated in real-time which increases
accuracy and introduces adaptability to
environment change.
S Kumar, L Behera, TM McGinnity, Kinematic control of a redundant manipulator using an inverseforward adaptive scheme with a KSOM based hint generator, Robotics and Autonomous Systems,
vol. 58, 2010, page: 622-633
Inverse Jacobian Estimation using Self-Organizing Map
•Kinematic Jacobian from joint space to image space is a nonlinear function of joint
angle, camera parameters and 3-D depth.
•The accurate computation of the Jacobian over the entire workspace is difficult and in
general estimation is performed around a small operating system.
•Self-organizing map gives a holistic approach to learn the inverse Jacobian from the
vision space to joint space over the entire workspace.
•The accurate positioning of the end-effector is achieved with SOM as
•Empirical studies on Cartesian space shows that the linear map of each neuron
approximates the inverse Jacobian
•It is easier to develop a globally Lyapunov stable controller with SOM in conjunction
with a proportional controller
•Weighted Jacobian can also be learned with SOM based on the desired additional task
P K Patchaikani, L. Behera, Visual Servoing of Redundant Manipulator with Jacobian matrix
estimation using self-organizing map, Robotics and Autonomous Systems, vol. 58, 2010 page:978990
Adaptive Distributed Fuzzy PD Controller
Online adaptation algorithm
The online adaptation algorithm is derived
based on the Lyapunov stability condition
• The Lypunov candidate function
• The derivative of the Lypunov
candidate function
• The derivative of the Lyapunov candidate function will be negative definite
Indrazno Sirajuddin, L Behera, TM McGinnity, S Coleman, IEEE Trans Mechatronics, 2013
Adaptive Critic based Redundancy Resolution
A redundant Manipulator reaches the given endeffector position in multiple joint configurations.
The control challenge is to identify the optimal joint
configuration which accomplishes the desired
additional cost
Critic network is used to learn the costate dynamics
Weighted Minimum Norm and Kinematic Limit
Avoidance are achieved and reported
Obstacle Avoidance and Maximization of Manipulability
are under progress
Patchaikani, Behera and Prasad, IEEE conf. SMC 2009, Patchaikani, Behera and Prasad, IEEE Tr. IE (2011)
Avoiding Kinematic Limit with Adaptive Critic Based
Redundancy Resolution
• 4th link is kinematically
constrained
•Sampling Time 200 ms
•Redundancy resolution is
performed for Cartesian
space trajectory
Challenges
•
Computation of inverse kinematic Jacobian
– The existing control algorithm needs to compute inverse Jacobian . The matrix pseudo-inverse
computation is intensive and suffers from numerical instability near singularity. Can the
manipulator be kinematically controlled without numerical inversion of the Jacobian?
•
Redundancy Resolution
– Existing model based schemes perform redundancy resolution with obstacles known a priori
and is usually an off-line scheme, with end-effector motion and obstacle geometry defined in
Cartesian space. Can real-time redundancy resolution be performed as per the demand of
the dynamic situation, through visual feedback with trajectories and constraints are
defined in image space?
•
Computation of image Jacobian
– Once features are selected, the image Jacobian, for a specific object is unique, but has to be
recomputed for another object. The existing methods focus on visual manipulation of known
objects with known image Jacobians. Can the image Jacobian be computed for unknown
objects given a set of Jacobians for a set of known objects? Can this image Jacobian
computation be object-invariant?
Tennis playing Robot
Goal 4
Following the
manipulation of human
demonstrator
Motion Analysis with
VIicon and Path planning
1. Visual motor co-ordination of 29 DOF system
2. Dexterous Manipulation
3. Learning from Experience and Observation
Robotic grasping of novel objects using
Kinect
Introduction:
This work presents a novel solution to identify grasping point in a single RGB image and calculating the 3D
coordinates using kinect depth stream.
Methodology:
Use of Probabilistic Neural Network and supervised learning to train the PNN.
1
2
Use of binomial logistic
.
regression to model probability
.
Use of Maximum likelihood to
.
obtain the parameters of PNN
459
460
Learned 460 parameters
459 features per
for the PNN
patch are calculated
1
2
.
.
.
459
Synthetic images for training
with labeled grasping points
(for 1000 images per object)
Image Processing to calculate the feature vectors for image.
Edge and Texture details are
extracted using NevatiaBabu and Laws’ Mask.
All these filters are
also applied on
scales of 1/3 and 1/9
For a patch, sum
squared energy
is calculated.
459 patches for
a patch are thus
obtained
The probability of a patch being a grasping point is calculated.
For a image (left) the heat map of probability is shown (right)
From patch of highest probability the grasping point is obtained.
Experimental Results
The images show the grasp points detected on various objects.
Visual tracking of non-rigid object
Problem statement : Given a non-holonomic mobile robot having visual & sonar sensors,
how to efficiently track a human in a cluttered environment
Work done
Face detection algorithms are developed
 Leg detection algorithms based on the
laser scan are developed
•Segmentation of human body in 3 parts
•Learn the hue histogram of human torso & legs
Human detection in dynamic environment
•Back project the hue histogram of human torso &
legs in current frame
Problem encountered
•Human silhouette is reconstructed by fusing depth
While tracking a human, human face is
information with back-projected image.
rarely visible to the robot
• A shape analysis algorithm is developed to find the
Leg detection in 2-D map, generated using
“two legs apart pattern” in the vertical projection
laser scan, does not provide any discriminating histogram of the detected foreground.
feature to distinguish between a human leg &
Motion controller design
other objects having leg shaped base e.g. A
shaped ladder
• (x,y) centroid coordinates of human, yc is the
contact point of human with ground in image frame,
φ is the view angle of camera, Ψ is the angle of
Our approach
human with respect to robot, xm and ym is width
Automatic detection of human
and height of the image respectively
• Background modeling and subtraction
•Shadow removal
• Outliers are removed using Aspect ratio filter
horizontal projection histogram
•Human is detected using template matching
 x
  

•Motion controller
  k p
2
xm
v  k p1 ( ym  yc )
Initial automatic
human detection
Capture RGB & depth
image
Background modeling &subtraction
I
AR & HPH
Template matching
Human tracking flowchart
Segmentation & learn histogram
Human detection in
subsequent frame
Backprojection
(BP)
I
Human silhouette construction using
depth & BP image
UKF prediction
Shape analysis
Template matching
UKF update
No
yes
Human detected ?
Human detection in static environment (a) Background image (BI) (b) RGB image (c) Background subtracted (BS)
image (d) new RGB image (e) BS image (f) image after shadow removal (g) ROI image (h) resized ROI image (i) edge
image (j) distance transformed image (k) matched template (l) HPH of foreground blob (m) segmented human
image (n) hue histogram of torso (o) hue histogram of legs
Human detection in dynamic environment (a) RGB image (b) depth image (c) backprojected image
(d) human silhouette (e) VPH of human silhouette (f) detected human image (g) magnified image
of human detection (h) RGB image (i) depth image (j) backprojected image (k) human silhouette
(l) ROI image (m) resized ROI image (n) edge image (o) distance transformed image (p) matched
template (q) detected human image (r) magnified image of human detection
Learning Through Imitations
1. Robot learns to dance
2. Robot learns to play Tennis
3. Robot learns to arrange a dining table/ study table
What is Imitation?
• Imitation is an advanced behavior whereby an individual
observes and replicates another's behavior. Imitation is also a
form of social learning that leads to the "development of
traditions, and ultimately our culture. It allows for the transfer
of information (behaviours, customs, etc.) between individuals
and down generations without the need for genetic inheritance.
• Cognitive Imitation: [Cognitive Imitation in Rhesus
Macaques] (Science, Vol 305, July 16, 2004), Subiaul et al.
defined cognitive imitation as "a type of observational
learning in which a naïve student copies an expert’s use of a
rule."
Robot Imitates a Human
• Research challenges
– How to convey robot intentions ?
– How to infer user intent from visual observation?
– How to integrate human input with autonomous operation
?
– How does robot adapt to a new situation?
– Can a robot learn through cognitive imitation?
Outline
2. Programming
By Demonstration
1. Movement Imitation
3. Robotic Movement Imitation
• Primitives Based Approach (Mataric’)
• Real Time Tracking (“mirror-game”) (Ude et al.)
4. Direct Perception
and Imitation
PbD for Movement Imitation
Primitive-Based Approach
The Problem:
How to convert visual input to motor output?
=
X1
X2
.
.
.
Xn
movement
primitive1
movement
primitive2
...
movement
primitive K
Primitives: Vandana, Thaat, Salaami in Kathak
J1
J2
.
.
.
Jm
=
Movement Imitation Using
Sensory-Motor Primitives
Motor primitives:
Sequences of action that accomplish
a complete goal-directed behavior.
Examples:
1. Move hand in “straight line”,
“parabola” (Felix…).
2. Perform “grasping”, “a tennis
serve”.
Imitation Learning Using
Sensory-Motor Primitives
(Schaal, Ijspeert & Billard, 2003)
Robot Learns through Kinesthetic
Training
Deep Network Model For Grasping
Two Stage Detection
Deep
Network 1
Deep
Network 2
2 Hidden
Layers
2 Hidden
Layers
50 Neurons
each
200
Neurons
each
Results
The network gives us rectangles with most probable grasping region.
Red-blue Rectangle : the most probable graspable region .
Yellow-green Rectangle : possible graspable regions
Stacking Operation
Quantum Neural Network
Saccadic Movement of Eye Explained
Laxmidhar Behera, Indrani Kar and Avshalom C. Elitzur, Recurrent Quantum Neural Network Model to
Describe Eye Tracking of Moving Target, Foundations of Physics Letters, Vol. 18, No. 4, 357-370, 2005
Laxmidhar Behera, Indrani Kar and A.C. Elitzur, Chapter 9: Recurrent Quantum Neural Network and
Its Applications, in The Emerging Physics of Consciousness, Jack Tuszynski (Ed), Springer Verlag, 2005
Brain Wave Controlled Robotic
Systems
RQNN
RQNN for
for signal
signal
pre-processing
pre-processing
Feature
Feature
Extraction
Extraction
Signal
Signal Acquisition
Acquisition
Classification
Classification
LEFT
LEFT or
or RIGHT
RIGHT
hand
hand imagery
imagery
Interface
Interface
Visual
Visual
Feedback
Feedback
PostPostprocessing
processing
Vaibhav Gandhi, Girijesh Prasad, Damien Coyle, Laxmidhar Behera, TM McGinnity,
Quantum Neural Network Based EEG Filtering for a Brain Computer Interface,
57
IEEE Trans Neural Networks and Learning Systems, 2013
RQNN filtered EEG and iAUI for
physical mobile robot control
Target
User
Robot
Exo-skeleton Design with Vicon
• The brain signal control of the hand exoskeleton is under
progress.
• A more versatile thumb exoskeleton can be designed using the
Vicon data of the thumb.
Control of Exo-skeleton
• As the human finger cannot be
modeled using single revolute joint, 4bar mechanism is designed optimally
to model each joint.
• The exoskeleton is actuated using the
surface EMG signals extracted from
the subject’s healthy forearm part
through the muscle computer
interface.
• The designed hand exoskeleton tracks
the human finger motion successfully.
Felix Orlando, Ashish Dutta, Anupam Saxena, Laxmidhar Behera and T Shibata,
Manipulability Analysis of Human Thumb, Index and Middle Finger in Cooperative 3D
Rotational Movements of a Small Object, Robotica, Volume 31 / Issue 05 / August 2013, pp 797-809
Formation Control
Conclusion
• Soft computing approaches are data driven.
• Learning the model from data is a scientific
art.
• Learning works as different robotic systems
are controlled in real-time as presented in this
lecture.
• Emergence and self-organization are big
challenges that soft-computing approaches
are confronted with.
Thank You