Download Dynamic Inverse Models in Human-Cyber

Dynamic Inverse Models
in Human-Cyber-Physical Systems
Sam Burden
S. Shankar Sastry
currently: Postdoc in EECS at UC Berkeley
Sep 2015: Asst Prof in EE at UW Seattle
Dean of Engineering at UC Berkeley
Embedded Humans: Provably Correct Decision Making for
Networks of Humans and Unmanned Systems (N00014-13-1-0341)
Human interaction with the physical world

Increasingly mediated by automation
–
–
Augmented by hardware and software
Machines adapt to collaborate and assist
Human-Cyber-Physical Systems (HCPS)
Embedding humans amid automation

Can lead to performance degradation
–
pilot-induced oscillations in rotory/fixed-wing aircraft
McRuer, Krendal 1974; Hess J. Guid. Cont. Dyn. 1997
Pavel et al. Prog. Aero. Sci. 2013
–
overreliance on adaptive cruise control in cars
Rudin-Brown, Parker Trans. Rch. F: Traffic Psych. and Behav. 2004

Requires predictive models for human behavior
Predictable behavior from internal models

Popular paradigm posits pairs of internal models
–
forward model predicts sensory effect of motor action
Sutton, Barto Psych. Rev. 1981; Jordan, Rumehlart Cog. Sci. 1992; Wolpert, et al. Science 1995
–
inverse model computes motor command expected to
yield desired behavior
Kawato Curr. Opin. Neurobio. 1999; Thoroughman, Shadmehr Nature 2000; Conditt, Mussa-Ivaldi PNAS 1999
–
Theoretical and empirical evidence for paired
forward + inverse models
Bhushan, Shadmehr Bio. Cybern. 1999; Sanner, Kosha Bio. Cybern. 1999
Hanuschkin, Ganguli, Hahnloser Front. Neural Circ. 2013; Giret, Kornfeld, Ganguli, Hahnloser PNAS 2014

Parallels in control theory, robotics, AI
–
Internal models, adaptive control, learning
Francis, Wonham Automatica 1976; Sastry, Bodson Prentice Hall 1989; Sutton, Barto, Williams IEEE CSM 1992
Crawford, Sastry UCB EECS 1996; Atkeson, Schaal ICML 1997; Papavassiliou, Russell IJCAI 1999
Instantiating internal models

Forward model ( M : U  Y ): static vs. dynamic
–
static map (linear or nonlinear) y = M(u)
Hanuschkin, Ganguli, Hahnloser Front. Neural Circ. 2013
Giret, Kornfeld, Ganguli, Hahnloser PNAS 2014
–
dynamic map depends on
intermediate state (q, q)
q = f (q, q) + g(q, q)u
y = h(q, q)
Thoroughman, Shadmehr Science 2000
Wolpert, Diedrichsen, Flanagan Nature Neurosci. 2011

Inverse model ( M-1 : Y  U ): hard to define
–
–
static map may fail to be one-to-one or onto
dynamic map may be acausal or need state estimate
Dynamic inverse models in HCPS

Today’s talk: dynamic inverse models from the
perspective of mathematical control theory
1. derivation of dynamic
x 1g = b(x, z )+ a(x, z )u
inverse model
z = q(x, z )
u = (v - b(x, z )) / a(x, z )
2. properties and implications
for design of HCPS
Single input/single output forward model

Consider forward model in control-affine form:
–
–

x in Rn, u in R, y in R
f, g in Cr(Rn, Rn), h in Cr(Rn, R)
x = f (x) + g(x)u
y = h(x)
Suppose model has strict relative degree g in N:
" Î {0,… , g - 2} : Lg L f h º 0 Lg Lgf-1h(x0 ) ¹ 0
–
Expressed in terms of Lie derivatives Lf h(x), Lg h(x):
–
y = Dh(x)[ f (x)+ g(x)u] =: L f h(x)+ Lg h(x)u
intuitively, input affects g-th derivative of output
e.g. g =2 for Lagrangian mechanical systems
–
applicable to interaction with physical world
Transformation of forward model
Forward model: x = f (x) +g(x)u, y = h(x)
 Suppose model has strict relative degree g in N:
g -1
" Î {0,… , g - 2} : Lg L f h º 0 Lg L f h(x0 ) ¹ 0

–

e.g. g =2 for Lagrangian mechanical systems
Then model is linear in new coordinates:
–
There exists z Î C1 (Rn, Rn-g )
such that in coordinates F = (h, L f h,… , Lgf-1h, z ) =: (x, z )
forward model has the form x 1g = b(x , z ) + a(x , z )u
simpler forward model
–
z = q(x , z ), y = x1
Choosing u = (v - b(x, z )) / a(x, z ) yields x 1g =: xg = v
Dynamic inverse model
Forward model: x 1g = v, z = q(x, z ), y = x1
 Given desired output yd, we seek desired input ud
g
g
g
– Since y =x1, there exists unique vd such that y = x1 = y d

–
States z rendered unobservable by input vd!
Note that exact tracking is too stringent
g -1
– need initial cond. x (0) = (yd (0), yd (0),… , yd (0)) =: h(0)
 But it’s easy to achieve exponential tracking
g
g -1
– applying input v = y d - ag -1 (y - yd ) - - a0 (y - yd )
yields " Î {0,… , g } : x 1 (t) - yd (t) £ exp(-c t) x (0) - h(0)
 How does tracking affect unobservable states z ?

Tracking with stable model pair
Forward model: x 1g = b(x, z )+ a(x, z )u, z = q(x, z ), y = x1
 Dynamic inverse model: u = (v - b(x, z )) / a(x, z )
v = ygd - ag -1 (y - yd )g -1 - - a0 (y - yd )
 Theorem: If forward and inverse models are
exponentially stable, then feedforward input
from dynamic inverse of internal model
achieves exponential tracking for physical system.

–
–
Trajectories converge for stable model pairs (M, M-1)
Feedforward input “asymptotically inverts” dynamics
Tracking with stable model pair (M, M-1)
x̂(t)
x(0)
(M,

M-1)
x̂(t)
x '(0)
Theorem implies:
–
–
For stable model pair, trajectories x, x’ converge to x̂
Feedforward input “asymptotically inverts” dynamics
Application to provably-correct
interventions

Suppose human (H) implements inverse model:
–
–
can infer desired task yH from observed input uH
nominal forward model becomes:
x 1g = ygH , z = q(hH , z ;a ), hH := (yH , yH , … , ygH-1 )
–
automation can intervene to improve performance by
minimizing cost function J:RnR using input a
a Î argmin{J(hH , z ) : z = q(hH , z ;a ), a Î A}
–
guarantees performance improvement following
intervention in human-cyber-physical system
Dynamics of humans embedded w/ machines

Today: predictive models for interaction

Future: enhance human ability to interact with
and control the built world
–
–
–
Human-Cyber-Physical
Human Intranet
Cybathlon
Humans are the enabling technology
Appendix
- Extensions and Generalizations
- Properties of dynamic inverse model
- Behavioral repertoire of humans
Extensions and generalizations
Forward model: x 1g = b(x, z )+ a(x, z )u, z = q(x, z ), y = x1
 Dynamic inverse model: u = (v - b(x, z )) / a(x, z )
v = ygd - ag -1 (y - yd )g -1 - - a0 (y - yd )
 Results easily extend to accommodate:

–
–
multiple inputs / multiple outputs
(small) perturbations in dynamics
Sastry Springer 1999
–
approximate input-output linearization
Hauser PhD Thesis 1989; Hauser, Sastry, Kokotovic IEEE TAC 1992; Banaszuk, Hauser SIAM JCO 1996
–
learning / adaptation / estimation of dynamics
Sutton, Barto, Williams IEEE CSM 1992; Papavassiliou, Russell ICJAI 1999
Sastry, Bodson Prentice Hall 1989; Vrabie, Vamvoudakis, Lewis IET 2013
Properties of dynamic inverse model
Forward model: x 1g = b(x, z )+ a(x, z )u, z = q(x, z ), y = x1
 Dynamic inverse model: u = (v - b(x, z )) / a(x, z )
v = ygd - ag -1 (y - yd )g -1 - - a0 (y - yd )
 Property: dynamic inverse model is unique

–
–
Exact tracking input determined by yd
Independent of how internal model is represented or
obtained (e.g. reinforcement learning, adaptive ctrl.)
Sutton, Barto, Williams IEEE CSM 1992; Papavassiliou, Russell ICJAI 1999
Sastry, Bodson Prentice Hall 1989; Vrabie, Vamvoudakis, Lewis IET 2013
–
Impossible to learn if inverse model is unstable
Behavioral repertoire of humans

Too rich to model from first principles
–
Spans computational, algorithmic, & physical “levels of analysis”
Marr, Poggio MIT AI MEMO 1976
–
Influenced by neurophysiological state (cognitive load, hunger)
LaPointe, Stierwalt, Maitland Int. J. Speech-Lang. Pathology 2010; Danziger, Levav, Avnaim-Pesso PNAS 2011

Can reduce dramatically during particular tasks
–
Bernstein posed the “problem of motor redundancy”
Bernstein Pergamon Press 1967.
–
Perhaps instead we “exploit the bliss of motor abundance”
e.g. using synergies, uncontrolled manifolds, optimality
Latash Exp. Brain Rch. 2012; Ting, Macpherson J. Neurophys. 2005; Scholz, Schoner, Exp. Brain Rch. 1999
Todorov, Jordan Nature Neurosci. 2002; Diedrichsen, Shadmehr, Ivry Trends Cog. Sci. 2010
–
For instance, locomotion naturally reduces dimensionality
Burden, Revzen, Sastry IEEE TAC 2015