Download daveppt_june06

Activity Recognition
Journal Club
“Neural Mechanisms for the
Recognition of Biological Movements”
Martin Giese, Tomaso Poggio
(Nature Neuroscience Review, 2003)
Objective
• Recognition of complex movements and
actions using a neurophysiologically
plausible and quantitative model
• Biology has already generated a system
that is robust and has high selectivity let’s mimic it.
Biological Intuitions
• Separate dorsal and ventral streams
http://upload.wikimedia.org/wikipedia/commons/thumb/c/c0/Gray722.png/300px-Gray722.png
http://hebb.mit.edu/courses/8.515/lecture3/sld007.htm
Biological Intuitions/Assumptions
• Hierarchical architecture with increasing
complexity along the hierarchy
• Mainly feedforward
• Activities and views are learned
• Interaction between two streams not
necessary for some recognition
Biological Intuitions
Artificial Neural Networks
• Feedforward
tr dv/dt = -v + F(W·u)
• Recurrent
tr dv/dt = -v + F(W·u + M·u)
Form Pathway
• Object Recognition
Riesenhuber, Poggio 2002
Form Pathway
• Simple Cells
– Modeled by Gabor Filters
• Output via Linear threshold function
• Complex Cells
– ‘MAX’ function
– Output via linear threshold function
– Invariant bar detectors
Giese, Poggio, AIM-2002-012, 2002.
Form Pathway
• View/Object Tuned Units
– Radial Basis Function
– u is the vector of the responses of the
significant invariant bar detector
– u0 signifies the preferred input pattern
– C is a diagonal matrix with the elements Cu
that are inversely proportional to the variance
of the l-th component of u in the training set
Giese, Poggio, AIM-2002-012, 2002.
Form Pathway
• Motion pattern neurons
– Leaky integrator
– ty dy/dt + y(t) = S f(un(t))
– Most active when input follows synaptically
encoded sequence
Motion Pathway
• Represents dorsal stream
• Similar hierarchy
– Increasing complexity and invariance
Motion Pathway
• Lowest level
– Optical flow, computed directly
• Second Level
– First Class
• Translational flow (four tuned directions)
– Second Class
• Motion edges (horizontal and vertical)
• MAX operator
• Third Level
– “Snapshot Neurons”
– Modelled by RBF’s
• Fourth Level
– Motion Pattern Neurons
Motion Pathway
Model Parameters
• Simple Cells
– 8 orientations, 2 scales
– s1=10, s2=7, k=0.35
• Motion Pattern & Motion Snapshot Neurons
– ts=150ms
Giese, Poggio, AIM-2002-012, 2002.
Results
Results
Results
Limitations
• Does not address ‘attentional’ effects
• No model for computing optical flow
• Does not address interaction between
form and motion streams
• Form stream does not recognize pointlight motion as per experimental data
Recap
Recap
Example
• Pattern (sn)
• a
• b
• u(:,t+1) = sn(:,t) + w * tanh(u(:,t));
• v(:,t+1) = sum( tanh(u(1:3,t)) );
Example