Download Lecture17 - POSTECH Computer Vision Lab

Tracking Algorithms
• Deterministic methods
CSED441: Introduction to Computer Vision (2015S)
Lecture17: Stochastic Tracking
Bohyung Han
CSE, POSTECH
[email protected]
Given input video and current state, tracking result is always same.
Local search
 Least‐square tracking
 Mean‐shift
 Gradient ascent (or decent) algorithms
• Probabilistic or stochastic methods
Each trial may produce a different tracking result.
Statistical search (e.g., by sampling)
 Kalman filter/extended Kalman filter
 Particle filter
2
Probabilistic Tracking
• Target state is determined through a statistical process.
 For example, the target state is estimated based on density function, sometimes by sampling for target state and corresponding observation. CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Joint Probability and Graphical Model
• Graphical model
 Probabilistic model for which a graph denotes the conditional independence structure between random variables
 Enables a simpler representation of joint probability
 Directed or undirected
, , ,
, ,
,
,
3
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
4
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Markov Property
Sequential Bayesian Filtering
• In stochastic process • Estimation of an unknown probability density function
 The conditional probability distribution of future states of the process depends only upon the present state, not on the sequence of events that preceded it.
 In the discrete time stochastic process, Markov property can be defined as
|
,
|
,…,
 PDF is estimated recursively over time using • Incoming measurement
• Mathematical process model (dynamic model)
 Extension of (static) Bayesian estimation
• Observation changes overtime
)
• In graphical model
: state variable
: observation variable
:
5
posterior
∝
|
likelihood
|
:
6
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Components of Sequential Bayesian Filter
:
∝
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Derivation of Sequential Bayesian Filtering
:
prior
|
:
• Markov assumption for process model
,
process transition probability
,
,
,…,
|
• Joint distribution of all variables
noises
• Observation model
,
7
|
• Conditional independence in observation
• Process model (dynamic model)
|
,…,
,
,…,
,
,…,
|
We should estimate the marginal posterior CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
8
|
:
sequentially.
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Derivation of Sequential Bayesian Filtering
•
•
•
•
:
:
:
,
:
,
:
,
|
:
|
:
:
:
|
:
:
:
Kalman filter
Extended Kalman filter
Unscented Kalman filter
Particle filter





Sequential Importance Sampling (SIS)
Sampling Importance Resampling (SIR)
Regularized particle filter
Auxiliary particle filter
…
:
|
:
,
:
Examples of Sequential Bayesian Filtering
:
normalization constant
9
Examples of Sequential Bayesian Filtering
• They are different in
∝
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Kalman Filter and Its Extension
• Kalman filter is optimal if
 Flexibility of density representation
 Flexibility of dynamic model :
10
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
 The initial posterior is Gaussian
 The process model is linear
 The observation density is Gaussian
|
:
Real‐world problems hardly meet the conditions.
Posterior density
Process
Model
Observation
Model
KF
Gaussian
Linear
Linear
EKF
Gaussian
Approximation
Linear
Approximation
Linear
Approximation
PF
Arbitrary
Arbitrary
Arbitrary
11
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
• Limitations of (extended) Kalman filter
 They are limited to handle non‐linear process models
 They cannot model multi‐modal posterior density functions
Critical limitations for real‐world problems
12
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Monte Carlo Approximation
Particle Filter
• Basic idea
• Most flexible implementation of sequential Bayesian filtering
 Sample based
 The more we draw samples, the more accurate the estimation is.
 Useful when analytical solution is unknown or hard to compute




The posterior is estimated in The posterior is estimated by a sequential manner
the population of samples.
• MC in sequential Bayesian filtering
 Estimating unknown probability distribution using a set of samples
 MC can easily be used to compute |
marginal posterior distribution,  It does not require any assumption on the underlying distribution.
13
Representation of the state with (location, weight) pair of each sample
No restriction of posterior density representation
No restriction of process and measurement models
Also known as Sequential Monte Carlo (SMC)
Actually, there are some other SMC techniques other than PF.
• Advantages
:
 Able to handle multi‐modal (arbitrary) posterior density functions
 Able to handle non‐linear process model
 Very simple implementation
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Posterior Representation
14
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Procedure
• Sequential Importance Sampling (SIS)
probability
posterior density
weighted sample
:
state
transition
:
state
. . .
observation
,
:
Unknown measurement function
15
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
16
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Limitations
Condensation Algorithm
• Degeneracy problem




• Giving more diversity in samples by resampling (SIR)
Most particles have very small and negligible weights.
Most of the weights are concentrated on a few particles.
Most of particles are useless.
Density estimation becomes inaccurate.
resampling
:
state
transition
:
observation
:
17
Resampling
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Particle Filter for Visual Tracking
• Properties
 Identical sample weights
 More sample diversity
 Less degeneracy
18
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Condensation algorithm animation
• Sample a set of particles (samples) from the prior.
• Perform an observation for each particle.
• Obtain the target state
observation
observation
initial sample
However, resampling does not solve the degeneracy p
roblem completely.
sample after prediction
Isard and Blake, 1998
http://homepages.inf.ed.ac.uk/rbf/CVonline/LOCAL_
COPIES/ISARD1/condensation.html
19
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
frame t‐1
frame t
20
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Observation: Color Histogram
Particle Filter for Visual Tracking
• Likelihood computation for each particle
 By histogram comparison
∗,
∝ exp
where
∗
,
∗
1
;
P. Perez, C. Hue, J. Vermaak, and M. Gangnet. Color‐Based Probabilistic Tracking, ECCV, 2002
21
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Characteristics
22
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Tracking Results
• It is practically (almost) impossible to find the proper dynamic model.
 A suggestion: Auto‐Regressive (AR) model
,
~
0,
 Random walk is frequently used.
• Tracking control and observation are independent.
 Any reasonable observation technique can be integrated into the observation model (e.g., histogram comparison)
∗
∝ exp
where
∗
,
1
23
,
∗
;
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
24
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Applications of Particle filter
• Computer vision
Reference
• Tutorial on particle filter
 Visual tracking
 Dynamic parameter estimation
 M.S. Arulampalam, S. Maskell, N. Gordon, T Clapp, "A tutorial on particle filters for online nonlinear/non‐Gaussian Bayesian tracking," IEEE Trans. on Signal Processing 50(2), 174–188, 2002
• Robotics
• Particle filtering for visual tracking
 SLAM: Simultaneous Localization And Mapping
 M. Isard and A. Blake, “Condensation—Conditional Density Propagation for Visual Tracking,” Int’l Journal of Computer Vision 29(1), 5‐28, 1998
 P. Pérez, C. Hue, J. Vermaak and M. Gangnet, “Color‐Based Probabilistic Tracking,” ECCV 2002
• Signal processing
• Financial engineering
 Prediction of stock price
25
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
26
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
Tracking‐by‐Detection
• Combination of tracking and detection techniques
 Exploits the recent advance of object detection techniques
 Typically needs to design online classifiers
 Requires to handling outliers and noises effectively
• Advantages
 More convenient to handle initialization problem
 More robust to abrupt changes in lighting condition and motion
 More rigorous to deal with occlusion and entrance/leaving event
• Examples




Online boosting
STRUCK
TLD (Tracking‐Learning‐Detection)
etc.
27
CSED441: Introduction to Computer Vision
by Prof. Bohyung Han, Spring 2015
28