Download Robust Estimation Problems in Computer Vision

Efficient Sampling Methods for Robust Estimation
Problems
Wei Zhang and Jana Kosecka
Department of Computer Science
Computer Vision and Robotics Laboratory
George Mason University
Motivational Problems
• Localization
• How to enhance navigational capabilities of
robots/people in urban areas
• Where am I ? – Localization problem
GPS - global coordinates - longitude/latitude
Relative pose with respect to known
landmarks
 Given database of location views
 Recognize the most likely location
 Compute relative pose of the camera with
respect to the reference view
First part of the talk
• Wide base line correspondences in natural
environment are difficult to obtain
Robust Estimation Problems in Computer
Vision
• Motion estimation problems
- estimating rigid body motion (essential, fundamental
matrix, homography)
• Range estimation problems
• Data are corrupted with outliers
- mismatches due to errorneous correspondence and/or
tracking errors
- range data outliers
• Traditional means for tackling the presence of outliers
- sampling based methods (RANSAC)
- use of robust objective function
Sampling RANSAC-like methods
•
•
Original algorithm developed by Fischler and Bolles
RANSAC algorithm
1. Generate number of model hypothesis by sampling
minimal number of points needed to estimate the
model
2. For each hypothesis compute the residual of all
datapoints with respect to the model
3. Points with the (residual)^2 < th are classified as
inliers and the rest as outliers
4. Find the hypothesis with maximal support
5. Re-estimate the model using all inliers
Theoretical Analysis
• Given m samples and p points per sample
probability of an outlier free sample is
• Given the knowledge of the percentage of the
outliers in the data and the desired probability of
finding at least one outlier free sample the
number of samples which needs to be generated
can be computed as
Practice
• For fundamental matrix computation
using 8-point alg. 766 samples is needed
for 95% confidence of outlier free sample
1177 samples - 99% confidence
766 samples – 95% confidence
• Theoretical estimates are too optimistic
• In practice around 5000 samples needed
• Once the hypothesis are generated by sampling
- evaluation process is very expensive – each data
point needs to be evaluated
Drawbacks
• Drawbacks of currently existing methods
- require large number of samples
- need to know the percentage of outliers
- require threshold for determining which points
are inliers and which outliers
- additional efficiency concerns are related to the
time consuming process of evaluation of individual
hypotheses
• Improvements to standard approaches [Torr’99,
Murray’02, Nister’04, Matas’05, Sutter’05 and many others]
• Mostly improvement in stopping sampling criteria,
hypothesis evaluation criteria, efficient search though
hypothesis evaluation space
• Hypothesis evaluation idea remained unchanged
Robust Objective function
• Techniques using robust statistics - use
of robust objective function, nonlinear minimization
• Robust estimators have typical cut-off point
below 50% of outliers [MeerStewart’99, MeerComaniciu’00]
• LMeds estimators [Rouseaw’95, Torr, Black]
Our Approach
• Observation: the final step of the sampling based
methods is for classification of points as inliers and
outliers
• We instead of classifying points based on finding a good
hypothesis and evaluating points with respect to that
hypothesis, we propose to classify the points directly
• Basic observation – analyze the distribution of the
residuals for each datapoint with respect to all
hypotheses
• Example:
Example
• Distribution of residuals of one datapoint with respect
to all generated hypothesis (line fitting example, 20% of
outliers
inlier
outlier
• Distributions of outliers and inliers are qualitatively different
• Compute n-th order statistics and use it for classification
Example
• Inlier/outlier distributions with 50% outliers
inlier
outlier
• Computing n-th order statitics of these distributions will
enable us to disambiguate inlier and outlier distributions
• Formulate outlier and inlier identification as classification
problem
Skewness and Kurtosis
• Compute skewness and kurtosis of each data point
skewness
inliers outliers
kurtosis
inliers outliers
Skewnews/Kurtosis Features
inliers
outliers
• Classification by k-means clustering in kurtosis space
Classification Results
• True positive rate of 68% and false positive rate 1%
Final Algorithm
1. Randomly generate N hypotheses by sampling
2. For each datapoint compute its residual error
with respect to each hypothesis
3. Compute distribution of the errors for each datapoint
4. Compute skewness and kurtosis of each distribution
5. Perform K-means clustering in skewness (kurtosis) space
6. Classify the points as inliers/outliers depending
which cluster they belong
Alternatively we can assign a probability of each points
being and inlier or outlier
Fixed number of samples used to compute the histogram
(500 for Fundamental matrix estimation)
in these experiments
Sensitivity
Means and 95% confidence intervals for inliers/outliers
Experiments
• Feature detection – SIFT features
• Matching – find the nearest neighbour in the
next view with the distance below 0.96 threshold
on the cos of the angle between two descriptors
• Due to the repetitive structures often present in natural,
street/building scenes, wide baseline matching contains
many outliers (sometimes not problem for recognition)
Experimental Results
776 matches, 50% outliers
No false positives
285 matches, 70% outliers,
36 correctly identified, 1 false positive
Conclusions
• New inlier identification scheme for RANSAC
like robust regression problems
• Inliers can be identified directly without evaluating each
hypothesis
• Up to 70% of inliers can be tolerated, with
only 500 samples (depends on the estimation
problem and objective function surface sensitivity) –
experiments for Fundamental matrix and homography
estimation
• References (submitted to ECCV 2006)
Novel inlier identification scheme for robust estimation
problems. W. Zhang and J. Kosecka
Nonparametric estimation of multiple structures with outliers.
W. Zhang and J. Kosecka
Robust Estimation of Multiple Structure
What does the distribution of residuals reveal
in the presence of multiple structures ?
Line fitting examples
Distribution of residuals
for one datapoint
In case of inliers # of peaks
corresponds to
# of models
Distribution of residuals
• Observation – hypothesis generated by sampling form
clusters in the hypothesis spaces, yielding the similar
residual errors
• These correspond to the peaks in histogram of residuals for
each datapoint
outliers 83% for each line
peaks identification
more difficult
Mode detection
• possibility – mean shift algorithm – difficulties
with choosing the bandwith parameter (window size)
Peak detection algorithm
1. Smooth the histogram and
located local maxima and
minima
2. Remove spurious weak modes
so only one valley is present
between the two modes
3. Choose the weakest mode, if
distinct enough add it to the
list of modes, else remove
Calculating the number of models
• Due to the difficulties of peak detection –
different number of peaks is detected in different
distributions
• Median – reliable estimate of # of models
Example: # of models = 3, estimates obtained from
distributions for all datapoints
Identifying the model parameters
correct line hypotheses
spurious line hypotheses
1. Select datapoints with distinctive modes in error distribution
2. Modes generated by various hypotheses (good and bad)
yielding same residuals
3. Choose another point, although the residuals are different
cluster can now be identified
4. Enables efficient seach for clusters of correct hypotheses
Examples – Line Fitting
Examples Motion Estimation
a) Piecewise planar models - homography estimation
b) Multiple motions – purely translational models
Dataset
•
Zurich building database
201 buildings, each with 5 reference images. 115 test images in query database.
•
Additional campus dataset
68 buildings, 3 grayscale images each.
Approach overview
• Hierarchical Building Recognition
 Localized color histogram based screening.
 Local appearance based decision.
• Relative Pose recovery
 Based on rectangular structure:
 Rectangular structure extraction.
 Given single (query) view, recover the relative camera pose
(translation up to scale and orientation).
 With respect to the reference view:
 Feature matching between query and model view
 Compute the relative pose between the current and reference view
Vanishing point estimation
• Vanishing point:
a group of parallel lines in the world intersect
in the image.
• Based on detected line segments,
estimate both grouping information and
vanishing points locations using EM
algorithm. [Kosecka&Zhang, ECCV02]
Localized color histograms
4000
3500
5000
3000
4500
3500
4000
3000
3000
2500
3500
2500
2500
3000
2000
2000
2000
2500
1500
2000
1500
1500
1500
1000
1000
1000
1000
500
0
0
500
500
500
0
5
10
15
20
25
30
35
0
0
5
10
15
20
25
30
35
0
5
10
15
20
25
30
35
0
0
5
10
15
20
25
30
35
Localized color histogram indexing
 Given query view select a variable number of candidate
models. The number of models depends on confidence of
each individual recognition .
 Correct recognition results
 Recognition is not correct, but true model is listed.
Local feature based refinement
•
In this stage we exploit the SIFT keypoints and their associated
descriptors introduced by D. Lowe [Lowe04].
•
Keypoints of test image are matched to those of the models selected
by the first stage.
•
Voting determines the most likely model - model with highest
number of matches
Summary of recognition results
• Recognition based on localized color histogram alone is fairly
good, 90% recognitions are correct, comparable to other
appearance based works [Hshao03][Goedeme04].
• When using keypoint matching directly, it takes over 10
minutes to compare a query image with all the reference
images.
• The localized color histogram uses about 3 seconds to process
same number of reference images.
• The hierarchical scheme achieves 96% recognition rate, better
than previously reported results.