Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Efficient Sampling Methods for Robust Estimation Problems Wei Zhang and Jana Kosecka Department of Computer Science Computer Vision and Robotics Laboratory George Mason University Motivational Problems • Localization • How to enhance navigational capabilities of robots/people in urban areas • Where am I ? – Localization problem GPS - global coordinates - longitude/latitude Relative pose with respect to known landmarks Given database of location views Recognize the most likely location Compute relative pose of the camera with respect to the reference view First part of the talk • Wide base line correspondences in natural environment are difficult to obtain Robust Estimation Problems in Computer Vision • Motion estimation problems - estimating rigid body motion (essential, fundamental matrix, homography) • Range estimation problems • Data are corrupted with outliers - mismatches due to errorneous correspondence and/or tracking errors - range data outliers • Traditional means for tackling the presence of outliers - sampling based methods (RANSAC) - use of robust objective function Sampling RANSAC-like methods • • Original algorithm developed by Fischler and Bolles RANSAC algorithm 1. Generate number of model hypothesis by sampling minimal number of points needed to estimate the model 2. For each hypothesis compute the residual of all datapoints with respect to the model 3. Points with the (residual)^2 < th are classified as inliers and the rest as outliers 4. Find the hypothesis with maximal support 5. Re-estimate the model using all inliers Theoretical Analysis • Given m samples and p points per sample probability of an outlier free sample is • Given the knowledge of the percentage of the outliers in the data and the desired probability of finding at least one outlier free sample the number of samples which needs to be generated can be computed as Practice • For fundamental matrix computation using 8-point alg. 766 samples is needed for 95% confidence of outlier free sample 1177 samples - 99% confidence 766 samples – 95% confidence • Theoretical estimates are too optimistic • In practice around 5000 samples needed • Once the hypothesis are generated by sampling - evaluation process is very expensive – each data point needs to be evaluated Drawbacks • Drawbacks of currently existing methods - require large number of samples - need to know the percentage of outliers - require threshold for determining which points are inliers and which outliers - additional efficiency concerns are related to the time consuming process of evaluation of individual hypotheses • Improvements to standard approaches [Torr’99, Murray’02, Nister’04, Matas’05, Sutter’05 and many others] • Mostly improvement in stopping sampling criteria, hypothesis evaluation criteria, efficient search though hypothesis evaluation space • Hypothesis evaluation idea remained unchanged Robust Objective function • Techniques using robust statistics - use of robust objective function, nonlinear minimization • Robust estimators have typical cut-off point below 50% of outliers [MeerStewart’99, MeerComaniciu’00] • LMeds estimators [Rouseaw’95, Torr, Black] Our Approach • Observation: the final step of the sampling based methods is for classification of points as inliers and outliers • We instead of classifying points based on finding a good hypothesis and evaluating points with respect to that hypothesis, we propose to classify the points directly • Basic observation – analyze the distribution of the residuals for each datapoint with respect to all hypotheses • Example: Example • Distribution of residuals of one datapoint with respect to all generated hypothesis (line fitting example, 20% of outliers inlier outlier • Distributions of outliers and inliers are qualitatively different • Compute n-th order statistics and use it for classification Example • Inlier/outlier distributions with 50% outliers inlier outlier • Computing n-th order statitics of these distributions will enable us to disambiguate inlier and outlier distributions • Formulate outlier and inlier identification as classification problem Skewness and Kurtosis • Compute skewness and kurtosis of each data point skewness inliers outliers kurtosis inliers outliers Skewnews/Kurtosis Features inliers outliers • Classification by k-means clustering in kurtosis space Classification Results • True positive rate of 68% and false positive rate 1% Final Algorithm 1. Randomly generate N hypotheses by sampling 2. For each datapoint compute its residual error with respect to each hypothesis 3. Compute distribution of the errors for each datapoint 4. Compute skewness and kurtosis of each distribution 5. Perform K-means clustering in skewness (kurtosis) space 6. Classify the points as inliers/outliers depending which cluster they belong Alternatively we can assign a probability of each points being and inlier or outlier Fixed number of samples used to compute the histogram (500 for Fundamental matrix estimation) in these experiments Sensitivity Means and 95% confidence intervals for inliers/outliers Experiments • Feature detection – SIFT features • Matching – find the nearest neighbour in the next view with the distance below 0.96 threshold on the cos of the angle between two descriptors • Due to the repetitive structures often present in natural, street/building scenes, wide baseline matching contains many outliers (sometimes not problem for recognition) Experimental Results 776 matches, 50% outliers No false positives 285 matches, 70% outliers, 36 correctly identified, 1 false positive Conclusions • New inlier identification scheme for RANSAC like robust regression problems • Inliers can be identified directly without evaluating each hypothesis • Up to 70% of inliers can be tolerated, with only 500 samples (depends on the estimation problem and objective function surface sensitivity) – experiments for Fundamental matrix and homography estimation • References (submitted to ECCV 2006) Novel inlier identification scheme for robust estimation problems. W. Zhang and J. Kosecka Nonparametric estimation of multiple structures with outliers. W. Zhang and J. Kosecka Robust Estimation of Multiple Structure What does the distribution of residuals reveal in the presence of multiple structures ? Line fitting examples Distribution of residuals for one datapoint In case of inliers # of peaks corresponds to # of models Distribution of residuals • Observation – hypothesis generated by sampling form clusters in the hypothesis spaces, yielding the similar residual errors • These correspond to the peaks in histogram of residuals for each datapoint outliers 83% for each line peaks identification more difficult Mode detection • possibility – mean shift algorithm – difficulties with choosing the bandwith parameter (window size) Peak detection algorithm 1. Smooth the histogram and located local maxima and minima 2. Remove spurious weak modes so only one valley is present between the two modes 3. Choose the weakest mode, if distinct enough add it to the list of modes, else remove Calculating the number of models • Due to the difficulties of peak detection – different number of peaks is detected in different distributions • Median – reliable estimate of # of models Example: # of models = 3, estimates obtained from distributions for all datapoints Identifying the model parameters correct line hypotheses spurious line hypotheses 1. Select datapoints with distinctive modes in error distribution 2. Modes generated by various hypotheses (good and bad) yielding same residuals 3. Choose another point, although the residuals are different cluster can now be identified 4. Enables efficient seach for clusters of correct hypotheses Examples – Line Fitting Examples Motion Estimation a) Piecewise planar models - homography estimation b) Multiple motions – purely translational models Dataset • Zurich building database 201 buildings, each with 5 reference images. 115 test images in query database. • Additional campus dataset 68 buildings, 3 grayscale images each. Approach overview • Hierarchical Building Recognition Localized color histogram based screening. Local appearance based decision. • Relative Pose recovery Based on rectangular structure: Rectangular structure extraction. Given single (query) view, recover the relative camera pose (translation up to scale and orientation). With respect to the reference view: Feature matching between query and model view Compute the relative pose between the current and reference view Vanishing point estimation • Vanishing point: a group of parallel lines in the world intersect in the image. • Based on detected line segments, estimate both grouping information and vanishing points locations using EM algorithm. [Kosecka&Zhang, ECCV02] Localized color histograms 4000 3500 5000 3000 4500 3500 4000 3000 3000 2500 3500 2500 2500 3000 2000 2000 2000 2500 1500 2000 1500 1500 1500 1000 1000 1000 1000 500 0 0 500 500 500 0 5 10 15 20 25 30 35 0 0 5 10 15 20 25 30 35 0 5 10 15 20 25 30 35 0 0 5 10 15 20 25 30 35 Localized color histogram indexing Given query view select a variable number of candidate models. The number of models depends on confidence of each individual recognition . Correct recognition results Recognition is not correct, but true model is listed. Local feature based refinement • In this stage we exploit the SIFT keypoints and their associated descriptors introduced by D. Lowe [Lowe04]. • Keypoints of test image are matched to those of the models selected by the first stage. • Voting determines the most likely model - model with highest number of matches Summary of recognition results • Recognition based on localized color histogram alone is fairly good, 90% recognitions are correct, comparable to other appearance based works [Hshao03][Goedeme04]. • When using keypoint matching directly, it takes over 10 minutes to compare a query image with all the reference images. • The localized color histogram uses about 3 seconds to process same number of reference images. • The hierarchical scheme achieves 96% recognition rate, better than previously reported results.