Download Presentation

Constrained Parametric Min-Cuts
for
Automatic Object Segmentation
SasiKanth Bendapudi
Yogeshwar Nagaraj
S
What is a ‘Good Segmentation’?
http://www.eecs.berkeley.edu/Research/Projects/CS/vision/groupin
g/resources.html
“Geometric context from a single image”, Hoiem et al., ICCV 2005
“Using Multiple Segmentations to Discover Objects and their Extent in Image
Collections”, Russel et al., CVPR 2006
“Improving Spatial Support for Objects via Multiple Segmentations”,
Malisiewicz & Efros, BMVC 2007
“Towards Unsupervised Whole-Object Segmentation: Combining Automated
Matting with Boundary Detection”, Stein & Hebert, CVPR 2008
The Paper
S Figure-Ground segmentation
S Solve CPMC by minimizing the objective function using
various seeds and parameters
S Reject redundancies and obvious negatives based on
segment energies and similarities
S Learn the characteristics of a ‘Figure’ segment to
qualitatively assess the remaining segments
Objective Function
if
Objective Function
if
Objective Function
if
Synthetic Example
Synthetic Example
Initialization
S Foreground
S Regular 5x5 grid geometry
S Centroids of large N-Cuts regions
S Centroids of superpixels closest to grid positions
S Background
S Full image boundary
S Horizontal boundaries
S Vertical boundaries
S All boundaries excluding the bottom one
Performance broadly invariant to different initializations
Fast Rejection
Large set of initial segmentations (~5500)
Low Energy
High Energy
~2000 segments with the lowest energy
Cluster segments based on spatial overlap
Lowest energy member of each cluster (~154)
Segment Ranking
S Model data using a host of features
S Graph partition properties
S Region properties
S Gestalt properties
S Train a regressor with the largest overlap ground-truth
segment using Random Forests
S Diversify similar rankings using Maximal Marginal Relevance
(MMR)
Graph Partition Properties
S Cut – Sum of affinities along segment boundary
S Ratio Cut – Sum along boundary divided by the number
S Normalized Cut – Sum of cut and affinity in foreground
and background
S Unbalanced N-cut – N-cut divided by foreground affinity
S Thresholded boundary fraction of a cut
Region Properties
S
Area
S
Convex Area
S
Perimeter
S
Euler Number
S
Relative Centroid
S
Diameter of Circle with the same
area of the segment
S
Bounding Box properties
S
Percentage of bounding box
covered
S
Absolute distance to the center of
the image
S
Fitting Ellipse properties
S
Eccentricity
S
Orientation
Gestalt Properties
S
Inter-region texton similarity
S
Intra-region texton similarity
S
Inter-region brightness similarity
S
Intra-region brightness similarity
S
Inter-region contour energy
S
Intra-region contour energy
S
Curvilinear continuity
S
Convexity – Ratio of foreground area to convex hull area
Feature Importance
Feature Importance
Feature Importance
What has been modeled?
Databases
S Weizmann database
S
F-measure criterion
F=
2RP
R+P
S MSR-Cambridge database & Pascal VOC2009
S
Segmentation covering
C(S ', S) =
1
| R | *max{O(R, R')}
å
R'ÎS '
N RÎS
Performance
Test of the algorithm
S Berkeley segmentation dataset
S Complete pool of images collected
S Ranked using the ranking methodology
S Top ranks evaluated to test the ranking procedure
S How well does the algorithm perform?
Berkeley Database
Rank 269!
Berkeley Database
Rank 142!
Berkeley Database
Rank 98!
Berkeley Database
S Compute the Segment Covering score for the top 40
segments of each image in the database
Database
Segment Covering Score (Top 40)
BSDS
0.52
MSR Cambridge
0.77
Pascal VOC
0.63
Database
Segment Covering Score
(All segments)
BSDS
0.61
MSR Cambridge
0.85
Pascal VOC
0.78
Conclusion
S Does Constrained Parametric Min-Cuts work well?
S Yes
S Does Fast Rejection work well?
S Yes
S Does Segment Ranking work well?
S I don’t think so
Interesting follow up
Image Segmentation by Figure-Ground Composition into Maximal Clique, Ion,
Carreira, Sminchisescu, ICCV 2011
Interesting follow up
S Obtain pool of FG segmentations from CPMC
S Define tiling and a probabilistic model for the same
S Represent the probabilistic models using mid-level features
S Compute and rank various tilings by implementing discrete
searches from each of the nodes
Image Segmentation by Figure-Ground Composition into Maximal Clique, Ion,
Carreira, Sminchisescu, ICCV 2011
Interesting follow up
Image Segmentation by Figure-Ground Composition into Maximal Clique, Ion,
Carreira, Sminchisescu, ICCV 2011
Questions?