Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Automatic 3D modelling of Architecture Anthony Dick1 1 Phil Torr2 Department of Engineering University of Cambridge Roberto Cipolla1 2 Microsoft Research, Cambridge The goal • Generate 3D models of architectural scenes from several images automatically – Including accurate geometry, texture + Interactively built using Photobuilder! Available at http://svr-www.eng.cam.ac.uk/photobuilder/download.html Automatic 3D modelling of architecture BMVC'00 2 Our approach • Previous structure from motion algorithms use only image data • We integrate image data with prior knowledge of architecture – The scene will be piecewise planar – Walls are likely to intersect at right angles – Walls are likely to be perpendicular to a common ground plane – Walls are likely contain doors and windows which have a highly constrained shape Automatic 3D modelling of architecture BMVC'00 3 Model representation • Scene is modelled as a collection of “wall” planes • Each wall plane has a plane equation and a boundary • Each wall plane may contain offset layers such as doors, windows c •Each offset layer is one of a collection of parameterised shapes b (x,y) d r a a Front view Automatic 3D modelling of architecture BMVC'00 Overhead view 4 Model estimation • Structure estimation has 2 parts: – How many walls are in the scene and what are their parameters? – How many offset layers does each wall contain, what shape are they, and what are their parameters? [ECCV2000] • Model selection between different shapes Automatic 3D modelling of architecture BMVC'00 5 Previous work • Manually defined homography • Initialise offset layer estimates using dense correspondence • Fit 4 different shape models to each region – Use Bayesian model selection criterion to select best shape model Initial After model fitting + selection Automatic 3D modelling of architecture BMVC'00 6 What’s new • Extension to scenes with multiple wall planes • Automatic segmentation of walls Automatic 3D modelling of architecture BMVC'00 7 Initialisation • Feature-based structure from motion – Track points – Estimate pairwise epipolar geometry – Camera self-calibration [Mendonca CVPR99] Automatic 3D modelling of architecture BMVC'00 8 Plane segmentation • Recursive RANSAC plane extraction • Assume all planes perpendicular to common ground plane • Project onto ground plane • Derive plane boundaries perpendicular and parallel to ground plane Reconstruction projected onto ground plane Automatic 3D modelling of architecture BMVC'00 9 Optimising the planar model • Gradient descent search – Cost function: SSE of model projected into each image • Parameters to vary: – Ground plane orientation – Boundary and intersection points of each plane Before fitting After fitting Automatic 3D modelling of architecture BMVC'00 10 Evaluating the cost function • • • • • Search requires many evaluations of cost function This is expensive Green’s Theorem: R f x, y R n A , A f Sum vector field A around region boundary Cache results for best efficiency e1 e1 e2 e2 e3 e4 R1 R2 Cost of R2, L(R2) = L(R1) – L(e1) – L(e2) + L(e3) + L(e4) Automatic 3D modelling of architecture BMVC'00 11 Results • Courtyard corner Automatic 3D modelling of architecture BMVC'00 12 The “castle” sequence • Images from http://www.esat.kuleuven.ac.be/~pollefey/demos/castle.html Automatic 3D modelling of architecture BMVC'00 13 Future work • Use of lines to initialise offset layers – Join nearby lines into rectangles – Use knowledge of window height/width ratios • More extensive and structured set of shapes – Rather than simply testing each possibility – Possible use of architectural shape grammars • And in conclusion… – General framework of combining prior knowledge and image data is a useful one – Challenge is to formulate prior knowledge usefully Automatic 3D modelling of architecture BMVC'00 14 The Bayesian framework evidence Bayes Rule: Pr Modeli | Data, prIor Evidence: model prior Pr( D | Mi I) Pr( Mi | I) Pr( D | I) constan t Pr D | Mi , I Pr D | Mi θi I Pr θi | Mi I dθi θi likelihood prior Model parameters q: Wall planes: plane boundary, plane equations Offset layers: Height, width, x, y position Automatic 3D modelling of architecture BMVC'00 15 Planar parallax • Having optimised for main walls, want to fit doors, windows etc. • This is the same problem tackled earlier, but initialisation is now more difficultx y ~ ~ Ax, y ~x 0 f ( x , y ) – Each plane covers less of the image – There may be some fitting errors ~ y 0 f ( x, y ) T • Manually set number of primitives on each plane – Assumes evenly spaced, vertically centred – Fits each model from this initialisation Automatic 3D modelling of architecture BMVC'00 16