Download Slide

Automatic 3D modelling of
Architecture
Anthony Dick1
1
Phil Torr2
Department of Engineering
University of Cambridge
Roberto Cipolla1
2
Microsoft Research,
Cambridge
The goal
• Generate 3D models of architectural scenes from
several images automatically
– Including accurate geometry, texture
+
Interactively built using Photobuilder!
Available at http://svr-www.eng.cam.ac.uk/photobuilder/download.html
Automatic 3D modelling of architecture BMVC'00
2
Our approach
• Previous structure from motion algorithms use only
image data
• We integrate image data with prior knowledge of
architecture
– The scene will be piecewise planar
– Walls are likely to intersect at right angles
– Walls are likely to be perpendicular to a common ground
plane
– Walls are likely contain doors and windows which have a
highly constrained shape
Automatic 3D modelling of architecture BMVC'00
3
Model representation
• Scene is modelled as a collection of “wall” planes
• Each wall plane has a plane equation and a
boundary
• Each wall plane may contain offset layers such as
doors, windows
c
•Each offset layer is one
of a collection of
parameterised shapes
b
(x,y)
d
r
a
a
Front view
Automatic 3D modelling of architecture BMVC'00
Overhead view
4
Model estimation
• Structure estimation has 2 parts:
– How many walls are in the scene and what are their
parameters?
– How many offset layers does each wall contain, what shape
are they, and what are their parameters? [ECCV2000]
• Model selection between different shapes
Automatic 3D modelling of architecture BMVC'00
5
Previous work
• Manually defined homography
• Initialise offset layer estimates using dense correspondence
• Fit 4 different shape models to each region
– Use Bayesian model selection criterion to select best shape model
Initial
After model fitting + selection
Automatic 3D modelling of architecture BMVC'00
6
What’s new
• Extension to scenes with multiple wall planes
• Automatic segmentation of walls
Automatic 3D modelling of architecture BMVC'00
7
Initialisation
• Feature-based structure from motion
– Track points
– Estimate pairwise epipolar geometry
– Camera self-calibration [Mendonca CVPR99]
Automatic 3D modelling of architecture BMVC'00
8
Plane segmentation
• Recursive RANSAC plane extraction
• Assume all planes perpendicular to common ground
plane
• Project onto ground plane
• Derive plane boundaries perpendicular and parallel
to ground plane
Reconstruction projected onto
ground plane
Automatic 3D modelling of architecture BMVC'00
9
Optimising the planar model
• Gradient descent search
– Cost function: SSE of model projected into each image
• Parameters to vary:
– Ground plane orientation
– Boundary and intersection points of each plane
Before fitting
After fitting
Automatic 3D modelling of architecture BMVC'00
10
Evaluating the cost function
•
•
•
•
•
Search requires many evaluations of cost function
This is expensive
Green’s Theorem: R f x, y   R n  A ,   A  f
Sum vector field A around region boundary
Cache results for best efficiency
e1
e1
e2
e2
e3
e4
R1
R2
Cost of R2, L(R2) = L(R1) – L(e1) – L(e2) + L(e3) + L(e4)
Automatic 3D modelling of architecture BMVC'00
11
Results
• Courtyard corner
Automatic 3D modelling of architecture BMVC'00
12
The “castle” sequence
• Images from http://www.esat.kuleuven.ac.be/~pollefey/demos/castle.html
Automatic 3D modelling of architecture BMVC'00
13
Future work
• Use of lines to initialise offset layers
– Join nearby lines into rectangles
– Use knowledge of window height/width ratios
• More extensive and structured set of shapes
– Rather than simply testing each possibility
– Possible use of architectural shape grammars
• And in conclusion…
– General framework of combining prior knowledge and image
data is a useful one
– Challenge is to formulate prior knowledge usefully
Automatic 3D modelling of architecture BMVC'00
14
The Bayesian framework
evidence
Bayes Rule:
Pr Modeli | Data, prIor  
Evidence:
model prior
Pr( D | Mi I) Pr( Mi | I)
Pr( D | I)
constan
t
Pr D | Mi , I    Pr D | Mi θi I Pr θi | Mi I  dθi
θi
likelihood
prior
Model parameters q: Wall planes: plane boundary, plane equations
Offset layers: Height, width, x, y position
Automatic 3D modelling of architecture BMVC'00
15
Planar parallax
• Having optimised for main walls, want to fit doors,
windows etc.
• This is the same problem tackled earlier, but
initialisation is now more difficultx
y
~
~

Ax, y   ~x 0 f ( x , y )
– Each plane covers less of the image
– There may be some fitting errors

~
y 0
f ( x, y )

T
• Manually set number of primitives on each plane
– Assumes evenly spaced, vertically centred
– Fits each model from this initialisation
Automatic 3D modelling of architecture BMVC'00
16