Survey
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
CREDITS Rasmussen, UBC (Jim Little), Seitz (U. of Wash.), Camps (Penn. State), UC, UMD (Jacobs), UNC, CUNY Computer Vision : CISC 4/689 Multi-View Geometry Relates • 3D World Points • Camera Centers • Camera Orientations Computer Vision : CISC 4/689 Multi-View Geometry Relates • 3D World Points • Camera Centers • Camera Orientations • Camera Intrinsic Parameters • Image Points Computer Vision : CISC 4/689 Stereo scene point image plane optical center Computer Vision : CISC 4/689 Stereo • Basic Principle: Triangulation – Gives reconstruction as intersection of two rays – Requires • calibration • point correspondence Computer Vision : CISC 4/689 Stereo Constraints p’ ? p Given p in left image, where can the corresponding point p’ in right image be? Computer Vision : CISC 4/689 Stereo Constraints M Image plane Y1 Epipolar Line p p’ Y2 Z1 O1 X2 X1 Focal plane O2 Epipole Computer Vision : CISC 4/689 Z2 Stereo • The geometric information that relates two different viewpoints of the same scene is entirely contained in a mathematical construct known as fundamental matrix. • The geometry of two different images of the same scene is called the epipolar geometry. Computer Vision : CISC 4/689 Stereo/Two-View Geometry • • The relationship of two views of a scene taken from different camera positions to one another Interpretations – “Stereo vision” generally means two synchronized cameras or eyes capturing images – Could also be two sequential views from the same camera in motion • Assuming a static scene http://www-sop.inria.fr/robotvis/personnel/sbougnou/Meta3DViewer/EpipolarGeo Computer Vision : CISC 4/689 3D from two-views There are two ways of extracting 3D from a pair of images. • Classical method, called Calibrated route, we need to calibrate both cameras (or viewpoints) w.r.t some world coordinate system. i.e, calculate the so-called epipolar geometry by extracting the essential matrix of the system. • Second method, called uncalibrated route, a quantity known as fundamental matrix is calculated from image correspondences, and this is then used to determine the 3D. Either way, principle of binocular vision is triangulation. Given a single image, the 3D location of any visible object point must lie on the straight line that passes through COP and image point (see fig.). Intersection of two such lines from two views is triangulation. Computer Vision : CISC 4/689 Mapping Points between Images • What is the relationship between the images x, x’ of the scene point X in two views? • Intuitively, it depends on: – The rigid transformation between cameras (derivable from the camera matrices P, P’) – The scene structure (i.e., the depth of X) • Parallax: Closer points appear to move more Computer Vision : CISC 4/689 Example: Two-View Geometry x3 x2 x’3 x1 x’2 x’1 courtesy of F. Dellaert Is there a transformation relating the points Computer Vision : CISC 4/689 xi to x’i ? Epipolar Geometry • Baseline: Line joining camera centers C, C’ • Epipolar plane ¦: Defined by baseline and scene point X Computerbaseline Vision : CISC 4/689 from Hartley & Zisserman Epipolar Lines • Epipolar lines l, l’: Intersection of epipolar plane ¦ with image planes • Epipoles e, e’: Where baseline intersects image planes – Equivalently, the image in one view of the other camera center. C’ C Computer Vision : CISC 4/689 from Hartley & Zisserman Epipolar Pencil • As position of X varies, epipolar planes “rotate” about the baseline (like a book with pages) – This set of planes is called the epipolar pencil • Epipolar lines “radiate” from epipole—this is the pencil of epipolar lines Computer Vision : CISC 4/689 from Hartley & Zisserman Epipolar Constraint • • • Camera center C and image point define ray in 3-D space that projects to epipolar line l’ in other view (since it’s on the epipolar plane) 3-D point X is on this ray, so image of X in other view x’ must be on l’ In other words, the epipolar geometry defines a mapping in the other x ! l’, of points in one image to lines x’ C’ C from Hartley & Zisserman Computer Vision : CISC 4/689 Example: Epipolar Lines for Converging Cameras Left view Right view Intersection of epipolar lines = Epipole ! Indicates direction of other camera Computer Vision : CISC 4/689 from Hartley & Zisserman Special Case: Translation Parallel to Image Plane Note that epipolar lines are parallel and corresponding points lie on correspondVision : CISC 4/689 ing epipolar lines (the latter is Computer true for all kinds of camera motions) From Geometry to Algebra P p p’ O’ O Computer Vision : CISC 4/689 From Geometry to Algebra P p p’ O’ O Computer Vision : CISC 4/689 Rotation arrow should be at the other end, to get p in p’ coordinates Linear Constraint: Should be able to express as matrix multiplication. Computer Vision : CISC 4/689 Review: Matrix Form of Cross Product Computer Vision : CISC 4/689 Review: Matrix Form of Cross Product Computer Vision : CISC 4/689 Matrix Form Computer Vision : CISC 4/689 The Essential Matrix If calibrated, p gets multiplied by Intrisic matrix, K Computer Vision : CISC 4/689 The Fundamental Matrix F • Mapping of point in one image to epipolar line in other image x ! l’ is expressed algebraically by the fundamental matrix F line point = Fx Since x’ is on l’, by the point-on-line definition we know that x’T l’ = 0 Substitute l’ = Fx, we can thus relate corresponding points in the camera pair (P, P’) to each other with the following: x’T Fx = 0 • Write this as l’ • • Computer Vision : CISC 4/689