* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Linear algebra refresher and transformations
Euclidean vector wikipedia , lookup
Quadratic form wikipedia , lookup
Tensor operator wikipedia , lookup
Jordan normal form wikipedia , lookup
Basis (linear algebra) wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Determinant wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
System of linear equations wikipedia , lookup
Bra–ket notation wikipedia , lookup
Rotation matrix wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Homogeneous coordinates wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Linear algebra wikipedia , lookup
Cartesian tensor wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Matrix calculus wikipedia , lookup
CS 4501: Introduction to Computer Vision Brief Tutorial of Linear Algebra and Transformations Connelly Barnes Slides from Fei Fei Li, Juan Carlos Niebles, Jason Lawrence, Szymon Rusinkiewicz, David Dobkin, Adam Finkelstein, Tom Funkhouser Outline • Vectors and matrices • Operations • Solving linear systems • Representation in Python • Transformation matrices • 2D • Homogeneous coordinates • 3D • Camera transformations Vectors • A column vector 𝐯 ∈ ℝ𝑛×1 𝑣1 𝐯= ⋮ 𝑣𝑛 • A row vector 1×𝑛 𝐯′ ∈ ℝ 𝐯′ = 𝑣1 • Default to column vectors. … 𝑣𝑛 Matrix • A matrix 𝐀 ∈ ℝ𝑚×𝑛 𝑎11 𝐀= ⋮ 𝑎𝑚,1 … … • Convention: indexed by (row, column). • Square matrix: m = n. 𝑎1,𝑛 ⋮ 𝑎𝑚,𝑛 Matrix/Vector Operations • Addition • Scaling • Transpose • Dot product • Multiplication • Inverse • Algebraic properties Matrix Addition • Matrices (and vectors) add elementwise: 𝑎11 𝑎21 𝑎12 𝑏11 + 𝑎22 𝑏21 𝑎11 + 𝑏11 𝑎21 + 𝑏21 𝑏12 = 𝑏22 𝑎12 + 𝑏12 𝑎22 + 𝑏22 Matrix Scaling • Matrix (and vector) product with a scalar multiplies each element: 𝑎11 𝑐 𝑎 21 𝑎12 𝑐 𝑎11 = 𝑎22 𝑐 𝑎21 𝑐 𝑎12 𝑐 𝑎22 Transpose • Transpose swaps the rows and columns of a vector or matrix: 𝑣1 ⋮ 𝑣𝑛 𝑎 𝑑 𝑇 = 𝑣1 𝑏 𝑒 𝑐 𝑓 𝑇 … 𝑣𝑛 𝑎 = 𝑏 𝑐 𝑑 𝑒 𝑓 Dot Product • Inner (dot) product of two vectors multiplies corresponding entries and sums the result: 𝑛 𝐱∙𝐲= 𝑥𝑖 𝑦𝑖 𝐱 ∙ 𝐲 = |𝐱||𝐲|𝑐𝑜𝑠𝜃 𝑖=1 • Also written as: 𝐱 𝑇 𝐲 = 𝑥1 … 𝑥𝑛 𝑦1 ⋮ 𝑦𝑛 x Shortest angle between x and y Matrix Multiplication • Matrix-vector product Av: take dot product between each row of matrix and the column vector: 𝑎11 𝑎21 𝑎12 𝑣1 𝑎11 𝑣1 + 𝑎12 𝑣2 = 𝑎22 𝑣2 𝑎21 𝑣1 + 𝑎22 𝑣2 • Matrix-matrix product AB: entry (i, j) is dot product between row i of A and column j of B. − 𝐫1 − − 𝐫2 − − 𝐫3 − | 𝐜1 | | 𝐜2 | 𝐫1 ∙ 𝐜1 | 𝐜3 = 𝐫2 ∙ 𝐜1 𝐫3 ∙ 𝐜1 | 𝐫1 ∙ 𝐜2 𝐫2 ∙ 𝐜2 𝐫3 ∙ 𝐜2 𝐫1 ∙ 𝐜3 𝐫2 ∙ 𝐜3 𝐫3 ∙ 𝐜3 Matrix Multiplication • If A is shape 𝑚 × 𝑛 and B is shape 𝑛 × 𝑘 then AB is shape 𝑚 × 𝑘 . • Why? Because we take each row of A dot with each column of B. • (And the “inner” dimension 𝑛 must match to take the product). − 𝐫1 − − 𝐫2 − − 𝐫3 − | 𝐜1 | | 𝐜2 | 𝐫1 ∙ 𝐜1 | 𝐜3 = 𝐫2 ∙ 𝐜1 𝐫3 ∙ 𝐜1 | 𝐫1 ∙ 𝐜2 𝐫2 ∙ 𝐜2 𝐫3 ∙ 𝐜2 𝐫1 ∙ 𝐜3 𝐫2 ∙ 𝐜3 𝐫3 ∙ 𝐜3 Matrix Inverse • Identity matrix is a square matrix with all ones along main diagonal: 1 0 0 𝐈= 0 1 0 0 0 1 • For square matrices, matrix inverse A-1 is defined (when it exists) such that: A−1A= AA-1 = I • Beyond the scope of this course to actually compute the inverse: instead, just call a matrix inverse library routine. Algebraic Properties • For matrices A, B, C and scalar c: • Scalar multiplication: 𝑐 𝐴𝐵 = 𝑐𝐴 𝐵 = 𝐴 𝑐𝐵 • Associative: 𝐴𝐵 𝐶 = 𝐴 𝐵𝐶 • Distributive: 𝐴 + 𝐵 𝐶 = 𝐴𝐶 + 𝐵𝐶 • Identity: 𝐴𝐼 = 𝐼𝐴 = 𝐴 • Transpose: 𝐴𝐵 • Inverse: 𝐴𝐵 −1 𝑇 = 𝐵𝑇 𝐴𝑇 , 𝐴 + 𝐵 𝑇 = 𝐴𝑇 + 𝐵𝑇 = 𝐵−1 𝐴−1 • Not commutative: in general, 𝐴𝐵 ≠ 𝐵𝐴 Outline • Vectors and matrices • Operations • Solving linear systems • Representation in Python • Transformation matrices • 2D • Homogeneous coordinates • 3D • Camera transformations Solving Linear Systems • If we have a linear system Ax = b: 𝑎11 𝑎21 𝑎12 𝑥1 𝑏1 = 𝑎22 𝑥2 𝑏1 • By using our multiplication rule, this is a system of linear equations: 𝑎11 𝑥1 + 𝑎12 𝑥2 = 𝑏1 𝑎21 𝑥1 + 𝑎22 𝑥2 = 𝑏2 • If matrix A and invertible, then the solution is x = A-1 b • What to do if A is non-square or non-invertible? Solving Linear Systems (Pseudoinverse) • If we have a linear system Ax = b and the matrix A is non-square or non-invertible, then often cannot solve the system exactly. • Usual solution: solve using psuedoinverse A+, defined as: A+ = (ATA)-1AT A+ = AT (AAT)-1 (if (ATA)-1 exists) (if (AAT)-1 exists) • If neither inverse exists, A+ can still be computed. • Solution to the linear system: x = A+ b • If system over-determined, solution minimizes least-squares error. • If system under-determined, finds solution with smallest norm. Outline • Vectors and matrices • Operations • Solving linear systems • Representation in Python • Transformation matrices • 2D • Homogeneous coordinates • 3D • Camera transformations Representation in Python • Vector: v = numpy.array([1.0, 2.0]) • Matrix: A = numpy.array([[1.0, 2.0], [3.0, 4.0]]) • Number of elements in vector: len(v) • Number of rows: A.shape[0] (height of image) • Number of columns: A.shape[1] (width of image) • Access element in vector: v[i] (i starts at 0) • Access element in matrix: v[row, column] (starts at 0) Representation in Python • Vector: v = numpy.array([1.0, 2.0]) • Matrix: A = numpy.array([[1.0, 2.0], [3.0, 4.0]]) • Elementwise sum: A + A • Scalar product: 2*A • Dot product: v.dot(v) • Matrix product: A.dot(v), A.dot(A) • Magnitude (Euclidean length): numpy.linalg.norm(v) • Matrix inverse: numpy.linalg.inv(A) • Pseudoinverse: numpy.linalg.pinv(A) • Solve linear systems: numpy.linalg.solve, numpy.linalg.lstsq Outline • Vectors and matrices • Operations • Solving linear systems • Representation in Python • Transformation matrices • 2D • Homogeneous coordinates • 3D • Camera transformations 2D Transformations: Translation 2D Transformations: Translation 2D Transformations: Scaling 2D Transformations: Rotation Inverse of any rotation matrix R is just RT. Transformation Matrices Slide from Fei-Fei Li and Juan Carlos Niebles Homogeneous Coordinates Slide from Fei-Fei Li and Juan Carlos Niebles Homogeneous Coordinates Slide from Fei-Fei Li and Juan Carlos Niebles 2D Translation Using Homogeneous Coordinates 𝐩′ = 𝐓𝐩 1 𝑥′ 𝑦′ = 0 1 0 0 𝑡𝑥 1 𝑡𝑦 0 1 𝑥 𝑦 1 2D Transformations: Scaling 𝐩′ = 𝐒𝐩 𝑠𝑥 𝑥′ 𝑦′ = 0 1 0 0 𝑠𝑦 0 0 0 1 𝑥 𝑦 1 2D Rotation Using Homogeneous Coordinates ′ 𝐩 = 𝐑𝐩 cos𝜃 𝑥′ 𝑦′ = sin𝜃 0 1 −sin𝜃 cos𝜃 0 0 𝑥 0 𝑦 1 1 3D Homogeneous Coordinates • Represent a 3D point as: 𝑥 𝑦 𝑧 1 • If last coordinate ends up non-one, convention is to divide by it: 𝑥/𝑤 𝑥 𝑦 𝑦/𝑤 => 𝑧 𝑧/𝑤 𝑤 1 3D Homogeneous Coordinates • Translation by (tx, ty, tz): 𝐩′ = 𝐓𝐩 1 0 𝑦 = 0 𝑧′ 1 0 ′ 𝑥 ′ 0 1 0 0 0 0 1 0 𝑡𝑥 𝑡𝑦 𝑡𝑧 1 𝑥 𝑦 𝑧 1 3D Homogeneous Coordinates • Scaling by (sx, sy, sz): 𝐩′ = 𝐒𝐩 𝑠𝑥 0 𝑦 = ′ 0 𝑧 1 0 ′ 𝑥 ′ 0 𝑠𝑦 0 0 0 0 𝑠𝑧 0 0 0 0 1 𝑥 𝑦 𝑧 1 3D Homogeneous Coordinates • Rotation around z axis: 𝐩′ = 𝐑z𝐩 ′ 𝑥 ′ cos𝜃 sin𝜃 𝑦 = ′ 0 𝑧 1 0 −sin𝜃 cos𝜃 0 0 0 0 1 0 0 0 0 1 𝑥 𝑦 𝑧 1 • (Rotates counterclockwise in right-handed coordinate system when axis is pointed towards observer) 3D Homogeneous Coordinates • Rotation around x axis: 𝐩′ = 𝐑x𝐩 1 0 𝑥′ ′ 0 cos𝜃 𝑦 = ′ 0 sin𝜃 𝑧 1 0 0 0 −sin𝜃 cos𝜃 0 0 0 0 1 𝑥 𝑦 𝑧 1 3D Homogeneous Coordinates • Rotation around y axis: 𝐩′ = 𝐑y𝐩 ′ 𝑥 ′ cos𝜃 0 𝑦 = −sin𝜃 𝑧′ 1 0 0 1 0 0 sin𝜃 0 cos𝜃 0 0 0 0 1 𝑥 𝑦 𝑧 1 Rotation Mathematics • Inverse of any rotation matrix R is just RT. • Euler angles: any 3D rotation matrix R can be written as the composition of three rotation matrices about the three axes: R = Rx(𝜃𝑥) Ry(𝜃𝑦) Rz(𝜃𝑧) • How many parameters for an arbitrary 3D rotation matrix? Outline • Vectors and matrices • Operations • Solving linear systems • Representation in Python • Transformation matrices • 2D • Homogeneous coordinates • 3D • Camera transformations Pinhole Camera Center of Projection Illustration by Steve Seitz Perspective Transformation Perspective Transformation Perspective Transformation • Perspective projection is the result of projecting from a point in a 3D scene to a 2D pixel coordinate on a camera sensor. In homogeneous coordinates: Perspective Transformation • Perspective projection is the result of projecting from a point in a 3D scene to a 2D pixel coordinate on a camera sensor. • Often we do not care about the depth (“z”) associated with a given (x, y) pixel. So we can remove that row and write as a 3x4 matrix: Combining Transformations • If we first translate (T), then scale (S), then rotate (R), then perspective project (P), we can model this as: p' = P(R(S(Tp))) • The transformations are applied in order from right to left. • Can use associativity to combine multiple transformations into a single transformation matrix M: p' = Mp= (PRST)p Camera Matrices: Intrinsic and Extrinsic • Camera matrix: a 3x4 matrix that transforms from 3D scene points (x, y, z) to 2D screen points (x′/w′, y′/w′): 𝑥 ′ 𝑐 𝑐 𝑐 𝑐 𝑥 11 12 13 14 𝑦 ′ = 𝑐 𝑦 21 𝑐22 𝑐23 𝑐24 𝑧 ′ 𝑐31 𝑐32 𝑐33 𝑐34 𝑤 1 • Intrinsic parameters model properties within the camera and lens: focal length, image sensor format, optical center. • Extrinsic parameters model the rotation and translation of a camera within a 3D scene. Camera Matrices: Intrinsic and Extrinsic • Can rewrite the camera matrix in terms of intrinsic and extrinsic parameters. 𝑥 ′ 𝑥 ′ =𝐊 𝑦 𝐑 ′ 𝑤 𝐭 𝑦 𝑧 1 𝛼𝑥 K= 0 0 𝛾 𝛼𝑥 0 𝑢0 𝑣0 1 Intrinsic matrix • 𝛼𝑥 , 𝛼𝑦 focal length in pixels, 𝛾 skew between x and y axes (often zero), 𝑢0 , 𝑣0 principal point (typically center of image). Camera Matrices: Intrinsic and Extrinsic • Can rewrite the camera matrix in terms of intrinsic and extrinsic parameters. 𝑥 ′ 𝑥 𝑦 ′ =𝐊 𝑦 𝐑 𝐭 𝑧 Extrinsic ′ 𝑤 parameters 1 • t: 3x1 vector representing origin of scene coordinate system represented in camera coordinates. • R: 3x3 rotation matrix that rotates from scene coordinates to camera coordinates. Next classes… • Camera calibration • Stereo • Optical flow