* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Linear Algebra Review Vectors By Tim K. Marks UCSD
Cross product wikipedia , lookup
Euclidean vector wikipedia , lookup
Matrix completion wikipedia , lookup
Vector space wikipedia , lookup
Exterior algebra wikipedia , lookup
Linear least squares (mathematics) wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
Rotation matrix wikipedia , lookup
System of linear equations wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Principal component analysis wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Jordan normal form wikipedia , lookup
Determinant wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Four-vector wikipedia , lookup
Gaussian elimination wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Matrix calculus wikipedia , lookup
Linear Algebra Review By Tim K. Marks UCSD Borrows heavily from: Jana Kosecka [email protected] http://cs.gmu.edu/~kosecka/cs682.html Virginia de Sa Cogsci 108F Linear Algebra review UCSD Vectors The length of x, a.k.a. the norm or 2-norm of x, is x = x12 + x 22 + L + x n2 e.g., ! x = 32 + 2 2 + 5 2 = 38 ! 1 Good Review Materials http://www.imageprocessingbook.com/DIP2E/dip2e_downloads/review_material_downloads.htm (Gonzales & Woods review materials) Chapt. 1: Linear Algebra Review Chapt. 2: Probability, Random Variables, Random Vectors Online vector addition demo: http://www.pa.uky.edu/~phy211/VecArith/index.html 2 Vector Addition v u+v u Vector Subtraction u u-v v 3 Example (on board) Inner product (dot product) of two vectors #6& % ( a =%2 ( %$"3(' "4% $ ' b = $1' $#5'& a " b = aT b ! ! ! #4& % ( = [6 2 "3] %1( %$5(' = 6 " 4 + 2 "1+ (#3) " 5 ! = 11 ! ! 4 Inner (dot) Product v The inner product is a SCALAR. α u 5 Transpose of a Matrix Transpose: Examples: If , we say A is symmetric. symmetric. Example of symmetric matrix 6 7 8 9 Matrix Product Product: A and B must have compatible dimensions In Matlab: >> A*B Examples: Matrix Multiplication is not commutative: 10 Matrix Sum Sum: Example: A and B must have the same dimensions Determinant of a Matrix Determinant: A must be square Example: 11 Determinant in Matlab Inverse of a Matrix If A is a square matrix, the inverse of A, called A-1, satisfies AA-1 = I and A-1A = I, Where I, the identity matrix, is a diagonal matrix with all 1’s on the diagonal. "1 0% I2 = $ ' #0 1& ! "1 0 0% $ ' I3 = $0 1 0' $#0 0 1'& ! 12 Inverse of a 2D Matrix Example: Inverses in Matlab 13 Other Terms 14 Matrix Transformation: Scale A square diagonal matrix scales each dimension by the corresponding diagonal element. Example: "2 0 0% $ ' $0 .5 0' $#0 0 3'& "6% "12% $ ' $ ' $8' = $ 4 ' $#10'& $#30&' ! 15 http://www.math.ubc.ca/~cass/courses/m309-8a/java/m309gfx/eigen.html 16 Some Properties of Eigenvalues and Eigenvectors – If λ1, …, λn are distinct eigenvalues of a matrix, then the corresponding eigenvectors e1, …, en are linearly independent. – A real, symmetric square matrix has real eigenvalues, with eigenvectors that can be chosen to be orthonormal. Linear Independence • A set of vectors is linearly dependent if one of the vectors can be expressed as a linear combination of the other vectors. "1% "0% "2% Example: $' $' $' $0' , $1' , $1' $#0'& $#0&' $#0&' • A set of vectors is linearly independent if none of the vectors can be expressed as a linear ! combination of the other vectors. Example: "1% "0% "2% $' $' $' $0' , $1' , $1' $#0'& $#0&' $#3&' ! 17 Rank of a matrix • The rank of a matrix is the number of linearly independent columns of the matrix. Examples: "1 0 2% $ ' has rank 2 0 1 1 $ ' $#0 0 0'& ! "1 0 2% $ ' $0 1 0' $#0 0 1'& has rank 3 • Note: the rank of a matrix is also the number of linearly!independent rows of the matrix. Singular Matrix All of the following conditions are equivalent. We say a square (n × n) matrix is singular if any one of these conditions (and hence all of them) is satisfied. – – – – – The columns are linearly dependent The rows are linearly dependent The determinant = 0 The matrix is not invertible The matrix is not full rank (i.e., rank < n) 18 Linear Spaces A linear space is the set of all vectors that can be expressed as a linear combination of a set of basis vectors. We say this space is the span of the basis vectors. – Example: R3, 3-dimensional Euclidean space, is spanned by each of the following two bases: "1% "0% "0% $' $' $' $0' , $1' , $0' $#0'& $#0&' $#1&' ! "1% "0% "0% $' $' $' $0' , $1' , $0' $#0'& $#2&' $#1&' ! Linear Subspaces A linear subspace is the space spanned by a subset of the vectors in a linear space. – The space spanned by the following vectors is a two-dimensional subspace of R3. "1% "0% $' $' What does it look like? $0' , $1' $#0'& $#0'& – The space spanned by the following vectors is a two-dimensional subspace of R3. "1% "0% ! $' $' What does it look like? $1' , $0' $#0'& $#1'& ! 19 Orthogonal and Orthonormal Bases n linearly independent real vectors span Rn, n-dimensional Euclidean space • They form a basis for the space. – An orthogonal basis, a1, …, an satisfies ai ⋅ aj = 0 if i j – An orthonormal basis, a1, …, an satisfies ai ⋅ aj = 0 if i j ai ⋅ aj = 1 if i = j – Examples. Orthonormal Matrices A square matrix is orthonormal (also called unitary) if its columns are orthonormal vectors. – A matrix A is orthonormal iff AAT = I. • If A is orthonormal, A-1 = AT AAT = ATA = I. – A rotation matrix is an orthonormal matrix with determinant = 1. • It is also possible for an orthonormal matrix to have determinant = -1. This is a rotation plus a flip (reflection). 20 SVD: Singular Value Decomposition Any matrix A (m × n) can be written as the product of three matrices: A = UDV T where – U is an m × m orthonormal matrix – D is an m × n diagonal matrix. Its diagonal elements, σ1, σ2, …, are called the singular values of A, and satisfy σ1 ≥ σ2 ≥ … ≥ 0. – V is an n × n orthonormal matrix Example: if m > n A "• $ $• $• $ $• $#• • •% "( ' $ • •' $ | • •' = $u1 ' $ • •' $ | • •'& $#) U ( | ( | u2 | u3 | ) ) VT D (% ' |' L um ' ' |' ) '& "*1 0 0 % $ '" T $ 0 * 2 0 '$+ v1 $ 0 0 * n '$ M M $ '$ T $ 0 0 0 '#+ v n $# 0 0 0 '& ,% ' M' ,'& ! SVD in Matlab >> x = [1 2 3; 2 7 4; -3 0 6; 2 4 9; 5 -8 0] x= 1 2 3 2 7 4 -3 0 6 2 4 9 5 -8 0 >> [u,s,v] = svd(x) u= -0.24538 0.11781 -0.11291 -0.47421 -0.82963 -0.53253 -0.11684 -0.52806 -0.45036 0.4702 -0.30668 0.24939 0.79767 -0.38766 0.23915 -0.64223 0.44212 -0.057905 0.61667 -0.091874 0.38691 0.84546 -0.26226 -0.20428 0.15809 s= 14.412 0 0 0 8.8258 0 0 0 5.6928 0 0 0 0 0 0 v= 0.01802 -0.68573 -0.72763 0.48126 -0.63195 0.60748 -0.87639 -0.36112 0.31863 21 Some Properties of SVD – The rank of matrix A is equal to the number of nonzero singular values σi – A square (n × n) matrix A is singular iff at least one of its singular values σ1, …, σn is zero. Geometric Interpretation of SVD If A is a square (n × n) matrix, A U ! D VT "• • •% " ( L ( % "*1 0 0 %"+ v1T $ ' $ ' $ '$ M $• • •' = $u1 L un ' $ 0 * 2 0 '$ M $#• • •'& $# ) L ) '& $# 0 0 * n '&$#+ v Tn – U is a unitary matrix: rotation (possibly plus flip) – D is a scale matrix T – V (and thus V ) is a unitary matrix ,% ' M' ,'& Punchline: An arbitrary n-D linear transformation is equivalent to a rotation (plus perhaps a flip), followed by a scale transformation, followed by a rotation Advanced: y = Ax = UDV Tx T – V expresses x in terms of the basis V. – D rescales each coordinate (each dimension) – The new coordinates are the coordinates of y in terms of the basis U 22