* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Matrix inversion
Capelli's identity wikipedia , lookup
Matrix completion wikipedia , lookup
Linear least squares (mathematics) wikipedia , lookup
Rotation matrix wikipedia , lookup
System of linear equations wikipedia , lookup
Principal component analysis wikipedia , lookup
Determinant wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Four-vector wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Jordan normal form wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Matrix calculus wikipedia , lookup
Matrix Primer Reference: http://femur.wpi.edu/Learning-Modules/Linear-Algebra/home/ Definitions Matrix: An m n matrix is a rectangular array of numbers with m rows and n columns. Only real number entries will be considered in this tutorial. A general matrix will be represented by an underlined uppercase letter and a row or column matrix is represented by an underlined lowercase letter. Example: 1 2 3 A= 4 5 6 1 b = (1 2 3 ) c= 7 8 9 2 3 Matrix element: A matrix element aij is the element in the ith row and jth column. Example: 1 4 3 5 6 2 3 0 9 6 7 3 a22 = 6, a32 = 0, a41 = 6 Special Matrices Column matrix (column vector) and Row matrix (row vector): Example: See matrix b and matrix c above. Square matrix: m = n Example: See matrix A above. Diagonal matrix: A square matrix with all zer0 elements except the diagonal terms. Example Identity matrix (unity matrix): A square matrix with the diagonal elements equal 1 and remaining elements equal to 0. It is designated by a symbol I. Example: 1 0 0 I= 0 1 0 0 0 1 1 Inverse of a matrix: B is the inverse of A if A B = B A = I. If B is the inverse of A, then A is also the inverse of B. Both matrices are invertible. A A-1 = A-1 A = I (A-1) -1 = A Transpose of a matrix: The transpose of a matrix results from the interchanging of the rows and the columns of the matrix. It is designated by the superscript T or ‘. Example: 1 2 3 4 5 6 7 8 1 5 9 T 2 6 10 = 3 7 11 9 10 11 12 Symmetric matrix: AT = A (Note: aij = aji) Example: 1 3 2 T 2 1 = 3 8 5 4 8 12 5 9 3 8 5 2 5 9 Skew symmetric matrix: AT = -A (Note: aij = -aji The diagonal elements = 0) Example: 0 3 2 3 0 5 T = 2 5 0 3 2 0 3 2 3 0 5 2 5 0 Triangular matrix: (must be a square matrix) Example: 1 2 3 Upper: 1 0 0 Lower: 0 4 5 2 3 0 0 0 6 4 5 6 Zero or null matrix: All matrix elements are zero. Matrix transposition Properties: o (AT)T = A o (kA)T = k(AT) o (A + B)T = AT + BT o (A B)T = BT AT and (A B C)T = CT BT AT 2 Matrix inversion Properties: o (A-1)-1 = A o (A B )-1 = B-1A-1 o (kA)-1 = (1/k)A-1 o (AT)-1 = (A-1)T o If A is invertible, then x = A-1 b is the unique solution for A x = b Methods: o Diagonal matrix: 1 0 0 x 1 x 0 0 0 y 0 = 1 0 0 y 0 0 z 1 0 0 z o Triangular matrix: 1 0 0 x 1 1 0 x 0 0 1 x 0 x w 1 = = 0 w y 0 ( x y ) y y z y 1 v u z w u y v u 1 ( x z) z ( y z) z ( x y z) 1 x y = 0 z 1 y x ( x z) 0 1 z 1 x w v 0 y u = 0 0 z o Row reduction: (A | I) (I | A-1) Example: 1 0 0 1 1 1 0 3 0 0 1 1 3 = 0 1 1 1 3 o Determinant: T adj ( A) cij A | A| | A| 1 a For a 2 2 matrix : 11 a21 a 22 a12 a a12 a11 21 a 22 a11a22 a12 a21 1 3 1 w w u y v x ( x y) ( x y z) u 0 1 y ( y z) 1 0 0 z Linear dependence Definition: If there exists a unique solution (Ki = 0) to the equation set K1U1 + K2U2 + … + KnUn = 0, then the vector set (U1, U2, … , Un) is linearly independent. Example: o U1 = (1,5) U2 = (3,2) K1 + 3K2 = 0 5K1 + 2K2 = 0 K1 = K2 = 0 is the only solution U1and U2 are linearly independent. o U1 = (1,5) U2 = (2,10) K1 + 2K2 = 0 5K1 + 10K2 = 0 There are infinite number of solutions U1and U2 are linearly dependent. Addition and subtraction Properties: o Additions and subtractions are performed on each respective elements of the matrix. o The order of the matrices must be the same. o A + B = B +A o A + B + C = (A + B) + C = A + (B + C) o A+0=0+A=A o A – B = A + (-B) Multiplication and division Scalar multiplication and division: Multiplication or division is performed on each element of the matrix. Matrix multiplication: X Y = Z X–mt Y–tn Z–mn t zij = xik ykj k=1 a11 a12 a13 a21 a22 a23 b11 b12 b21 b22 b31 b32 = ( a11 b11 ( a21 b11 a12 b21 a22 b21 4 a13 b31 ) ( a11 b12 a23 b31 ) ( a21 b12 a12 b22 a22 b22 a13 b32 ) a23 b32 ) Properties: o o o o o ABBA A (B C) = (A B) C A (B + C) = A B + A C, (A + B) C = A C + B C a (B C) = B (a C) = (a B) C A B = A B-1 Solving linear equations Formulation: a11 x1 + a12 x2 + a13 x3 = b1 a21 x1 + a22 x2 + a23 x3 = b2 a31 x1 + a32 x2 + a33 x3 = b3 Matrix form: A x = b where a11 a12 a13 x1 b1 A = a21 a22 a23 x = x2 b = b2 a31 a32 a33 x3 b3 Solution possibilities: o Unique solution o Infinite solutions o No solution Solution methods: o Algebraic (addition/subtraction, substitution, comparison, …) o Graphical (2-dimensional only) o Matrix inversion (or Gaussian elimination—lecture example): x = A-1 b o Cramer’s rule (determinant): xi = |Ai| / |A| , where |Ai| is found by substituting the ith column of A with b. If |A| = 0 then it is called a singular matrix. A singular matrix has no inverse. a11 a12 a21 a22 = a11 a22 a12 a21 a11 a12 a13 a21 a22 a23 = a31 a32 a33 a11 a22 a33 a11 a23 a32 a21 a12 a33 5 a21 a13 a32 a31 a12 a23 a31 a13 a22 Determinants Definition: For an n n matrix using any column j : n A aij cij i 1 where cij is the cofactor correspond ing to aij : cij (1) i j M where M is the (n-1 ) (n-1 ) submatrix of A with i th row and jth column deleted. Diagonal or triangular: |A| equals to the product of all diagonal entries. Minor: Any square submatrix A is called a minor of A. o Principal minors: a11 a12 a13 a11 a12 a13 a11 a12 a a 21 a22 a23 a21 a22 a23 21 a22 a31 a32 a33 a31 a32 a33 a31 a32 o Leading principal minors: a11 a12 a11 a12 a a11 21 a22 a a 22 21 a31 a32 Matrix inversion: a11 a 21 a13 a23 a33 T adj ( A) cij A | A| | A| For a 2 2 matrix : 1 a13 a23 a33 a12 a22 1 a22 a12 a a11 21 a11 a22 a12 a21 Properties of determinant o Non-invertible if |A| = 0 o |A| = 0 if any row of A is filled entirely with zeros. o |A| = 0 if any two rows of A are equal or proportional to each other. o |A| =|AT| o |A| |B| = |A B| o |I| = 1 o |A| changes sign when two rows are exchanged. o |c A| = cn |A| where n is the order of A. 6 Eigenvalue Definitions: A x = x o Eigenvalue: A value for which the above equation set has and a non-trivial solution. o Eigenvector: A vector of A corresponding to . o Characteristic determinant/equation: D() = |A – I | = 0. can be determined from this equation. Example: 5 4 5 4 A D ( ) 2 7 6 0 1 2 1 2 1 1, 2 6 (eigenvalu es) 5 1 1 1 1 x x 0 4 x11 0 11 12 x11 x12 0 2 1 x12 0 1 (One of infinite possibilit ies) x1T 1 x21 4 x22 0 4 x21 0 5 6 2 6 x21 4 x22 0 2 6 x22 0 1 4 (Solving either equation) x21 4 x22 (One of infinite possibilit ies) x 2T 1 (Solving either equation) x11 x12 1/ 2 x x1 xˆ 1T 1 x1 12 (1) 2 1 / 2 1/ 2 xˆ xˆ 1T | xˆ 2T 1 / 2 Matrix Calculus: 4 / 17 1 / 17 xˆ 2T 4 / 17 x2 x2 x2 12 4 2 1 / 17 (eigenvecto rs - normalized ) Differentiation: Term by term t 2t 2 d sin( t ) 1 / t dt 2 3t t and 4 1 4t 0 3t cos(t ) ln( t ) 3 cos(t ) 2t 3 sin( t ) Integration: Term by term t 4t 0 2t 2 1 cos(t ) ln( t ) 3 dt sin(2 t ) 1/ t t 2t 3 sin( t ) 3t 7 4 3t cos(t ) Cayley-Hamilton theorem: Theory: For a 3 3 matrix A: f ( A) 1 I 2 A 3 A 2 1 1 0 where 2 1 1 3 1 2 0 2 12 2 2 1 f (0 ) f (1 ) f (2 ) Example: Find A10, where 1 2 A in terms of f ( A) 1 I 2 A 3 4 Steps: 1. Determine the eigenvalues of the matrix. 2 2. Find the values. 0 00 1 1 0 0 01 01 1 f (0 ) 0 11 2 f (1 ) 1 2 1 3. Evaluate f(A). 8 1 f (0 ) f (1 ) Another example: A100 0.1 0.3 0.6 A 0.4 0.5 0.1 0.3 0.5 0.2 0 1 eigenvals ( A) 2 0 1 1 0.1 0.1i 0.1 0.1i 2 One more example (This one cannot be computed directly): sin(A) The answer can be obtained by expanding the sin function into a Taylor series: Since the 3-term series and 4-term series have the same answer, we know the series has converged. 9 Now use the Cayley-Hamilton theorem: Conclusion: The Cayley-Hamilton theorem allows us to evaluate matrix functions with a finite series. Although matrix functions can also be evaluated using the Taylor series expansion techniques, it requires the evaluation of an infinite series, which often converges slowly especially for large numbers. Homework: Use Matlab to evaluate the following matrix function: 0.2 0.3 0.5 f ( A) cos 0.5 0.1 0.4 0.8 0.1 0.1 a) Use the Taylor series expansion technique. You need to include enough terms to ensure convergence. b) Use the Cayley-Hamilton technique. You need to Clearly present the results for each step (, , …). Include all necessary Matlab printouts. Circle the important steps and results on the printouts. 10 More matrix theory: Linear dependence (text p. 45): o Definition: A set of vectors {x1, x2, …, xm} in n is linearly independent iff 1 x1 2 x 2 ... m x m 0 1 2 ... m 0 o Examples: x1 = [1 0]T and x2 = [1 2]T 1(1) + 2(1) = 0 and 1(0) + 2(2) = 0 1 = 2 = 0 they are linearly independent x1 = [1 1]T and x2 = [2 2]T 1(1) + 2(2) = 0 and 1(1) + 2(2) = 0 1 = -22 they are linearly dependent Rank and nullity of a matrix (text p. 48): o Definition: The rank of A is the number of linearly independent columns in A. o The nullity of a matrix A is the difference between the total number of columns and the number of linearly independent columns in the matrix A. o | A| = 0 rank deficiency | A| 0 full rank o Matlab usage: rank(a) o Example: 1 2 3 rank 4 5 6 1 2 3 2 rank 4 5 5 7 8 9 3 7 8 9 Diagonal form and Jordan form (text p. 55): A square matrix A can be transformed into a diagonal or block diagonal form: 1 Aˆ Q A Q where Q [q1 q 2 ... q n ] and q1 q 2 ... q n are eigenvectors of A. o Distinct real s: All real diagonal elements, every element corresponds to a . o Distinct complex s: All real/complex elements, every element corresponds to a . Additional transform can remove the imaginary parts but the transformed matrix becomes block diagonal (modal form). o Repeated s (text p. 60): If the s are not all distinct, the resulting matrix may comprise upper triangular blocks along the diagonal (Jordan form). o Matlab usages (see p. 61): eig(a) and Jordan(a) Norm of a vector (P. 46): The generalized length (magnitude) of the vector n o Definition: x k k xi k i 1 o Examples: n l1 norm: x 1 xi , l2 norm: x 2 i 1 n xi i 1 2 , l norm: x max i xi o Properties: ||x|| 0 ||ax|| = |a| ||x|| for real a. ||x1 + x2|| ||x1|| + ||x2|| (triangular inequality) 11 o Matlab usages: norm(x, 1), norm(x, 2) = norm(x), and norm(x, inf) Norm of a matrix (P. 78): magnification capability Amn o ||A||1 = Largest column absolute sum o ||A||2 = Largest singular value o ||A|| = Largest row absolute sum o Matlab usages: norm(a, 1), norm(a, 2) = norm(a), and norm(a, inf) Singular-value decomposition (SVD): o Eigenvalues/vectors are defined for square matrices only. o Non-square matrices are important in linear system analysis (controllability/observability) o For an mn matrix H, we can define a symmetric matrix (nn) M = HT H and the eigenvalues of M are real and nonnegative (positive semidefinite). o The eigenvalues of M are called the singular values of H (example 3.13, p.76). o SVD: H can be decomposed into a product of 3 matrices (example 3.14, p.77). H = R S QT R RT = RT R = Im Q QT = QT Q = In S is an mn matrix with the singular values of H on the diagonal. o Matlab usages: s = svd(H) gives the singular values of H. [R, S, Q] = svd(H) gives all 3 matrices of the decomposition. o Applications of SVD: Norm of a matrix: ||A||2 = 1 (Largest singular value) Rank of a matrix: equal to the number of non-zero singular values. Condition number = max / min: Indicates how close a matrix is to rank deficiency (How much numerical error is likely to be introduced by computations involving the matrix). Large condition numbers imply ill-conditioned systems and small (close to 1) condition numbers imply well-conditioned systems. Matlab usage: cond(a) Lyapunov theorem o The Lyapunov theorem provides an alternate means to check the asymptotic stability of a system. o Lyapunov equation (text p. 70-1): A(M) = A M + M B = C o A(M) = M, where ’s are the eigenvalues of A and they represent all possible sums of the eigenvalues of A and B. o A symmetric matrix M is said to be positive definite (denoted by M > 0) if xTM x > 0 for nonzero x. o If M > 0, then xTM x = 0 iff x = 0. o M > 0 iff any one of the following conditions holds: Every eigenvalue of M is positive. All leading principal minors of M are positive. There exists an nn nonsingular matrix such that M = NTN. o Matlab usage: m = lyap(a, b, -c) 12