* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Chapter 4
Covariance and contravariance of vectors wikipedia , lookup
Matrix completion wikipedia , lookup
Capelli's identity wikipedia , lookup
Linear least squares (mathematics) wikipedia , lookup
System of linear equations wikipedia , lookup
Rotation matrix wikipedia , lookup
Principal component analysis wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Jordan normal form wikipedia , lookup
Determinant wikipedia , lookup
Singular-value decomposition wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Four-vector wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Gaussian elimination wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Chapter 4 Linear Models and Matrix Algebra Matrix algebra • Provides a compact way of writing an equation system. • Leads to a way of testing the existence of a solution by evaluation of a determinant. • Gives a method of finding solution (if it exists) Matrices and Vectors Matrices as Arrays: a x a x ..... a x d 11 1 12 2 1n n 1 a x a x ..... a x d 12 1 22 2 2n n 2 ................................................. a x a x ..... a x d m m1 1 m2 2 mn n There are three types of ingredients in the equation system shown above: 1. The set of coefficients aij. 2. The set of variables x1, ….xn 3. The set of constant terms d1, ….dm a11 a12 a21 a22 A ... ... am1 am 2 ... a1n ... a2 n ... ... ... amn x1 x2 . x . . xn d d . d . . d m 1 2 Numerical Example: The following equation can be written in matrix form as follows: 2x1 + 3x2 + 5x3 = 10 x1 + 3x2 + 2x3 = 15 6x1 + x2 + 8x3 = 20 2 3 5 A 1 3 2 6 1 8 x1 x x2 x3 10 d 15 20 This can also be written as Ax=d A matrix is defined as a rectangular array of numbers, parameters or variables. The members of the array are referred to as the elements of the matrix. Vectors as Special Matrices The number of rows and columns in a matrix define the dimension of a matrix. If there are m rows and n columns, the it is said to be a m x n matrix (read as: “m by n”). If m = n, then the matrix is called a square matrix. 2 1 4 6 2 2 8 12 1 3 0 7 4 1 3 2 This is a 4 x 3 matrix (“4 by 3”) where m = 4 and n=3 This is 2 x 2 matrix (“2 by 2”) where m = 2 and n = 2. This is a square matrix Vectors---Column vectors: These are matrices with dimensions n x 1. For example, 2 4 C 6 8 Dimension: 4 x 1 Row Vectors: These are matrices with dimensions 1 x n. For example, B 4 3 1 0 Dimensions: 1 x 4 Matrix Operations Addition of matrices: Two matrices can be added only when they have the same dimensions. When this condition is met, the matrices are said to be conformable for addition. General formula: aij bij cij Numerical Example: 3 2 1 2 4 7 3 2 2 4 1 7 5 6 8 2 4 3 11 2 3 2 11 4 2 3 3 13 2 0 8 10 2 8 1 4 8 8 10 1 2 4 16 11 6 3x3 3x3 3x3 Matrix Operations.. Subtraction of matrices: Two matrices can be subtracted only when they have the same dimensions. When this condition is met, the matrices are said to be conformable for addition. General formula: aij bij dij Numerical Example: 2 0 3 1 6 8 2 1 0 6 3 8 3 6 5 8 5 3 10 1 3 8 10 5 1 3 3 2 4 0 8 3 2 4 9 1 8 4 3 9 2 1 4 6 1 3x3 3x3 3x3 Scalar Multiplication: To multiply a matrix by a number – or in matrix algebra terminology, by a scalar – is to multiply every element by the given scalar. The scalar number can be positive or negative. 8 4 2 4 2 3 3 Here the fraction 2/3 is a scalar 2 10 3 1 5 number. 3 3 Multiplication of Matrices: The matrix multiplication requires that the number of columns of the first factor A be the same as the number of rows of the second factor B in order to form the product AB. If this condition is not satisfied, then the product is undefined. General Form: Numerical Example: Matrix Operations.. Division of matrices: While matrices can undergo addition, subtraction, and multiplication, subject to the comfortability conditions, it is not possible to divide a matrix by another. That is, we cannot have A/B. The ∑ notation: The summation shorthand makes use of Greek letter ∑ (sigma, for sum). For instance, 4 x1 x2 x3 x4 xi i 1 3 a1x0 a2 x1 a3 x2 a j x j 1 j 1 Notes on Vector Operations Multiplication of vectors: 5 Given p 3 and q 3 2 1 5(3) 5(2) 5(1) 15 10 5 then pq 9 6 3 3 ( 3 ) 3 ( 2 ) 3 ( 1 ) 3x3 8 Similarly, given a 2 4 and b then ab 2(8) 4(2) 24 2 1x1 It is important to distinguish between the meaning of matrix pq (a matrix larger than 1x1) and matrix ab (a 1x1 matrix, or a scalar). Geometric Interpretation of Vector Operations:In this diagram, consider two vectors v and w. The sum v + w can be directly plotted as the arrow shown. If we construct a parallelogram with the two vectors v and w as two of its sides, however, the diagonal of the parallelogram will turn out exactly to be the arrow representing the vector sum v + w. If we have vector u and then the multiplication of u by k will produce an overlapping arrow, but the arrowhead will be relocated, unless k =1. If k > 1, then the arrow will be extended out, if 0<k<1, the arrow will be shortened, and if k = 0, the arrow will shrink into the point of origin. A negative multiplier will reverse the direction of the arrow. Linear Dependence: A set of vectors v1, v2, …,vn is said to be linearly dependent if (and only if) any one of them can be expressed as a linear combination of the remaining vectors; otherwise they are linearly independent. For instance, 2 v1 5 , 1 v2 3 , and 6 v3 14 then 2 1 6 4v1 – 2v2 = v3 because 4 5 23 14 = v3 Commutative, Associative, and Distributive Laws Commutative law: A+B=B+A Given that 2 4 1 2 A and B 3 2 3 4 3 6 A B B A 6 6 Associative law: (A + B) + C = A + (B +C) 1 A 3 , 2 B 5 3 and C 8 6 ( A B) C 16 6 A (B C) 16 In matrix multiplication, AB BA. To show this, lets consider: 2 3 1 5 11 16 4 6 3 2 22 32 AB 1 5 2 3 22 33 3 2 4 5 14 21 BA This proves that matrix multiplication is not commutative, i.e. AB BA Associative law: (AB)C = A(BC) = ABC If A is m x n matrix, and if C is p x q matrix, then conformability requires that B be n x p matrix. Distributive law: A(B+C) = AB + AC [premultiplication by A] (B+C)A = BA + CA [postmultiplication by A] Identity Matrices and Null Matrices Identity Matrices: A square matrix with 1s in its diagonal and 0s everywhere else. It is denoted by I or In where n serves to indicate its row (as well as column) dimension. Thus, 1 0 I 5 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 1 3 2 3 2 1 0 3 2 Also, let A AI A 4 1 4 1 0 1 4 1 1 0 3 2 3 2 IA A 0 1 4 1 4 1 Null Matrices Null Matrices: A null matrix is simply a matrix whose all elements are zero. Unlike I, the null matrix is not restricted to a square matrix. It can have any dimensions. Null matrices follow the following rules: 2 0 2 A0 A 4 0 4 0 4 2 1 0 A0 0 0 0 1 0 0 0 Transposes and Inverses Transposes: When the rows and columns of a matrix A are interchanged, we obtain a transpose of A. It is either denoted by A’ or AT. 2 2 4 If A A' 3 2 4 3 Here the dimension of the transpose matrix is 2 identical to the original matrix. Properties of transpose: 1. (A’)’ = A 2. (A+B)’ = A’ + B’ 3. (AB)’ = B’A’ Inverses: The inverse of a matrix A, denoted by A-1, is defined only if it’s a square matrix, in which case the inverse is the matrix that satisfies the condition: AA-1 = A-1A=I Inverse Properties: 1. Not every square matrix has an inverse. If a square matrix A has an inverse, then we call it nonsingular; if A has no inverse, then it is singular matrix. 2. If A-1 does exist, then matrix A can be regarded as the inverse of A-1. 3. If A is n x n matrix, then A-1 must also be n x n. 4. If an inverse exists, then it is unique. 5. The two parts of AA-1 = A-1A=I actually imply each other. Inverse Matrix and Solution Linear Equation System: We earlier referred to an equation Ax = d to solve a system of linear equations. The concept of inverse matrix can be applied to this: Example: Note: The method of testing the existence of inverse and its calculation will be discussed in Chapter 5. Finite Markov Chains A common application of matrix algebra is found in what is known as Markov processes or Markov Chains. Markov processes are used to measure or estimate movements over time. This involves the use of a Markov transition matrix, where each value in the transition matrix is a probability of moving from one state to another. There is also a vector containing the initial distribution across various states. Example: Let : PAA = probability that a current A remains in A. PAB = probability that a current A moves to B. PBB = probability that a current B remains in B. PBA = probability that a current B moves to A. At and Bt denote the populations of Abbotsford and Burnaby, respectively, at some t. If we denote the distribution of employees across locations at time t as a vector, ' xt At Bt and the transitional probabilities in matrix form. P M AA PBA PAB PBB Then the distribution of employees in the next period (t+1) is, ' ' xt M xt 1 At 1 Bt 1 In general, for n periods the distribution of employees will be, At PAA Bt PBA n PAB At n PBB Bt n