* Your assessment is very important for improving the workof artificial intelligence, which forms the content of this project
Download Math 104, Summer 2010 Homework 6 Solutions Note: we only
Euclidean vector wikipedia , lookup
Matrix completion wikipedia , lookup
Linear least squares (mathematics) wikipedia , lookup
System of linear equations wikipedia , lookup
Covariance and contravariance of vectors wikipedia , lookup
Rotation matrix wikipedia , lookup
Determinant wikipedia , lookup
Matrix (mathematics) wikipedia , lookup
Jordan normal form wikipedia , lookup
Non-negative matrix factorization wikipedia , lookup
Principal component analysis wikipedia , lookup
Four-vector wikipedia , lookup
Eigenvalues and eigenvectors wikipedia , lookup
Gaussian elimination wikipedia , lookup
Orthogonal matrix wikipedia , lookup
Cayley–Hamilton theorem wikipedia , lookup
Perron–Frobenius theorem wikipedia , lookup
Matrix multiplication wikipedia , lookup
Math 104, Summer 2010 Homework 6 Solutions Note: we only briefly mentioned in class how to compute the SVD. The textbook lists the algorithm on p. 234. You may find it useful in problems 1 and 3. 3 0 1. (a) A = 0 −2 Ae1 = 3e1 and Ae2 = −2e2 = 2(−e2 ), so we put v1 = e1 , v2 = e2 ; u1 = e1 , u2 = −e2 ; and σ1 = 3, σ2 = 2. Thus the SVD is 1 0 3 0 1 0 . A= 0 −1 0 2 0 1 (Part (b) wasn’t assigned, but as a diagonal matrix with nonnegative entries, the SVD has the matrix itself as Σ, while U, V are both the 2 × 2 identity matrix.) 0 2 (c) A = 0 0 0 0 Ae2 = 2e1 , while Ae1 = 0 = 0e2 . We chose e2 in the last expression because it’s a unit vector orthogonal to e1 . Thus we can put v1 = e2 , v2 = e1 ∈ C2 ; u1 = e1 , u2 = e2 u3 = e3 ∈ C3 (where u3 was just chosen so as to extend u1 , u2 to an orthonormal basis for C3 ); and σ1 = 2, σ2 = 0. Thus 1 0 0 2 0 0 1 A = 0 1 0 0 0 . 1 0 0 0 1 0 0 1 1 (d) A = 0 0 It is easy to see that the nullspace of A is nontrivial, but spanned by (1, −1)T . So we must have σ2 = 0 and v2 = √12 (1, −1)T (up to a scalar of magnitude 1). Orthogonality √ √ forces v1 = √12 (1, 1)T (again, up to a unit scalar). Since Av1 = 2e1 , u1 = e1 and σ1 = 2. Orthogonality forces u2 = e2 . (Actually, u2 is only determined up to a unit scalar, since σ2 = 0). So √ 1 1 1 1 0 2 0 √ . A= 0 1 0 0 2 1 −1 1 1 (e) A = 1 1 As in part (d), we must have σ2 = 0 and v2 = √12 (1, −1)T , and hence v1 = √12 (1, 1)T . √ Since Av1 = 2(1, 1)T , u1 = √12 (1, 1)T and σ1 = 2. We may then put u2 = √12 (1, −1)T (again u2 is only determined up to a unit scalar since σ2 = 0). Hence 1 1 1 1 1 1 2 0 √ . A= √ 0 0 2 1 −1 2 1 −1 2. It is true that if A and B are unitarily equivalent, then they have the same singular values: 1 Suppose A = QBQ∗ for some unitary Q, and suppose A = UA ΣA VA∗ and B = UB ΣB VB∗ . Then UA ΣA VA∗ = QUB ΣB VB∗ Q∗ ΣA = UA∗ (QUB ΣB VB∗ Q∗ )VA IΣA I ∗ = (UA∗ QUB )ΣB (VB∗ Q∗ VA ) where each side of the last line is a SVD. (For the right-hand side, note that a product of unitary matrices is unitary.) Since the set of singular values is uniquely determined, ΣA = ΣB , so A and B have the same singular values. The converse is not true. Consider the 2 × 2 identity matrix and the matrix for rotation by 90 degrees: 1 0 0 1 A= ,B = 0 1 −1 0 A and B cannot be unitarily equivalent, since they have different eigenvalues. (A has eigenvalue 1, while B has eigenvalues ±i.) But their singular values are the same: the singular values of A are both 1, and since B is a rotation, it is unitary and hence 0 1 1 0 1 0 B= −1 0 0 1 0 1 is an SVD. So the singular values of B are also both 1. −2 11 3. The matrix for this problem is A = . −10 5 104 −72 (a) We proceed by finding eigenvalues and eigenvectors of A A. First A A = . −72 146 We can compute that pA (λ) = λ2 − 250λ + 10 000. Thus√the eigenvalues of A∗ A are λ1 = 200 √ and λ2 = 50. Hence the singular values of A are σ1 = 200, σ2 = 50. We can compute that a unit eigenvector v2 for λ2 is (4/5, 3/5)T . Since v1 must be orthogonal to v2 (since A∗ A is symmetric), we can put v1 = (−3/5, 4/5)T . (When computing u1 , we see that the other possible choice (3/5, −4/5)T introduces more minus signs in U .) These two vectors are the right singular vectors of A. We compute Av1 and find that a unit vector in its direction is √12 (1, 1)T . (Note the other choice of v1 would have given √12 (−1, −1)T .) We compute Av2 and find that a unit vector in this direction is √12 (1, −1)T . Hence the SVD is √ 1 1 1 1 −3 4 200 √0 ∗ A = U ΣV = √ 0 50 5 4 3 2 1 −1 ∗ ∗ (b) The singular values are the diagonal entries of Σ, the left singular vectors are the columns of U , and the right singular vectors are the rows of V ∗ . (c) We know kAk1 is the “maximum column sum” of A, which is 16. kAk2 is the largest √ singular value A, which is 200 ≈ 14. kAk∞ is the “maximum row sum” of A, which is qof √ P 2 15. kAkF = 250 ≈ 16. i,j |aij | = (d) Since A = U ΣV ∗ , A−1 = (V ∗ )−1 Σ−1 U −1 = V Σ−1 U ∗ . So 1 √ 0 1 1 1 1 3 4 1/20 −11/100 200 √ = A= √1 1/10 −1/50 0 5 4 3 2 1 −1 50 which agrees with A−1 computed by the formula for the inverse of a√ 2 × 2 matrix. (e) We find that pA (λ) = λ2 − 3λ + 100, so the eigenvalues are 3± 2 391 . (f) First det A is the degree 0 term of the characteristic polynomial, which is 100. We compute ! ! √ √ 3 − i 391 9 − (−391) 3 + i 391 λ1 λ2 = = = 100 = det A. 2 2 4 Also σ1 σ2 = √ √ √ 200 50 = 10 000 = 100 = | det A|. 4. Block matrix multiplication can be carried out as ordinary matrix multiplication (with vi 0 each entry being the “dot product” of a row with a column). Thus we see B = . 0 σi ui ∗ ∗ ∗ ∗ ∗ Since A = U ΣV , we have A = V ΣU and hence A U = V Σ, showing that A ui = σi vi . 0 σi vi Hence B = . Neither of the vectors we’ve tried so far is an eigenvector, but ui 0 vi vi vi notice now that B = σi . Thus the m vectors of the form are eigenvectors ui ui ui vi vi of B with eigenvalue σi . Notice also that B = −σi , so the m vectors of the −ui −ui vi form are eigenvectors of B with eigenvalue −σi . It is easy to check (using that the −ui ∗ vi vi V V matrices V and U are unitary) that = δij . Hence the matrix is ui ±ui U −U unitary, and its inverse is just its transpose. Thus a diagonalization is V V Σ 0 V U . B= U −U 0 −Σ V −U 5. (a) Letting x = (x1 , x2 )T , we find x2 = 1010 , x1 = 2−1010 , so x = (2−1010 , 1010 )T . Letting y = (y1 , y2 )T , we find y1 = 1010 , so y2 = 2 − 1010 , y = (1010 , 2 − 1010 )T . (Alternatively, A and A + δA just have their columns swapped, so the entries of√x and y should be the same but swapped.) Then y − x ≈ 2 · 1010 (1, −1)T , so ky − xk2 ≈ 2 2 · 1010 , ky − xk∞ ≈ 2 · 1010 , and ky − xk1 ≈ 2 · 1010 ; all these numbers are on the order of 1010 (large). (b) We know that the 1-norm and ∞-norm of a matrix are respectively the “maximum column sum” and “maximum row sum” (of absolute values of all entries); we compute the qP √ 2 Frobenius norm as 2. i,j |aij | . Thus kAk1 ≈ 1, kAk∞ ≈ 1, and kAkF ≈ (c) From the well known formula for the inverse of a 2 × 2 matrix, we get that −10 1 10 −1 1 −1010 −1 A = −10 = . 0 1 0 1010 10 √ Thus kA−1 k1 ≈ 2 · 1010 , kA−1 k∞ ≈ 1010 , and kA−1 kF ≈ 2 · 1010 . Thus, with respect to these three norms, the condition numbers κ(A) = kAkkA−1 k are approximately 2 · 1010 , 1010 , and 2 · 1010 . These are all large numbers, showing that the problem of solving Ax = b for this particular A is ill-conditioned. This agrees with what was shown in part (a), namely that a small change δA (one can easily check that kδAk is small) caused a large change in x. One might also note that A is very close to being singular (not invertible); this example suggests that it makes sense to declare the condition number of a singular square matrix to be ∞. 6. Solution 1 : Following the hint, we know there is a vector u with kuk2 = 1 such that kAuk2 = kAk2 . Then Au/kAk2 is a unit vector in the 2-norm. From −1 Au = 1 kA−1 Auk2 A kAk2 kAk2 2 1 kuk2 kAk2 1 = kAk2 = 1 we see that kA−1 k2 ≥ kAk (from the definition of induced matrix norm), which is what we 2 wanted to show. Solution 2 : From the definition of an induced matrix norm, we have that for any induced matrix norm k · k, kAk ≥ kAxk/kxk for all x 6= 0, that is, kAxk ≤ kAk kxk. Let x be any nonzero vector. Then kxk = kAA−1 xk ≤ kAk kA−1 xk ≤ kAkkA−1 kkxk. Dividing both sides by kxk, we get the desired inequality. Since this is true for any induced matrix norm, it is true for the 2-norm of a matrix.