Download Math 104, Summer 2010 Homework 6 Solutions Note: we only

Math 104, Summer 2010
Homework 6 Solutions
Note: we only briefly mentioned in class how to compute the SVD. The textbook lists the
algorithm on p. 234. You may find it useful in problems 1 and 3.
3 0
1. (a) A =
0 −2
Ae1 = 3e1 and Ae2 = −2e2 = 2(−e2 ), so we put v1 = e1 , v2 = e2 ; u1 = e1 , u2 = −e2 ; and
σ1 = 3, σ2 = 2. Thus the SVD is
1 0
3 0
1 0
.
A=
0 −1
0 2
0 1
(Part (b) wasn’t assigned, but as a diagonal matrix with nonnegative entries, the SVD
has the matrix
 itself
 as Σ, while U, V are both the 2 × 2 identity matrix.)
0 2
(c) A = 0 0
0 0
Ae2 = 2e1 , while Ae1 = 0 = 0e2 . We chose e2 in the last expression because it’s a unit
vector orthogonal to e1 . Thus we can put v1 = e2 , v2 = e1 ∈ C2 ; u1 = e1 , u2 = e2 u3 = e3 ∈
C3 (where u3 was just chosen so as to extend u1 , u2 to an orthonormal basis for C3 ); and
σ1 = 2, σ2 = 0. Thus



1 0 0
2 0 0
1
A = 0 1 0 0 0
.
1 0
0 0 1
0 0
1 1
(d) A =
0 0
It is easy to see that the nullspace of A is nontrivial, but spanned by (1, −1)T . So we
must have σ2 = 0 and v2 = √12 (1, −1)T (up to a scalar of magnitude 1). Orthogonality
√
√
forces v1 = √12 (1, 1)T (again, up to a unit scalar). Since Av1 = 2e1 , u1 = e1 and σ1 = 2.
Orthogonality forces u2 = e2 . (Actually, u2 is only determined up to a unit scalar, since
σ2 = 0). So
√
1 1 1
1 0
2 0
√
.
A=
0 1
0 0
2 1 −1
1 1
(e) A =
1 1
As in part (d), we must have σ2 = 0 and v2 = √12 (1, −1)T , and hence v1 = √12 (1, 1)T .
√
Since Av1 = 2(1, 1)T , u1 = √12 (1, 1)T and σ1 = 2. We may then put u2 = √12 (1, −1)T
(again u2 is only determined up to a unit scalar since σ2 = 0). Hence
1 1 1
1 1 1
2 0
√
.
A= √
0 0
2 1 −1
2 1 −1
2. It is true that if A and B are unitarily equivalent, then they have the same singular
values:
1
Suppose A = QBQ∗ for some unitary Q, and suppose A = UA ΣA VA∗ and B = UB ΣB VB∗ .
Then
UA ΣA VA∗ = QUB ΣB VB∗ Q∗
ΣA = UA∗ (QUB ΣB VB∗ Q∗ )VA
IΣA I ∗ = (UA∗ QUB )ΣB (VB∗ Q∗ VA )
where each side of the last line is a SVD. (For the right-hand side, note that a product
of unitary matrices is unitary.) Since the set of singular values is uniquely determined,
ΣA = ΣB , so A and B have the same singular values.
The converse is not true. Consider the 2 × 2 identity matrix and the matrix for rotation
by 90 degrees:
1 0
0 1
A=
,B =
0 1
−1 0
A and B cannot be unitarily equivalent, since they have different eigenvalues. (A has eigenvalue 1, while B has eigenvalues ±i.) But their singular values are the same: the singular
values of A are both 1, and since B is a rotation, it is unitary and hence
0 1
1 0
1 0
B=
−1 0
0 1
0 1
is an SVD. So the singular values of B are also both 1.
−2 11
3. The matrix for this problem is A =
.
−10 5
104 −72
(a) We proceed by finding eigenvalues and eigenvectors of A A. First A A =
.
−72 146
We can compute that pA (λ) = λ2 − 250λ + 10 000. Thus√the eigenvalues
of A∗ A are λ1 = 200
√
and λ2 = 50. Hence the singular values of A are σ1 = 200, σ2 = 50.
We can compute that a unit eigenvector v2 for λ2 is (4/5, 3/5)T . Since v1 must be orthogonal to v2 (since A∗ A is symmetric), we can put v1 = (−3/5, 4/5)T . (When computing
u1 , we see that the other possible choice (3/5, −4/5)T introduces more minus signs in U .)
These two vectors are the right singular vectors of A.
We compute Av1 and find that a unit vector in its direction is √12 (1, 1)T . (Note the other
choice of v1 would have given √12 (−1, −1)T .) We compute Av2 and find that a unit vector
in this direction is √12 (1, −1)T .
Hence the SVD is
√
1 1 1
1 −3 4
200 √0
∗
A = U ΣV = √
0
50
5 4 3
2 1 −1
∗
∗
(b) The singular values are the diagonal entries of Σ, the left singular vectors are the
columns of U , and the right singular vectors are the rows of V ∗ .
(c) We know kAk1 is the “maximum
column sum” of A, which is 16. kAk2 is the largest
√
singular value
A, which is 200 ≈ 14. kAk∞ is the “maximum row sum” of A, which is
qof
√
P
2
15. kAkF =
250 ≈ 16.
i,j |aij | =
(d) Since A = U ΣV ∗ , A−1 = (V ∗ )−1 Σ−1 U −1 = V Σ−1 U ∗ . So
1
√
0
1 1 1
1 3 4
1/20 −11/100
200
√
=
A=
√1
1/10 −1/50
0
5 4 3
2 1 −1
50
which agrees with A−1 computed by the formula for the inverse of a√ 2 × 2 matrix.
(e) We find that pA (λ) = λ2 − 3λ + 100, so the eigenvalues are 3± 2 391 .
(f) First det A is the degree 0 term of the characteristic polynomial, which is 100. We
compute
!
!
√
√
3 − i 391
9 − (−391)
3 + i 391
λ1 λ2 =
=
= 100 = det A.
2
2
4
Also
σ1 σ2 =
√
√
√
200 50 = 10 000 = 100 = | det A|.
4. Block matrix multiplication can be carried out as ordinary matrix multiplication
(with
vi
0
each entry being the “dot product” of a row with a column). Thus we see B
=
.
0
σi ui
∗
∗
∗
∗
∗
Since A =
U ΣV
, we have
A = V ΣU and hence A U = V Σ, showing that A ui = σi vi .
0
σi vi
Hence B
=
. Neither of the vectors we’ve tried so far is an eigenvector, but
ui
0
vi
vi
vi
notice now that B
= σi
. Thus the m vectors of the form
are eigenvectors
ui
ui
ui
vi
vi
of B with eigenvalue σi . Notice also that B
= −σi
, so the m vectors of the
−ui
−ui
vi
form
are eigenvectors of B with eigenvalue −σi . It is easy to check (using that the
−ui
∗ vi
vi
V V
matrices V and U are unitary) that
= δij . Hence the matrix
is
ui
±ui
U −U
unitary, and its inverse is just its transpose. Thus a diagonalization is
V V
Σ 0
V U
.
B=
U −U
0 −Σ
V −U
5. (a) Letting x = (x1 , x2 )T , we find x2 = 1010 , x1 = 2−1010 , so x = (2−1010 , 1010 )T . Letting
y = (y1 , y2 )T , we find y1 = 1010 , so y2 = 2 − 1010 , y = (1010 , 2 − 1010 )T . (Alternatively, A
and A + δA just have their columns swapped, so the entries of√x and y should be the same
but swapped.) Then y − x ≈ 2 · 1010 (1, −1)T , so ky − xk2 ≈ 2 2 · 1010 , ky − xk∞ ≈ 2 · 1010 ,
and ky − xk1 ≈ 2 · 1010 ; all these numbers are on the order of 1010 (large).
(b) We know that the 1-norm and ∞-norm of a matrix are respectively the “maximum
column sum” and “maximum
row sum” (of absolute values of all entries); we compute the
qP
√
2
Frobenius norm as
2.
i,j |aij | . Thus kAk1 ≈ 1, kAk∞ ≈ 1, and kAkF ≈
(c) From the well known formula for the inverse of a 2 × 2 matrix, we get that
−10
1
10
−1
1 −1010
−1
A = −10
=
.
0
1
0 1010
10
√
Thus kA−1 k1 ≈ 2 · 1010 , kA−1 k∞ ≈ 1010 , and kA−1 kF ≈ 2 · 1010 . Thus, with respect to
these three norms, the condition numbers κ(A) = kAkkA−1 k are approximately 2 · 1010 , 1010 ,
and 2 · 1010 .
These are all large numbers, showing that the problem of solving Ax = b for this particular
A is ill-conditioned. This agrees with what was shown in part (a), namely that a small change
δA (one can easily check that kδAk is small) caused a large change in x.
One might also note that A is very close to being singular (not invertible); this example
suggests that it makes sense to declare the condition number of a singular square matrix to
be ∞.
6. Solution 1 : Following the hint, we know there is a vector u with kuk2 = 1 such that
kAuk2 = kAk2 . Then Au/kAk2 is a unit vector in the 2-norm. From
−1 Au = 1 kA−1 Auk2
A
kAk2 kAk2
2
1
kuk2
kAk2
1
=
kAk2
=
1
we see that kA−1 k2 ≥ kAk
(from the definition of induced matrix norm), which is what we
2
wanted to show.
Solution 2 : From the definition of an induced matrix norm, we have that for any induced
matrix norm k · k, kAk ≥ kAxk/kxk for all x 6= 0, that is, kAxk ≤ kAk kxk. Let x be any
nonzero vector. Then
kxk = kAA−1 xk ≤ kAk kA−1 xk ≤ kAkkA−1 kkxk.
Dividing both sides by kxk, we get the desired inequality. Since this is true for any induced
matrix norm, it is true for the 2-norm of a matrix.