Download MTH285 HOMEWORK 5 section 1.6. 2. Express u T Ku = 4u 1 +

MTH285 HOMEWORK 5
section 1.6.
2.
uT Ku = 4u21 + 16u1 u2 + 26u22 as the sum of two squares. Then find chol(K) =
√ Express
T
DL .
Solution: There was a typo in this problem the third matrix was missing a transpose T .
In the LDLT tells you how to group the terms to obtain the quadratic form as a sum of
squares: (u1 , u2 )LDLT (u1 , u2 )T = (u1 + 2u2 , u2 )D(u1 + 2u2 , u2 )T . Multiplying this out
gives uT Ku = 4u21 + 16u1 u2 + 26u22 = 4(u1 + 2u2 )2 + 10u22 . Completing the square is
equivalent to LDLT factorization.
You can also do the “tensor” version of multiplication. Let L = [v1 , v2 ] then the product
LDLT = 4v1 vT1 + 10v2 vT2 the sum of two 2 by 2 matrices. This looks a little like the
decomposition used for SVD.
3. A different A produces the circulant second-difference matrix C = AT A




1 −1
0
2 −1 −1
1 −1  gives AT A =  −1
2 −1 
A= 0
−1
0
1
−1 −1
2
How can you tell from A that C = AT A is only semidefinite? Which vectors solve Au = 0
and therefore Cu = 0? Note that chol(C) will fail.
Solution: A does not have independent columns (they sum to zero). So AT A will not have
full rank. In particular (1, 1, 1) is a null vector.
4. Confirm that the circulant C = AT A above is semidefinite by the pivot test. Write
uT Cu as a sum of two squares with the pivots as coefficients. (The eigenvalues of 0, 3, 3
give another proof that C is semidefinite.)


2.0 −1.0 −1.0
2.0 −1.0  u = 2u21 − 2u1 u2 − 2u1 u3 − 2u2 u3 + 2u22 If we factor
Solution: uT  −1.0
−1.0 −1.0
2.0
the matrix we get




1.0 −0.5 −0.5
1.0
0.0 0.0 2 0 
 −0.5
1.0 0.0 
0.0
1.0 −1.0 
0 32
−0.5 −1.0 1.0
0.0
0.0
1.0
1
2
MTH285 HOMEWORK 5
. This suggests we can complete the square on the quadratic by 2(u1 − 12 u2 − 12 u3 )2 +
3
2
2 (u2 − u3 ) . Multiplying this out should get the original quadratic expression. I don’t
think I could have figured this out without using the LDLT decomposition.
7. Is K = At A or M = BT B positive

1
A= 2
3
definite (independent columns in A or B)?



2
1 4
4 
B= 2 5 
6
3 6
We know that uT Mu = (Bu)T (Bu) = (u1 + 4u2 )2 + (2u1 + 5u2 )2 + (3u1 + 6u2 )2 Show how
the three squares for uT Ku = (Au)T (Au) collapse into one square.
Solution:
M is positive definite because B has independent columns. K is positive semi-definite
because A has dependent columns.
If we factor A first as in QR factorization the calculation of AT A will be a little easier.
It would be a lot easier if A had more columns!


1 0 1 2


A= 2 0
0 1
3 0
and
1 0
14 0
1 2
A A=
2 1
0 0
0 1
T
From this we predict that uT Ku = 14(u1 + 2u2 )2 which expands to the original quadratic
expression.
11. For the function f(xy) = 2xy certainly has a saddle point and not a minimum at (0, 0).
What symmetric matrix S produces this f? What are its eigenvalues?
Solution:
0 1
x
f(x, y) = x y
= 2xy
y
1 0
You can guess and check quickly that the matrix has (1, 1) as an eigenvector with eigenvalue
1 and that (1, −1) is another eigenvector with eigenvalue −1. This means that the quadratic
form is indefinite and that the graph will be a saddle shape.
MTH285 HOMEWORK 5
3
15. For what numbers c and d are A and B positive definite? Test the three upper left
determinants (1 by 1, 2 by 2, 3 by 3) of each matrix.




1 2 3
c 1 1
A =  1 c 1  and B =  2 d 4 
3 4 5
1 1 c
Solution:
Using the determinant test (all upper left corner determinants have to be positive)
we see that. c > 0 and for the next determinant (c2 − 1) > 0 and for the final one
c(c − 1) − 2(c − 1) = (c − 1)(c2 + c − 2 = (c − 1)2 (c + 2) > 0. Considering all these criteria
we see that c > 1 is the criteria for the quadratic form to be positive definite.
For the second matrix the determinants and conditions are: 1 > 0, d − 4 > 0 and 3 > d.
So there is no value of d that makes B positive definite.
16. If A is positive definite then A−1 is positive definite. Best proof: The eigenvalues of
A−1 are positive because:
c
−b
1
−1
pass the
Second proof: (only quick for 2 by 2) The entries of A = ac−b2
−ba
determinant tests:
Solution:
A symmetric matrix is positive definite if and only if its eigenvalues are positive. In that
case the reciprocals of the eigenvalues are also positive so the inverse is positive definite
as well. The determinant of the inverse is also positive if the original determinant was
positive (and the trace does not change sign) so this implies for a two by two matrix that
the eigenvalues are both positive.
20. With out multiplying A =
cos (θ) − sin (θ)
sin (θ)
cos (θ)
2 0
0 5
cos (θ) sin (θ)
− sin (θ) cos (θ)
find
(a)
(b)
(c)
(d)
the determinant of A
the eigenvalues of A
the eigenvectors of A
a reason why A is symmetric positive definite.
Solution:
The two outside matrices are rotations (unitary matrices) they have determinant one
and their transpose is also their inverse. The determinant of A is therefore 25̇ = 10, the
eigenvalues of A are 5 and 2. The eigenvectors of A are the columns of the first matrix:
(cos θ, sin θ) and (− sin θ cos θ). It is symmetric and positive definite because it is in the
form QDQT and the eigenvalues are positive.
4
MTH285 HOMEWORK 5
21. For the first function we have:
For f1 (x, y) = 14 x4 + x2 y + y2 and f2 (x, y) = x3 + xy − x find the second derivative
(Hessian) matrices H1 and H2 .
" 2
#
2
H=
∂ f
∂x2
∂2 f
∂y∂x
∂ f
∂x∂y
∂2 f
∂y2
H1 is positive definite so f1 is concave up (=convex). Find the minimum point of f1 and
the saddle point of f2 (look where the first derivatives are zero).
Solution:
2
3x + 2y 2x
f=
+
+
Df =
+
+ 2y) and
=
At a critical
2x
2
point y = −x2 /2 and we can see that the determinant of the hessian D2 f = 0. This means
that at least one eigenvalue is zero. Since the trace (the sum of the eigenvalues) is 2x2 the
other eigenvalue is strictly positive except at (0, 0). So this function is constant (in fact
equal to zero) along the parabola y = −x2 /2 and positive elsewhere.
6x 1
3
2
2
For the second function f = x + xy − x and Df = (3x + y = 1, x), D f =
so
1 0
since the determinant is negative the eigenvalues have opposite signs. This forms a saddle
at the critical point (0, 1).
In higher dimensions determining whether the matrix was positive definite might be a
bit tougher. We have 5 different methods to choose from, so one of them might work. The
determinants of the upper left squares might work efficiently for 3by3 or 4by4 matrices.
1 4
4x
x2 y
y2 ,
(x3
2xy, x2
D2 f
23. Which values of c give a bowl and which give a saddle point for the graph of z =
4x2 + 12xy + cy2 ? Describe this graph at the borderline value of c.
Solution: z = 4x2 +12xy+cy2 . This is a pure quadratic form so it could actually
bedefined
8 12
2
using a constant matrix. We have Dz = (8x + 12y, 12x + 2cy) and D z =
. This
12 2c
1 0
8
0
1 32
in turn can be written as 3
.
0 2(c − 9)
0 1
2 1
So we observe from the pivots that this will be positive definite (and bowl shaped) when
c > 9.
MTH285 HOMEWORK 5
5
Section 1.7.
3. If Q1 and Q2 are orthogonal matrices, show that their product Q1 Q2 is also an orthogonal matrix (Use AT Q = I.)
Solution: We only need to show that given Q = Q1 Q2 QT is its inverse. QQT = QT2 Q1 T Q1 Q2 .
Using the associativity we multiply out the matrices, starting with the inner most ones, to
get the identity matrix.
5. If a1 , a2 , a3 , is a basis for R 3 , any vector b can be written as
 
x1
b = x1 a1 + x2 a2 + x3 a3 or a1 a2 a3 x2  = b
x3
(a) Suppose the a’s are orthonormal. Show that x1 = aT1 b.
(b) Suppose the a’s are orthogonal. Show that x1 = aT1 b/aT1 a1 .
(c) if the a’s are independent, x1 is the first component of
times b.
 
 
x
1
x1
Solution: If the ai are orthogonal then aT1 b = aT1 a1 , a2 , a3 x2  = 1 0 0 x2  =
x3
x3
x1 If the ai are orthogonal but not unit length then you have to divide by the square of
the length aT1 a1 .
Finally if the ai ’s are only independent multiply b by the inverse of the matrix A =
[a1 , a2 , a3 ] to obtain the components of ~x.
8. Compute the norms and and condition numbers from the square roots of the eigenvalues
of AT A for each of these matrices.
1 7
1 1
1 1
1 1
0 0
−1 1
Solution: So we’ll get some help from the computer. First compute AT A. Find the eigenvalues and take the square root of the largest to find the norm. The square root of the
ratio of the max and min eigenvalues is the condition number.
M2=matrix(SR,[[1,7],[1,1]])
A2=M2.transpose()*M2
lam2 = A2.eigenvalues()
ratio = lam2[1]/lam2[0]
condition_number = ratio.n().sqrt()
√
√
2 8
The square of the original matrix is
which has eigenvalues −8 10 + 26, 8 10 + 26
8 50
and the square root
of
their
ratio
gives
the condition number 8.54970354689118. The norm
p √
of the matrix is 8 10 + 26 ≈ 7.16
For the second matrix
6
MTH285 HOMEWORK 5
M3=matrix(SR,[[1,1],[0,0]])
A3=M3.transpose()*M3
lam3 = A3.eigenvalues()
1 1
The square of the original matrix is
which has eigenvalues [0, 2]. So the norm is
1 1
√
2 and the condition number is infinity since the matrix is not invertible.
The last matrix is orthogonal (although not orthonormal).
M4=matrix(SR,[[1,1],[-1,1]])
A4=M4.transpose()*M4
lam4 = A4.eigenvalues()
2 0
The square of the matrix is
which has eigenvalues [2, 2]. Therefore the norm of
0 2
p
√
the matrix is 2. The norm of the inverse matrix is 1/2 and the condition number is 1.
Orthogonal matrices have good condition numbers.