Download Lyapunov Operator Let A ∈ F n×n be given, and define a linear

Lyapunov Operator
Let A ∈ Fn×n be given, and define a linear operator LA : Cn×n → Cn×n as
LA (X) := A∗X + XA
Suppose A is diagonalizable (what follows can be generalized even if this is
not possible - the qualitative results still hold). Let {λ1 , λ2 , . . . , λn } be the
eigenvalues of A. In general, these are complex numbers. Let V ∈ Cn×n be
the matrix of eigenvectors for A∗ , so
V = [v1 v2 · · · vn ] ∈ Cn×n
is invertible, and for each i
A∗vi = λ̄i vi
Let wi ∈ Cn be defined so that

V −1 =










w1T
w2T
..
.
wnT











Since V −1V = In, we have that



wiT vj = δij := 
For each 1 ≤ i, j ≤ n define

0
1
if i 6= j
if i = j
Xij := vivj∗ ∈ Cn×n
130
Eigenvalues
Lyapunov Operator
Lemma 63 The set {Xij }i=1,.,n;j=1,.,n is a linearly independent set, and this
set is a full set of eigenvectors for the linear operator LA . Moreover, the
o
n
eigenvalues of LA is the set of complex numbers λ̄i + λj i=1,.,n;j=1,.,n
Proof: Suppose {αij }i=1,.,n;j=1,.,n are scalars and
n X
n
X
αij Xij = 0n×n
(7.19)
j=1 i=1
Premultiply this by wlT and postmultiply by w̄k

0 = wlT 
=
=
n
n X
X

αij Xij  w̄k
j=1 i=1
n X
n
X
αij wlT vivj∗ w̄k
j=1 i=1
n
n X
X
αij δliδkj
j=1 i=1
= αlk
which holds for any 1 ≤ k, l ≤ n. Hence, all of the α’s are 0, and the set
{Xij }i=1,.,n;j=1,.,n is indeed a linearly independent set. Also, by definition
of LA , we have
LA (Xij ) = A∗Xij + Xij A
= A∗vi vj∗ + vi vj∗A
= λ̄i vivj∗ + vi λj vj∗
=
=
λ̄i + λj vi vj∗
λ̄i + λj Xij
So, Xij is indeed an eigenvector of LA with eigenvalue λ̄i + λj . ♯
131
Lyapunov Operator
Alternate Invertibility Proof
Do a Schur decomposition of A, namely A = QΛQ∗ , with Q unitary and Λ
upper triangular. Then the equation A∗X + XA = R can be written as
A∗X + XA = R ↔ QΛ∗Q∗ X + XQΛQ∗ = R
↔ Λ∗ Q∗XQ + Q∗ XQΛ = Q∗RQ
↔ Λ∗ X̃ + X̃Λ = R̃
Since Q is invertible, the redefinition of variables is an invertible transformation.
Writing out all of the equations (we are trying to get that for every R̃, there
is a unique X̃ satisfying the equation) yields a upper triangular system with
n2 unknowns, and the values λi + λ¯j on the diagonal. Do it. ♯
This is an easy (and computationally motivated) proof that the Lyapunov
operator LA is an invertible operator if and only if λi + λ̄j 6= 0 for all eigenvalues of A.
132
Lyapunov Operator
Integral Formula
If A is stable (all eigenvalues have negative real parts) then λ̄i + λj 6= 0
for all combinations. Hence LA is an invertible linear operator. We can
explicitly solve the equation
LA (X) = −Q
for X using an integral formula.
Theorem 64 Let A ∈ Fn×n be given, and suppose that A is stable. The LA
is invertible, and for any Q ∈ Fn×n, the unique X ∈ Fn×n solving LA (X) =
−Q is given by
Z ∞
∗
X=
eA τ QeAτ dτ
0
Proof: For each t ≥ 0, define
S(t) :=
Z t
∗
eA τ QeAτ dτ
0
Since A is stable, the integrand is made up of decaying exponentials, and
S(t) has a well-defined limit as t → ∞, which we have already denoted
X := lim S(t). For any t ≥ 0,
t→∞
∗
Z t
∗ A∗ τ
Aτ
A∗ τ
A e Qe + e
d A∗ τ Aτ =
dτ
e Qe
0 dτ
∗
= eA t QeAt − Q
A S(t) + S(t)A =
0
Z t
Taking limits on both sides gives
A∗X + XA = −Q
as desired. ♯
133
Aτ
Qe A dτ
Lyapunov Operator
More
A few other useful facts
Theorem 65 Suppose A, Q ∈ Fn×n are given. Assume that A is stable and
Q = Q∗ ≥ 0. Then the pair (A, Q) is observable if and only if
X :=
Z ∞
∗
eA τ QeAτ dτ
0
>0
Proof: ⇐ Suppose not. Then there is an xo ∈ Fn, xo 6= 0 such that
QeAt xo = 0
∗
for all t ≥ 0. Consequently x∗o eA t QeAt xo = 0 for all t ≥ 0. Integrating gives
R ∗
0 = 0∞ x∗o eA t QeAt xo dt
= x∗o
R
∞ A∗ t
At
e
Qe
dt
xo
0
= x∗o Xxo
Since xo 6= 0, X is not positive definite.
⇒ Suppose that X is not positive definite (note by virtue of the definition X is at least positive semidefinite). Then there exists xo ∈ Fn,
xo 6= 0 such that x∗o Xxo = 0. Using the integral form for X (since
A is stable by assumption) and the fact tha Q ≥ 0 gives
Z ∞ 1
Q 2 eAτ x dτ
o
0
=0
The integrand is nonnegative and continuous, hence it must be 0
for all τ ≥ 0. Hence QeAτ xo = 0 for all τ ≥ 0. Since xo 6= 0, this
implies that (A, Q) is not observable. ♯
134
Lyapunov Operator
More
Theorem 66 Let (A, C) be detectable and suppose X is any solution to
A∗X + XA = −C ∗C (there is no apriori assumption that LA is an invertible
operator, hence there could be multiple solutions). Then X ≥ 0 if and only
if A is stable.
Proof ⇒ Suppose not, then there is a v ∈ Cn , v 6= 0, λ ∈ C, Re (λ) ≥ 0
with Av = λv (and v ∗ A∗ = λ̄v ∗). Since (A, C) is detectable and the
eigenvalue in the closed right-half plane Cv 6= 0. Note that
− kCvk2 = −v ∗ C ∗Cv
= v ∗ (A∗ X + XA) v
=
λ̄ + λ v ∗ Xv
= 2Re (λ) v ∗ Xv
Since kCvk > 0 and Re (λ) ≥ 0, we must actually have Re (λ) > 0
and v ∗Xv < 0. Hence X is not positive semidefinite.
⇐ Since A is stable, there is only one solution to the equation A∗X +
XA = −C ∗C, moreover it is given by the integral formula,
X=
Z ∞
∗
eA τ C ∗CeAτ dτ
0
which is clearly positive semidefinite. ♯
135
Lyap
Ordering
Finally, some simple results about the ordering of Lyapunov solutions.
Let A be stable, so LA is guaranteed to be invertible. Use L−1
A (−Q) to
∗
denote the unique solution X to the equation A X + XA = −Q.
Lemma 67 If Q∗1 = Q1 ≥ Q2 = Q∗2, then
−1
L−1
A (−Q1 ) ≥ LA (−Q2 )
Lemma 68 If Q∗1 = Q1 > Q2 = Q∗2, then
−1
L−1
A (−Q1 ) > LA (−Q2 )
Proofs Let X1 and X2 be the two solutions, obviously
A∗ X1 + X1 A = −Q1
A∗ X2 + X2 A = −Q2
A∗ (X1 − X2 ) + (X1 − X2 ) A = − (Q1 − Q2)
Since A is stable, the integral formula applies, and
X1 − X2 =
Z ∞
∗
eA τ
0
(Q1 − Q2) eAτ dτ
Obviously, if Q1 − Q2 ≥ 0, then X1 − X2 ≥ 0, which proves Lemma 5. If
Q1 − Q2 > 0, then the pair (A, Q1 − Q2) is observable, and by Theorem
3, X1 − X2 > 0, proving Lemma 6. ♯
Warning: The converses of Lemma 5 and Lemma 6 are NOT true.
136
Algebraic Riccati Equation
The algebraic Riccati equation (A.R.E.) is
AT X + XA + XRX − Q = 0
(7.20)
where A, R, Q ∈ Rn×n are given matrices, with R = RT , and Q = QT , and
solutions X ∈ Cn×n are sought. We will primarily be interested in solutions
X of (7.20), which satisfy
A + RX
is Hurwitz
and are called stabilizing solutions of the Riccati equation.
Associated with the data A, R, and Q, we define the Hamiltonian matrix
H ∈ R2n×2n by


A R 

(7.21)
H := 
Q −AT
137
Invariant Subspace
Definition
Suppose A ∈ Cn×n. Then (via matrix-vector multiplication), we view A as
a linear map from Cn → Cn .
A subspace V ⊂ Cn is said to be “A-invariant” if for every vector v ∈ V, the
vector Av ∈ V too.
In terms of matrices, suppose that the columns of W ∈ Cn×m are a basis for
V. In other words, they are linearly independent, and span the space V, so
V = {W η : η ∈ Cm }
Then V being A-invariant is equivalent to the existence of a matrix S ∈ Cm×m
such that
AW = W S
The matrix S is the matrix representation of A restricted to V, referred to
the basis W . Clearly, since W has linearly independent columns, the matrix
S is unique.
If (Q, Λ) are the Schur decomposition of A, then for any k with 1 ≤ k ≤ n,
the first k columns of Q span a k-dimensional invariant subspace of A, as
seen below


Λ
Λ
11
12 

A [Q1 Q2 ] = [Q1 Q2] 
0 Λ22
Clearly, AQ1 = Q1Λ11 .
138
Eigenvectors in Invariant subspaces
Note that since S ∈ Cm×m has at least one eigenvector, say ξ 6= 0m, with
associated eigenvalue λ, it follows that W ξ ∈ V is an eigenvector of A,
A (W ξ) = AW ξ = W Sξ = λW ξ = λ(W ξ)
Hence, every nontrivial invariant subspace of A contains an eigenvector of A.
139
Stable Invariant Subspace
Definition
If S is Hurwitz, then V is a “stable, A-invariant” subspace.
Note that while the matrix S depends on the particular choice of the basis for
V (ie., the specific columns of W ), the eigenvalues of S are only dependent
on the subspace V, not the basis choice.
To see this, suppose that the columns of W̃ ∈ Cn×m span the same space as
the columns of W .
Obviously, there is a matrix T ∈ Cm×m such that W = W̃ T . The linear
independence of the columns of W (and W̃ ) imply that T is invertible. Also,
−1
AW̃ = AW T −1 = W ST −1 = W̃ T
{z }
| ST
S̃
Clearly, the eigenvalues of S and S̃ are the same.
The linear operator A|V is the transformation from V → V defined as A,
simply restricted to V.
Since V is A-invariant, this is well-defined. Clearly, S is the matrix representation of this operator, using the columns of W as the basis choice.
140
AT X + XA + XRX − Q = 0
Invariant Subspace
The first theorem gives a way of constructing solutions to (7.20) in terms of
invariant subspaces of H.
Theorem 69 Let V ⊂ C2n be a n-dimensional invariant subspace of H (H
is a linear map from C2n to C2n ), and let X1 ∈ Cn×n, X2 ∈ Cn×n be two
complex matrices such that

V = span 
X1
X2



If X1 is invertible, then X := X2X1−1 is a solution to the algebraic Riccati
equation, and spec (A + RX) = spec (H|V ).
Proof: Since V is H invariant, there is a matrix Λ ∈ Cn×n with



which gives
A
R
Q −AT



X1
X2




= 
X1
X2


Λ
(7.22)
AX1 + RX2 = X1 Λ
(7.23)
QX1 − AT X2 = X2Λ
(7.24)
and
Pre and post multiplying equation (7.23) by X2X1−1 and X1−1 respectively gives
X2 X1−1 A + X2X1−1 R X2 X1−1 = X2 ΛX1−1
(7.25)
and postmultiplying 7.24 by X1−1 gives
Q − AT X2 X1−1 = X2ΛX1−1
(7.26)
Combining (7.25) and (7.26) yields
AT (X2X1−1 ) + (X2X1−1 )A + (X2 X1−1)R(X2X1−1 ) − Q = 0
so X2 X1−1 is indeed a solution. Equation (7.23) implies that A+R X2 X1−1 =
X1 ΛX1−1 , and hence they have the same eigenvalues. But, by definition,
Λ is a matrix representation of the map H|V , so the theorem is proved.
♯
141
AT X + XA + XRX − Q = 0
Invariant Subspace
The next theorem shows that the specific choice of matrices X1 and X2 which
span V is not important.
Theorem 70 Let V, X1, and X2 be as above, and suppose
there
are two


X̃1 
 also span
other matrices X̃1 , X̃2 ∈ Cn×n such that the columns of 
X̃2
V. Then X1 is invertible if and only if X̃1 is invertible. Consequently, both
X2 X1−1 and X̃2 X̃1−1 are solutions to (7.20), and in fact they are equal.
Remark: Hence, the question of whether or not X1 is invertible is a property
of the subspace, and not of the particular basis choice for the subspace.
Therefore, we say that an n-dimensional subspace V has the invertibility
property if in any basis, the top portion is invertible. In the literature,
this is often called the complimentary property.
Proof: Since both sets of columns span the same n-dimensional subspace,
there is an invertible K ∈ Cn×n such that



X1
X2




= 
X̃1
X̃2


K
Obviously, X1 is invertible if and only if X̃1 is invertible, and X2 X1−1 =
−1
X̃2 K X̃1 K
= X̃2 X̃1−1 . ♯.
142
AT X + XA + XRX − Q = 0
Invariant Subspace
As we would hope, the converse of theorem 69 is true.
Theorem 71 If X ∈ Cn×n is a solution to the A.R.E., then there exist
matrices X1, X2 ∈ Cn×n , with X1 invertible, such that X = X2 X1−1 and the
X1 
 span an n-dimensional invariant subspace of H.
columns of 
X2
Proof: Define Λ := A + RX ∈ Cn×n. Multiplying this by X gives XΛ =
XA + XRX = Q − AT X, the second equality coming from the fact that
X is a solution to the Riccati equation. We write these two relations as



Hence, the columns of
A
R
Q −A

In
T



In
X




= 
In
X


Λ

span an n-dimensional invariant subspace
X
of H, and defining X1 := In , and X2 := X completes the proof. ♯.




Hence, there is a one-to-one correspondence between n-dimensional invariants subspaces of H with the invertibility property, and solutions of the
Riccati equation.
143
AT X + XA + XRX − Q = 0
Eigenvalue distribution of H
In the previous section, we showed that any solution X of the Riccati equation is intimately related to an invariant subspace, V, of H, and the matrix
A + RX is similiar (ie. related by a similarity transformation) to the restriction of H on V, H|V . Hence, the eigenvalues of A + RX will always
be a subset of the eigenvalues of H. Recall the particular structure of the
Hamiltonian matrix H,


A
R


H := 
T
Q −A
Since H is real, we know that the eigenvalues of H are symmetric about the
real axis, including multiplicities. The added structure of H gives additional
symmetry in the eigenvalues as follows.
Lemma 72 The eigenvalues of H are symmetric about the origin (and hence
also about the imaginary axis).
Proof: Define J ∈ R2n×2n by

J = 
0 −I
I
0



Note that JHJ −1 = −H T . Also, JH = −H T J = (JH)T . Let p (λ) be
the characteristic polynomial of H. Then
p(λ) := det (λI − H)
= det λI − JHJ −1
= det λI + H T
= det (λI + H)
(7.27)
= (−1)2ndet (−λI − H)
= p(−λ)
Hence the roots of p are symmetric about the origin, and the symmetry
about the imaginary axis follows. ♯
144
AT X + XA + XRX − Q = 0
Hermitian Solutions
Lemma 73 If H has no eigenvalues on the imaginary axis, then H has n
eigenvalues in the open-right-half plane, and n eigenvalues in the open-lefthalf plane.
Theorem 74 Suppose V is a n-dimensional invariant subspace of H, and
X1 , X2 ∈ Cn×n form a matrix whose colums span V. Let Λ ∈ Cn×n be a
matrix representation of H|V . If λi + λj 6= 0 for all eigenvalues λi , λj ∈
spec(Λ), then X1∗ X2 = X2∗ X1 .
Proof: Using the matrix J from the proof of Lemma 72 we have that
[X1∗ X2∗ ] JH



X1
X2



is Hermitian, and equals (−X1∗ X2 + X2∗ X1 ) Λ. Define W := −X1∗ X2 +
X2∗ X1 . Then W = −W ∗, and W Λ + Λ∗ W = 0. By the eigenvalue
assumption on Λ, the linear map LΛ : Cn×n → Cn×n defined by
LΛ (X) = XΛ + Λ∗X
is invertible, so W = 0. ♯
Corollary 75 Under the same assumptions as in theorem 74, if X1 is invertible, then X2 X1−1 is Hermitian.
145
AT X + XA + XRX − Q = 0
Real Solutions
Real (rather than complex) solutions arise when a symmetry condition is
imposed on the invariant subspace.
Theorem 76 Let V be a n-dimensional invariant subspace of H with the
invertibility property, and let X1, X2 ∈ Cn×n form a 2n × n matrix whose
columns span V. Then V is conjugate symmetric ({v̄ : v ∈ V} = V) if and
only if X2 X1−1 ∈ Rn×n .
Proof: ← Define X := X2 X1−1 . By assumption, X ∈ Rn×n , and

span 
In
X



=V
so V is conjugate symmetric.
→ Since V is conjugate symmetric, there is an invertible matrix K ∈
Cn×n such that



X1
X2




= 
X1
X2


K
Therefore, (X2X1−1 ) = X2X1−1 = X2 K(X1K)−1 = X2 X1−1 as desired. ♯
146
AT X + XA + XRX − Q = 0
Stabilizing Solutions
Recall that for any square matrix M, there is a “largest” stable invariant
subspace. That is, there is a stable invariant subspace VS∗ that contains every
other stable invariant subspace of M. This largest stable invariant subspace
is simply the span of all of the eigenvectors and generalized eigenvectors
associated with the open-left-half plane eigenvalues. If the Schur form has
the eigenvalues (on diagonal of Λ) ordered, then it is spanned by the “first”
set of columns of Q
Theorem 77 There is at most one solution X to the Riccati equation such
that A + RX is Hurwitz, and if it does exist, then X is real, and symmetric.
Proof: If X is such a stabilizing solution, then by theorem 71 it can be
constructed from some n-dimensional invariant subspace of H, V. If
A + RX is stable, then we must also have spec (H|V ). Recall that the
spectrum of H is symmetric about the imaginary axis, so H has at
◦
most n eigenvalues in C− , and the only possible choice for V is the
stable eigenspace of H. By theorem 70, every solution (if there are any)
constructed from this subspace will be the same, hence there is at most
1 stabilizing solution.
Associated with the stable eigenspace, any Λ as in equation (7.22) will
be stable. Hence, if the stable eigenspace has the invertibility property,
from corollary 75 we have X := X2X1−1 is Hermitian. Finally, since H
is real, the stable eigenspace of H is indeed conjugate symmetric, so the
associated solution X is in fact real, and symmetric.
147
AT X + XA + XRX − Q = 0
domS Ric
If H is (7.21) has no imaginary axis eigenvalues, then H has a unique ndimensional stable invariant subspace VS ⊂ C2n, and since H is real, there
exist matrices X1 , X2 ∈ Rn×n such that

span 

X1


X2
= VS .
More concretely, there exists a matrix Λ ∈ Rn×n such that

H 
X1
X2




= 
X1
X2


Λ
◦
spec (Λ) ⊂ C−
,
This leads to the following definition
Definition 78 (Definition of domS Ric) Consider H ∈ R2n×2n as defined
in (7.21). If H has no imaginary axis eigenvalues, and the matrices X1 , X2 ∈
Rn×n for which

span 
X1
X2



is the stable, n-dimensional invariant subspace of H satisfy det (X1 ) 6= 0,
then H ∈ domS Ric. For H ∈ domS Ric, define RicS (H) := X2 X1−1 , which is
the unique matrix X ∈ Rn×n such that A + RX is stable, and
AT X + XA + XRX − Q = 0.
Moreover, X = X T .
148
AT X + XA + XRX − Q = 0
Useful Lemma
Theorem 79 Suppose H (as in (7.21)) does not have any imaginary axis
eigenvalues, R is either positive semidefinite, or negative semidefinite, and
(A, R) is a stabilizable pair. Then H ∈ domS Ric.

X1

form a basis for the stable, n-dimensional invariant subX2
space of H (this exists, by virtue of the eigenvalue assumption about
H). Then





X
X
A
R
1  Λ
1 




=


(7.28)
X2
X2
Q −AT
Proof: Let




where Λ ∈ Cn×n is a stable matrix. Since Λ is stable, it must be that
X1∗ X2 = X2∗ X1 . We simply need to show X1 is invertible. First we prove
that KerX1 is Λ-invariant, that is, the implication
x ∈ KerX1 → Λx ∈ KerX1
Let x ∈ KerX1 . The top row of (7.28) gives RX2 = X1Λ − AX1.
Premultiply this by x∗X2∗ to get
x∗X2∗ RX2 x = x∗X2∗ (X1 Λ − AX1 ) x
= x∗X2∗ X1Λx
= x∗X1∗ X2Λx
since X2∗ X1 = X1∗ X2
.
= 0
Since R ≥ 0 or R ≤ 0, it follows that RX2 x = 0. Now, using (7.28)
again gives X1 Λx = 0, as desired. Hence KerX1 is a subspace that is
Λ-invariant. Now, if KerX1 6= {0}, then there is a vector v 6= 0 and
λ ∈ C, Re (λ) < 0 such that
X1 v = 0
Λv = λv
RX2 v = 0
149
We also have QX1 − AT X2 = X2 Λ, from the second row of (7.28).
Mutiplying by v ∗ gives
v
∗
X2∗
h
A − −λ̄I
i
R =0
Since Re λ̄ < 0, and (A, R) is assumed stabilizable, it must be that

X1

is full-column rank. Hence, v =
X2
0, and KerX1 is indeed the trivial set {0}. This implies that X1 is
invertible, and hence H ∈ domS Ric. ♯.
X2 v = 0. But X1 v = 0, and


150


AT X + XA + XRX − Q = 0
Quadratic Regulator
The next theorem is a main result of LQR theory.
Theorem 80 Let A, B, C be given, with (A, B) stabilizable, and (A, C)
detectable. Define


T
A
−BB 

H := 
−C T C −AT
Then H ∈ domS Ric. Moreover, 0 ≤ X := RicS (H), and

KerX ⊂










C
CA
..
.
CAn−1











Proof: Use stabilizablity of (A, B) and detectability of (A, C) to show that
H has no imaginary axis eigenvalues. Specifically, suppose v1, v2 ∈ Cn
and ω ∈ R such that



A
T
−C C
−BB T
−A
T



v1
v2




= jω 
v1
v2



Premultiply top and bottom by v2∗ and v1∗, combine, and conclude v1 =
T
v2 = 0. Show that if (A, B) is stabilizable, then A, −BB is also stabilizable. At this point , apply Theorem 79 to conclude H ∈ domS Ric.
Then, rearrange the Riccati equation into a “Lyapunov” equation and
use Theorem 64 to show that X ≥ 0. Finally, show that KerX ⊂ KerC,
and that KerX is A-invariant. That shows that KerX ⊂ KerCAj for
any j ≥ 0. ♯.
151
Derivatives
Linear Maps
Recall that the derivative of f at x is defined to be the number lx such that
f (x + δ) − f (x) − lx δ
=0
δ→0
δ
lim
Suppose f : Rn → Rm is differentiable. Then f ′(x) is the unique matrix
Lx ∈ Rm×n which satisfies
1
(f (x + δ) − f (x) − Lx δ) = 0
δ→0n kδk
lim
Hence, while f : Rn → Rm , the value of f ′ at a specific location x, f ′ (x),
is a linear operator from Rn → Rm . Hence the function f ′ is a map from
Rn → L (Rn , Rm ).
Similarly, if f : Hn×n → Hn×n is differentiable, then the derivative of f at x
is the unique linear operator LX : Hn×n → Hn×n such that satisfying
1
(f (X + ∆) − f (X) − LX ∆) = 0
∆→0n×n k∆k
lim
152
Derivative of Riccati
Lyapunov
f (X) := XDX − A∗ X − XA − C
Note,
f (X + ∆) = (X + ∆) D (X + ∆) − A∗ (X + ∆) − (X + ∆) A − C
= XDX − A∗X − XA − C + ∆DX + XD∆ − A∗ ∆ − ∆A + ∆D∆
= f (X) − ∆ (A − DX) − (A − DX)∗ ∆ + ∆D∆
For any W ∈ Cn×n , let LW : Hn×n → Hn×n be the Lyapunov operator
LW (X) := W ∗ X + XW . Using this notation,
f (X + ∆) − f (X) − L−A+DX (∆) = ∆D∆
Therefore
1
1
[f (X + ∆) − f (X) − L−A+DX (∆)] = lim
[∆D∆] = 0
∆→0n×n k∆k
∆→0n×n k∆k
lim
This means that the Lyaponov operator L−A+DX is the derivative of the
matrix-valued function
XDX − A∗ X − XA − C
at X.
153
Iterative Algorithms
Roots
Suppose f : R → R is differentiable. An iterative algorithm to find a root
of f (x) = 0 is
f (xk )
xk+1 = xk − ′
f (xk )
Suppose f : Rn → Rn is differentiable. Then to first order
f (x + δ) = f (x) + Lx δ
An iterative algorithm to find a “root” of f (x) = 0n is obtained by setting
the first-order approximation expression to zero, and solving for δ, yielding
the update law (with xk+1 = xk + δ)
xk+1 = xk − L−1
xk [f (xk )]
Implicitly, it is written as
Lxk [xk+1 − xk ] = −f (xk )
(7.29)
For the Riccati equation
f (X) := XDX − A∗ X − XA − C
we have derived that the derivative of f at X involves the Lyapunov operator.
The iteration in (7.29) appears as
L−A+DXk (Xk+1 − Xk ) = −Xk DXk + A∗Xk + Xk A + C
In detail, this is
− (A − DXk )∗ (Xk+1 − Xk )−(Xk+1 − Xk ) (A − DXk ) = −Xk DXk +A∗Xk +Xk A+C
which simplifies (check) to
(A − DXk )∗ Xk+1 + Xk+1 (A − DXk ) = −Xk DXk − C
Of course, questions about the well-posedness and convergence of the iteration must be addressed in detail.
154
Comparisons
Gohberg, Lancaster and Rodman
Theorems 2.1-2.3 in “On Hermitian Solutions of the Symmetric Algebraic
Riccati Equation,” by Gohberg, Lancaster and Rodman, SIAM Journal of
Control and Optimization, vol. 24, no. 6, Nov. 1986, are the basis for comparing equality and inequality solutions of Riccati equations. The definitions,
theorems and main ideas are below.
Suppose that A ∈ Rn×n , C ∈ Rn×n , D ∈ Rn×n , with D = DT 0, C = C T
and (A, D) stabilizable. Consider the matrix equation
XDX − AT X − XA − C = 0
(7.30)
Definition 81 A symmetric solution X+ = X+T of (9.39) is maximal if
X+ X for any other symmetric solution X of (9.39). Check: Maximal
solutions (if they exist) are unique.
Theorem 82 If there is a symmetric solution to (9.39), then there is a
maximal solution X+ . The maximal solution satisfies
max Reλi (A − DX+ ) ≤ 0
i
Theorem 83 If there is a symmetric solution to (9.39), then there is a
T
sequence {Xj }∞
j=1 such that for each j, Xj = Xj and
0 Xj DXj − AT Xj − Xj A − C
Xj X
for any X = X T solving 9.39
A − DXj is Hurwitz
limj→∞ Xj = X+
Theorem 84 If there are symmetric solutions to (9.39), then for every symmetric C̃ C, there exists symmetric solutions (and hence a maximal solution X̃+) to
XDX − AT X − XA − C̃ = 0
Moreover, the maximal solutions are related by X̃+ X+ .
155
Identities
For any Hermitian matrices Y and Ŷ , it is trivial to verify
Y (A − DY ) + (A − DY )∗Y + Y DY
∗
= Y (A − DŶ ) + (A − DŶ ) Y + Ŷ DŶ − Y − Ŷ D Y − Ŷ
(7.31)
Let X denote any Hermitian solution (if one exists) to the ARE,
XDX − A∗ X − XA − C = 0
(7.32)
This (the existence of a hermitian solution) is the departure point for all
that will eventually follow.
For any hermitian matrix Z
−C = −XDX + A∗X + XA
= X (A − DX) + (A − DX)∗ X + XDX
= X (A − DZ) + (A − DZ)∗ X + ZDZ − (X − Z) D (X − Z)
(7.33)
This exploits that X solves the original Riccati equation, and uses the identity (7.31), with Y = X, Ŷ = Z.
156
Iteration
Take any C̃ C (which could be C, for instance). Also, let X0 = X0∗ 0
be chosen so that A − DX0 is Hurwitz. Check: Why is this possible?
Iterate (if possible, which we will show is...) as
Xν+1 (A − DXν ) + (A − DXν )∗ Xν+1 = −Xν DXν − C̃
(7.34)
Note that by the Hurwitz assumption on A − DX0, at least the first iteration
is possible (ν = 0). Also, recall that the iteration is analogous to a first-order
n)
algorithm for root finding, xn+1 := xn − ff′(x
(xn ) .
Use Z := Xν in equation (7.33), and subtract from the iteration equation
(7.34), leaving
(Xν+1 − X) (A − DXν ) + (A − DXν )∗ (Xν+1 − X)
= − (X − Xν ) D (X − Xν ) − C̃ − C
which is valid whenever the iteration equation is valid.
157
(7.35)
Main Induction
Now for the induction: assume Xν = Xν∗ and A − DXν is Hurwitz. Note
that this is true for ν = 0.
Since A−DXν is Hurwitz, there is a unique solution Xν+1 to equation (7.34).
Moreover, since A−DXν is Hurwitz, and the right-hand-side of (7.35) is 0,
it follows that Xν+1 − X 0 (note that X0 − X is not necessarily positive
semidefinite).
Rewrite (7.31) with Y = Xν+1 and Ŷ = Xν .
Xν+1 (A − DXν+1 ) + (A − DXν+1 )∗ Xν+1 + Xν+1 DXν+1
= Xν+1 (A − DXν ) + (A − DXν )∗ Xν+1 + Xν DXν − (Xν+1 − Xν ) D (Xν+1 − Xν )
|
{z
−C̃
Rewriting,
}
−C̃ = Xν+1 (A − DXν+1 ) + (A − DXν+1 )∗ Xν+1
+ Xν+1 DXν+1 + (Xν+1 − Xν ) D (Xν+1 − Xν )
Writing (7.33) with Z = Xν+1 gives
−C = X (A − DXν+1 ) + (A − DXν+1 )∗ X
+ Xν+1 DXν+1 − (X − Xν+1) D (X − Xν+1 )
Subtract these, yielding
− C̃ − C = (Xν+1 − X) (A − DXν+1 ) + (A − DXν+1 )∗ (Xν+1 − X)
+ (Xν+1 − Xν ) D (Xν+1 − Xν ) + (X − Xν+1 ) D (X − Xν+1 )
(7.36)
Now suppose that Re (λ) ≥ 0, and v ∈ Cn such that (A − DXν+1 ) v = λv.
Pre and post-multiply (7.36) by v ∗ and v respectively. This gives
−v ∗ C̃ − C v = 2Re(λ) v ∗ (Xν+1 − X) v +
|
{z
≤0
}
|
{z
≥0
} |
{z
≥0
}



|
D1/2 (Xν+1 − Xν )
D1/2 (X − Xν+1 )
Hence, both sides are in fact equal to zero, and therefore
v ∗ (Xν+1 − Xν ) D (Xν+1 − Xν ) = 0.
158
{z
≥0
 2
  v
}
Since D 0, it follows that D (Xν+1 − Xν ) v = 0, which means DXν+1 v =
DXν v. Hence
λv = (A − DXν+1 ) v = (A − DXν ) v
But A − DXν is Hurwitz, and Re(λ) ≥ 0, so v = 0n as desired. This means
A − DXν+1 is Hurwitz.
Summarizing:
• Suppose there exists a hermitian X = X ∗ solving
XDX − A∗X − XA − C = 0
• Let X0 = X0∗ 0 be chosen so A − DX0 is Hurwitz
• Let C̃ be a Hermitian matrix with C̃ C.
• Iteratively define (if possible) a sequence {Xν }∞
ν=1 via
Xν+1 (A − DXν ) + (A − DXν )∗ Xν+1 = −Xν DXν − C̃
The following is true:
• Given the choice of X0 and C̃, the sequence {Xν }∞
ν=1 is well-defined,
∗
unique, and has Xν = Xν .
• For each ν ≥ 0, A − DXν is Hurwitz
• For each ν ≥ 1, Xν X.
It remains to show that for ν ≥ 1, Xν+1 Xν (although it is not necessarily
true that X1 X0 ).
159
Decreasing
Proof
For ν ≥ 1, apply (7.31) with Y = Xν and Ŷ = Xν−1 , and substitute the
iteration (shifted back one step) equation (7.34) giving
Xν (A − DXν ) + (A − DXν )∗ Xν + Xν DXν
= −C̃ − (Xν − Xν−1) D (Xν − Xν−1 )
The iteration equation is
Xν+1 (A − DXν ) + (A − DXν )∗ Xν+1 = −Xν DXν − C̃
Subtracting leaves
(Xν − Xν+1 ) (A − DXν ) + (A − DXν )∗ (Xν − Xν+1 )
= (Xν − Xν−1 ) D (Xν − Xν−1 )
The fact that A − DXν is Hurwitz and D 0 implies that Xν − Xν+1 0.
Hence, we have shown that for all ν ≥ 1,
X Xν+1 Xν
160
Monotonic Matrix Sequences
Convergence
Every nonincreasing sequence of real numbers which is bounded below has
a limit. Specifically, if {xk }∞
k=0 is a sequence of real numbers, and there is a
real number γ such that xk ≥ γ and xk+1 ≤ xk for all k, then there is a real
number x such that
lim xk = x
k→∞
A similar result holds for symmetric matrices: Specifically, if {Xk }∞
k=0 is a
sequence of real, symmetric matrices, and there is a real symmetric matrix
Γ such that Xk Γ and Xk+1 Xk for all k, then there is a real symmetric
matrix X such that
lim Xk = X
k→∞
n
o∞
This is proven by first considering the sequence eTi Xk ei k=0, which shows
that all of the diagonal entries of the sequence have limits. Then look at
n
o∞
(ei + ej )T Xk (ei + ej ) k=0 to conclude convergence of off-diagonal terms.
161
Limit of Sequence
Maximal Solution
The sequence {Xν }∞
ν=0 , generated by (7.34), has a limit
lim Xν =: X̃+
ν→∞
Remember that C̃ was any hermitian matrix C.
Facts:
• Since each A − DXν is Hurwitz, in passing to the limit we may get
eigenvalues with zero real-parts, hence
max Reλi (A − DX+ ) ≤ 0
i
• Since each Xν = Xν∗ , it follows that X̃+∗ = X̃+ .
• Since each Xν X (where X is any hermitian solution to the equality
XDX − A∗ X − XA − C = 0, equation (7.32)), it follows by passing to
the limit that X̃+ X.
• The iteration equation is
Xν+1 (A − DXν ) + (A − DXν )∗ Xν+1 = −Xν DXν − C̃
Taking limits yields
∗
X̃+ A − DX̃+ + A − DX̃+ X̃+ = −X̃+DX̃+ − C̃
which has a few cancellations, leaving
X̃+ A + A∗ X̃+ − X̃+ DX̃+ + C̃ = 0
• Finally, if X+ denotes the above limit when C̃ = C, it follows that the
above ideas hold, so X+ X, for all solutions X.
• And really finally, since X+ is a hermitian solution to (7.32) with C, the
properties of X̃+ imply that X̃+ X+ .
162
dx2 − 2ax − c = 0
Analogies with Scalar Case
Matrix Statement
Scalar Specialization
D = D∗ 0
d ∈ R, with d ≥ 0
(A, D) stabilizable
a < 0 or d > 0
existence of Hermitian solution a2 + cd ≥ 0
√
a+ a2 +cd
for d
d
c
for d = 0
−2a
Maximal solution X+
x+ =
Maximal solution X+
x+ =
√
− a2 + cd for d > 0
A − DX+
A − DX+
>0
a for d = 0
A − DX0 Hurwitz
x0 >
A − DX Hurwitz
x>
a
d
for d > 0
A − DX0 Hurwitz
a < 0 for d = 0
A − DX Hurwitz
a < 0 for d = 0
a
d
for d > 0
If d = 0, then the graph of −2ax − c (with a < 0) is a straight line with a
c
.
positive slope. There is one (and only one) real solution at x = − 2a
If d > 0, then the graph of dx2 − 2ax − c is a upward directed parabola.
The minimum occurs at ad . There are either 0, 1, or 2 real solutions to
dx2 − 2ax − c = 0.
• If a2 + cd < 0, then there are no solutions (the minimum is positive)
• If a2 + cd = 0, then there is one solution, at x = ad , which is where the
minimum occurs.
a
d
2
• If a + cd √> 0, then there are two solutions, at x− = −
2
x+ = ad + a d+cd which are equidistant from the minimum ad .
163
√
a2 +cd
d
and