Download ee 572 optimization theory

EE 572 OPTIMIZATION THEORY
SOLVED EXAMPLES
1-)
minimize f (x)  aT x
x  1,
subject to
x
a, x 
n
l ( x,  )  f ( x )   ( x  1 )
2
Lagrangian
  x l ( x,  )  f ( x)  2 xT  aT  2 xT  0  x  

Substitute in the constraint equation
2
1

2  a
2
a
x  
a
*
a
aT a
f (x )  
= 
 a
a
a
*
and
Note: x = xT x  x12  x22 
2

a
x 
a
2

2
x  (2 x1 2 x2
x

xn2
2 xn )T  2x
 x  2 xT
2
2-)
minimize f (x)  x
subject to aT x  c,
2
x
Lagrangian
a, x 
n
, c
l (x,  )  x   (aT x  c )
2
1
  x l (x,  )  f (x)  aT  2xT  aT  0  x   a
2
1
2
Substitute in the constraint equation  aT x    a  c
2
 x* 
ca
a
2
f ( x* ) 
and
2
c2 a
a

4
c2
a
2
3-) One-dimensional discrete-time system
x(k  1)   ( x(k ), u (k ), k )
x(0)  x0
Optimal control with terminal constraint
N
minimize
u
J   ( x( k ), u ( k ), k )
k 0
subject to
g  x( N  1)   0
where
u  u (0) u (1)
u ( N )
T


2c
a
2
z  [ x(1),
Let
=[x1 ,
, x( N ), x( N  1), u (0),
, u( N )]T
, u N ]T
, xN 1 , u0 ,
The constraint equations are:
h1 (z )   ( x0 , u0 ,0)  x1
hN 1 (z )   ( xN , u N , N )  xN 1
hN  2 (z )  g  x( N  1) 
Applying the first order necessary condition for constrained optimization:
N 2
 z J   k 1 z hk  0
(1)
k 1
where
 
z J  
 x1

xN 1

 z hk = 0


xk 1

 z hk = 0

Writing (1) explicitly gives,
k -1 = k
0


+k
0
uk
uk
N = N 1
and for k  N  1,
1
0
0

uk 1

0

k  1,..., N

0 0

g
xN 1
 
+
xk xk
 

u N 

u0
k  1,..., N
k  0,..., N
g
xN 1
Hence, the optimal control problem has a solution if there exist Lagrange multipliers
{k , k  1,..., N  1} which satisfy the above equations.
Example:
x(k  1)   x(k )  u (k )
N
J   (u (k ))  k
x(0)  F
0   1
k 0
x( N  1)  0

 ,
xk

 1,
uk

 0,
xk
g
1
xN 1
 k -1 = k
k  1,..., N
N = N 1
 k
  k
(2)
k  0,..., N
uk
Suppose  (u )  u1/ 2
(1)


1
 1/ 2  kk
uk 2uk


(3)
Eqns. (1), (2) are used to solve for the  's. Then uk 's are solved from (3).
4-)
minimize f (x)  x
subject to aT x  c, and bT x  d
2
x

g1 (x)  aT x  c  0,
a, b , x 
n
, c, d 
g 2 ( x)  bT x  d  0
KKT conditions 
f (x)  1g1 ( x)  2g 2 ( x)  0
(1)
1 (a x  c)  0
(2)
2 (bT x  d )  0
1  0
2  0
(3)
T
f ( x )  2 xT
In (1)
g1 ( x)  aT
g 2 ( x )  b T
2x  1a   2b  0
(1) 

x0
 a x  0,
T
b x0
This solution is feasible if c, d  0.
T
CASE 2 : Assume now that both constraints are active
1
1
(1)  x   1a  2 b
2
2
1
1
2

aT x   1 a  2aT b  c
2
2
1
1
2
and bT x   1bT a   2 b  d
2
2
Solution of (4), (5)
2c b  2daT b
2
1 
a
2
b  (aT b) 2
2
1  0 2  0

CASE 1 : Assume that both constraints are inactive

1  0  2  0
(4)
(5)
2d a  2caT b
2
2 
a
2
b  (aT b) 2
2
This solution is feasible if 1  0  daT b  c b
2
and
2  0  caT b  d a
Also, denominator in 1 and 2 must be greater than zero.  aT b  a b .
2
 1  0, 2  0
CASE 3 : (2) active, (3) inactive

1
1
2c
2
x   1a  aT x   1 a  c  1  2
2
2
a
1
cbT a
 bT x   1bT a 
d
2
2
a
 Feasible if cbT a  d a
1
1
2
x    2 b  bT x    2 b  d
2
2
 2 
1
daT b
 aT x    2 a T b 
c
2
2
b
2d
b
2
 Feasible if d  0.
 Feasible if daT b  c b
5-)
maximize
V ( x)  x1 x2 x3
subject to
2 x1  2 x2  4 x3  a
and
x1  0, x2  0,
x3  0

minimize f ( x)   x1 x2 x3
The constraint functions:
subject to the same constraints
g1 (x)   x1
g 2 (x)   x2
g3 (x)   x3
g 4 (x)  2 x1  2 x2  4 x3  a
The KKT conditions
2
 1  0, 2  0
CASE 4 : (2) inactive, (3) active

 Feasible if c  0.

4
f ( x )    i  g i ( x )  0
i 1
1 x1  0, 2 x2  0, 3 x3  0
1 , 2 , 3 , 4  0
2
The gradients of the functions:
f ( x)    x2 x3
 x1 x3
 x1 x2 
g1 (x)   1 0 0 g 2 ( x)  0 1 0  g 3 ( x)  0 0
 KKT :
 x2 x3  1  2 4  0
(1)
 x1 x3  2  2 4  0
(2)
 x1 x2  3  4  4  0
(3)
1 x1  0
2 x2  0
3 x3  0
4 (2 x1  2 x2  4 x3  a )  0
 1
(4)
(5)
(6)
(7)
It is in general difficult to solve the equations and determine which constraints are active and which are not.
And there is no general rule. An intuitive solution:
Solving 1 , 2 , 3 from (1),(2) and (3) and substituting in (4), (5), (6) gives
x1 (  x2 x3  24 )  0
(8)
x2 (  x1 x3  24 )  0
(9)
x3 (  x1 x2  44 )  0
(10)
Adding eqns. (8),(9),(10)
(2 x1  2 x2  4 x3 ) 4  3 x1 x2 x3
a  4  3 x1 x2 x3
Using (7),
Substituting  4 in (8),(9),(10) gives the solution for x1 , x2 , x3 as
a
a
x1  x2  , x3 
6
12
The Lagrange multipliers are:
1  2  3  0,
4 =
a2
144

4 
3 x1 x2 x3
a