Download Chapter 1 Basic Theorems from Optimal Control Theory

Chapter 1
Basic Theorems from Optimal Control
Theory
This chapter is taken from Greiner and Fincke (2015).
In this book we have assumed that economic agents are rational, behave intertemporally and perform dynamic optimization. In this appendix we present some basics of the
method of dynamic optimization using Pontryagin’s maximum principle and the Hamiltonian.
Let an intertemporal optimization problem be given by
Z ∞
e−ρt F (x(t), u(t))dt
(1.1)
max W (x(0), 0), W (·) ≡
u(t)
0
subject to
dx(t)
≡ ẋ(t) = f (x(t), u(t)), x(0) = x0
(1.2)
dt
with x(t) ∈ IRn the vector of state variables at time t and u(t) ∈ Ω ∈ IRm the vector of
control variables at time t and F : IRn × IRm → IR and f : IRn × IRm → IRn . ρ is the
discount rate and e−ρt is the discount factor.
F (x(t), u(t)), fi (x(t), u(t)) and ∂fi (x(t), u(t))/∂xj (t), ∂F (x(t), u(t))/∂xj (t) are continuous with respect to all n + m variables for i, j = 1, ..., n Further, u(t) is said to be
admissible if it is a piecewise continuous function on [0, ∞) with u(t) ∈ Ω.
Define the current-value Hamiltonian H(x(t), u(t), λ(t), λ0 ) as follows:
H(x(t), u(t), λ(t), λ0 ) ≡ λ0 F (x(t), u(t)) + λ(t) f (x(t), u(t))
(1.3)
with λ0 ∈ IR a constant scalar and λ(t) ∈ IRn the vector of co-state variables or shadow
prices. λj (t) gives the change in the optimal objective functional W o resulting from an
increment in the state variable xj (t). If xj (t) is a capital stock λj (t) gives the marginal
value of capital at time t. Assume that there exists a solution for (1.1) subject to (1.2).
Then, we have the following theorem.
1
2
Theorem 1 Let uo (t) be an admissible control and xo (t) is the trajectory belonging to
uo (t). For uo (t) to be optimal it is necessary that there exists a continuous vector function
λ(t) = (λ1 (t), ..., λn (t)) with piecewise continuous derivatives and a constant scalar λ0
such that:
(a) λ(t) and xo (t) are solutions of the canonical system
∂
H(xo (t), uo (t), λ(t), λ0 )
∂λ
∂
λ̇(t) = ρλ(t) −
H(xo (t), uo (t), λ(t), λ0 ),
∂x
ẋo (t) =
(b) For all t ∈ [0, ∞) where uo (t) is continuous, the following inequality must hold:
H(xo (t), uo (t), λ(t), λ0 ) ≥ H(xo (t), u(t), λ(t), λ0 ),
(c) (λ0 , λ(t)) 6= (0, 0) and λ0 = 1 or λ0 = 0.
Remarks:
1. If the maximum with respect to u(t) is in the interior of Ω, ∂H(·)/∂u(t) = 0 can
be used as a necessary condition for a local maximum of H(·).
2. ItR is implicitly assumed that the objective functional (1.1) takes on a finite value,
∞
that is 0 e−ρt F (xo (t), uo (t)) < ∞. If xo and uo grow without an upper bound1 F (·) must
not grow faster than ρ.
3. Working with the present-value Hamiltonian that contains the discount factor
−ρt
e
gives necessary conditions that are equivalent to those of theorem 1 after suitable
transformation. Working with the current-value Hamiltonian instead of the present-value
Hamiltonian implies that the differential equations are autonomous and do not explicitly
depend on time.
Theorem 1 provides us only with necessary conditions. The next theorem gives sufficient conditions.
Theorem 2 If the Hamiltonian with λ0 = 1 is concave in (x(t), u(t)) jointly and if the
transversality condition limt→∞ e−ρt λ(t)(x(t) − xo (t)) ≥ 0 holds, conditions (a) and (b)
from Theorem 1 are also sufficient for an optimum. If the Hamiltonian is strictly concave
in (x(t), u(t)) the solution is unique.
Remarks:
1. If the state and co-state variables are positive the transversality condition can be
written as stated in the above chapters, that is as limt→∞ e−ρt λ(t)xo (t) = 0.
2. Given some technical conditions it can be shown that the transversality condition
is also a necessary condition.
Theorem 2 requires joint concavity of the current-value Hamiltonian in the control
and state variables. A less restrictive theorem is the following.
1
Note that in the book we did not indicate optimal values by o .
3
Theorem 3 If the maximized Hamiltonian
Ho (x(t), λ(t), λ0 ) = max H(x(t), u(t), λ(t), λ0 )
u(t)∈Ω
with λ0 = 1 is concave in x(t) and if the transversality condition limt→∞ e−ρt λ(t)(x(t) −
xo (t)) ≥ 0 holds, conditions (a) and (b) from Theorem 1 are also sufficient for an optimum. If the maximized Hamiltonian Ho (x(t), λ(t), λ0 ) is strictly concave in x(t) for all t,
xo (t) is unique (but not necessarily uo (t)).
Since the joint concavity of H(x(t), u(t), λ(t), λ0 ) with respect to (x(t), u(t)) implies concavity of Ho (x(t), λ(t), λ0 ) with respect to x(t), but the reverse does not necessarily hold,
theorem 3 may be applicable where theorem 2 cannot be applied.
The above three theorems demonstrate how optimal control theory can be applied
to solve dynamic optimization problems. The main role is played by the Hamiltonian
function (1.3). It should be noted that in many economic applications, as in this book,
interior solutions are optimal so that ∂H(·)/∂u(t) = 0 can be presumed. For further
reading and more details concerning optimal control theory we refer to the books by
Feichtinger and Hartl (1986), Seierstad and Sydsæter (1987) or Beavis and Dobbs (1990).
References
Beavis B. and I.M. Dobbs (1990), Optimization and Stability Theory for Economic
Analysis. Cambridge University Press, Cambridge.
Feichtinger, G. and R.F. Hartl (1986), Optimale Kontrolle Ökonomischer Prozesse:
Anwendungen des Maximumprinzips in den Wirtschaftswissenschaften. De Gruyter,
Berlin.
Greiner, A. and B. Fincke (2015) Public Debt, Sustainability and Economic Growth:
Theory and Empirics. Springer Verlag, Heidelberg, New York.
Seierstad, A. and K. Sydsæter (1987) Optimal Control with Economic Applications.
North-Holland, Amsterdam.