Download Convex function

Extrema
Convexity and Optimality
max. f (x) s.t. x ∈ S
max{f (x) : x ∈ S}
Global maximum x∗ :
f (x∗ ) ≥ f (x) ∀ x ∈ S
Extrema
Convex function
Convex set
Other convexity concepts
Unconstrained optimization
Local maximum xo :
f (xo ) ≥ f (x) ∀ x in a neighborhood around xo
Strict maximum
Analogous: minimum
Weierstraß theorem:
A continuous function achieves its max and min
on a closed and bounded set
Henrik Juel, DTU Management
Convexity and Optimality – p.1/9
Supremum and infimum
Convexity and Optimality – p.2/9
Convex function
Supremum = least upper bound
Infimum = greatest lower bound
If the extrema are not achieved:
max → sup
min → inf
ex. inf{1/x : x > 0} = 0
A convex function lies below its chord
f (α1 x1 + α2 x2 ) ≤ α1 f (x1 ) + α2 f (x2 )
The convex combination of two points is
the line segment between them
α1 x1 + α2 x2 for α1 , α2 ≥ 0 and α1 + α2 = 1
Convexity and Optimality – p.3/9
Strictly convex function
The sum of convex functions is also convex
ex. x2 , x, x2 + y 2
A differentiable convex function lies above its tangent
A differentiable function is convex iff
its Hessian is positive semidefinite
Strictly convex not analogous!
Concave function:
f is concave iff −f is convex
Convexity and Optimality – p.4/9
Convex function examples
Convex sets
EOQ objective function:
T (Q) = dK/Q + cd + hQ/2
is strictly convex
Parametric programming of objective function:
Class exercise:
pP
2
Show that f (x) = ||x|| =
i xi is convex
A convex set contains all its convex combinations
α1 x1 + α2 x2 ∈ S ∀ x1 , x2 ∈ S
ex. (1, 2], x2 + y 2 < 4, ∅
max. Z = (c + θd)x, θ ≥ 0
s.t. Ax ≤ b, x ≥ 0 (or x ∈ S)
Level curve (2 dimensions):
(x, y) : f (x, y) = β
Z(θ) is convex and piecewise linear
Level set: x : f (x) ≤ β
Any level set of a convex function is a convex set
Proof?
Convexity and Optimality – p.5/9
Other types of convexity
Convexity and Optimality – p.6/9
Unconstrained problem
A differentiable function is pseudoconvex, whenever
f ′ (x0 )(x − x0 ) ≥ 0 implies f (x) ≥ f (x0 )
(f ′ (y) = ∇f (y)t , a row vector of partial derivatives)
A function is quasiconvex,
when all its level sets are convex
convex ⇒ pseudoconvex ⇒ quasiconvex
Questions:
Can a function be both convex and concave?
Is a convex function of a convex function convex?
Is a convex combination of convex functions convex?
Is the intersection of convex sets a convex set?
Convexity and Optimality – p.7/9
min. f (x) (s.t. x ∈ Rn )
Necessary optimality condition:
If xo is a local minimum,
then f ′ (xo ) = 0 and H(xo ) is positive semidefinite
Sufficient optimality condition:
If f ′ (xo ) = 0 and H(xo ) is positive definite, then xo is
a local minimum
Necessary and sufficient:
Suppose f is pseudoconvex.
x∗ is a global minimum iff f ′ (x∗ ) = 0
Convexity and Optimality – p.8/9
Unconstrained example
min. f (x) = (x2 − 1)3
f ′ (x) = 6x(x2 − 1)2 = 0 for x = 0, ±1
H(x) = 24x(x2 − 1) + 6(x2 − 1)2 ,
so H(0) = 6 and H(±1) = 0
x = 0 is a local minimum.
A sketch shows that it is the unique global minimum,
and that x = ±1 are saddle points.
Class exercise:
A is a given m ∗ n matrix, b is a given m vector
Solve min. ||Ax − b||2
Convexity and Optimality – p.9/9