Download Lecture 3. Differentiation of functionals

NUMERICAL METHODS
FOR MATHEMATICAL PHYSICS INVERSE PROBLEMS
Lecture 3. Differentiation of functionals
We know that the inverse problems can be transformed to the problems of extremum finding. So the practical
methods of inverse problems theory are based on the optimization methods. We considered methods of analysis for
the easiest extremum problem. It was the problem of the minimization for the function of one variable. We analyze
it with using of the stationary conditions and gradient method. However the practical inverse problems can be
transformed to the optimization problem for the functions of many variables and the functionals even. So we would
like to extend the known minimization methods to these problems. Therefore our next aim is the determination of
the differentiation for the general functionals.
3.1. Gateaux derivative
Let us consider a general functional I on the arbitrary set V. We could extend in principle the
stationary condition and the gradient method for its minimization if we had some methods of
differentiation for this functional. Let us try to use the standard technique for calculate its
derivative at a point v of the set V. It is know that the derivative of a function at a point is the
result of passing to the limit to the increment of the function devised by the increment of the
argument as the increment of the argument tends to zero.
Try to consider the increment of the argument at the point v. Let it be described by a value h.
So the corresponding increment of functional is the difference between its value at the new point
v  h and its value at the initial point v. There is the question, what is the sum of two elements
of the given set V? We have the necessity to guarantee belonging of the sum of two arbitrary
elements of V to this set. So the operation of the addition is determined on the set V. The
difference I (v  h )  I (v ) has the sense in this case because the difference between two numbers
(values of the functional) is a number.
Our next step is the calculation of the ratio of the increments  I ( v  h )  I ( v )  / h. We have the
functional I, so its numerator is a number. But what is the sense of the division of the number
I (v  h )  I (v ) to the element h of the set V. It is clear, if h is a number. However it can be a
vector or a function. Unfortunately the considered fraction does not have any sense in this case.
So we need to correct the definition of the derivative.
We can determine the derivative of the function f  f ( x ) at the point x by the following
method. We consider the ratio  f ( x  h )  f ( x )  /  , where  and h are number. Then we try to
pass to the limit here as  tends to zero. If there exists a limit of this value, and this limit is linear
with respect to h, then it has the form f ( x ) h for all value h, where f ( x ) is the usual derivative
of the function f at the point x.
We can try to use this idea for our case. Consider the ratio  I ( v  h )  I ( v )  /  , where  is a
number, and h is an element of the set V. We have the ratio of two numbers here. So this term
has the natural sense, and we will not any problem with division. However we need to interpret
the product  h . Our technique can be true if we can determine the product between an arbitrary
number  and an arbitrary element h of the set V. So the operation of the multiplication of the
element of V to the number is determined on the set V.
Definition 3.1. The set with addition of the elements and the multiplication of the element to
the number (with natural additional properties) is called the linear space.
Hence we will consider the set V as a linear space. The term  I ( v  h )  I ( v )  /  has the
concrete sense for this case. Our next step is passing to the limit at the last term. However the
convergence does not determined on the arbitrary set. We can pass to the limit on the topological
space only. So we suppose that our set V is a topological space. But we have the necessity of a
relation between the given algebraic operations and passing to the limit. Particularly we would
like to have the convergence  h  0 for all h as   0. Then we would like to obtain the
convergence v   h  v. We can guarantee these properties if our operations are continuous.
Let us have the convergence uk  u. The operation of the multiplication by number is
continuous if we obtain  uk   u. Let us have also the convergence vk  v. The operation of
the addition is continuous if we get (uk  vk )  (u  v).
Definition 3.2. If the set is linear and topological space with continuous operations, then it is
called the linear topological space.
Hence we will consider the set V as a linear topological space. So the passing to the limit for
the term  I ( v  h )  I ( v )  /  as   0 has the sense. Suppose the existence of the
corresponding limit. It depends from parameters v and h of course. We suppose also that its
dependence from h is linear. So we have the equality
lim
I (v   h )  I (v )

 0
 I ( v ) h ,
(3.1)
where the map h  I (v ) h is linear. The term in the left side of this equality is a number,
because it is the limit of the ration of the two numbers. So the term in the right side of (3.1) is
number too. Therefore the map I ( v ) is a linear functional.
Definition 3.3. A functional I is called Gateaux differentiable at the point v if there exists a
linear functional I ( v ) such that the equality (3.1) holds. Besides I ( v ) is called Gateaux
derivative of the functional I at the point v.
3.2. Examples of Gateaux derivatives
Consider examples of Gateaux derivatives.
Example 3.1. The function of one variable. Consider a function f  f ( x ). Suppose it is
differentiable at the point x. So we have the equality
lim
f ( x   h)  f ( x )

 0
 f ( x)h
(3.2)
for all number h. So Gateaux derivative of the function f at the point x is its classical derivative at
this point.

Example 3.2. The function of many variables. Consider a function f  f ( x1 ,..., xn ).
Determine its Gateaux derivative at the point x  ( x1 ,..., xn ). We would like to pass to the limit
in the equality
f ( x   h)  f ( x )


f ( x1   h1 ,..., xn   hn )  f ( x1 ,..., xn )

as   0 for all vector h  (h1 ,..., hn ). Let the function f be differentiable with respect to all its
arguments. Then we have
lim
 0
f ( x   h)  f ( x)

n

i 1
f ( x)
hi .
xi
The term at the right side of this equality is the scalar product between the gradient
(3.3)
 f ( x)
f ( x) 
f ( x)  
,...,

xn 
 x1
and the vector h. So Gateaux derivative of the function f of many variable at the point x is its
gradient at this point.

Example 3.3. Linear functional. The functional I on the set V is linear if it satisfies the
following equality
I ( u   v )   I (u )   I (v )
for all elements u and v of V and all number ,. Find the derivative of the linear functional I at
the arbitrary point v. We have
I (v   h )  I (v )
I (v )   I ( h )  I (v )
lim
 lim
 I ( h)

 0

 0
for all h. So the linear functional is Gateaux differentiable at the arbitrary point; besides its
derivative is equal to the initial functional.

Example 3.4. Affine functional. The functional I on the set V is called affine if it is determine
by the equality
I (v )  J (v )  a v  V ,
where J is a linear on V, and a is an element of V. We find
 J ( v  h )  a    J (v )  a 
I (v   h )  I (v )
lim
 lim
 J ( h).

 0

 0
So the affine functional is Gateaux differentiable at the arbitrary point; besides its derivative is
equal to the corresponding linear functional.

Example 3.5. Lagrange functional. Consider the functional
x2
I (v)   F [ x, v( x),v( x)]dx
x1
on the set V of the smooth enough functions on the interval [ x1 , x2 ] with zero values on the
boundary of this interval. This functional is called Lagrange one. Let the function F be smooth
enough. We find
I (v   h) 
x2

x2
x2
x1
x1
 F  x,v( x) h( x),v( x) h( x) dx   F  x,v( x),v( x) dx 

F x, v ( x ), v( x )  h ( x ) dx 
v


x1
x2


F x, v ( x ), v( x )  h( x ) dx   ( ),
v


x1
where  ( ) /   0 as   0 . Then we calculate
lim
 0
I (v   h)  I (v )

x2


x1

F x, v ( x ), v( x ) h( x ) dx 
v


x2

x1
for all function h from the set V. Differentiating by parts, we get

F x, v ( x ), v( x ) h( x ) dx
v


x2


F x, v ( x ), v( x ) h( x ) dx
v


x1
x2
 
d 
F x, v ( x ), v( x ) h( x ) dx 
dx v


x1
x
2

  F x, v ( x ), v( x ) h( x )
v
x1


x2
 
d 
F x, v ( x ), v( x ) h( x ) dx
dx v


x1
because the function h is equal to zero at the point x1 and x2. Therefore the derivative of
Lagrange functional at the point (function) v is determined by the equality
  v F  x, v( x), v( x)  dx v F  x, v( x), v( x) h( x)dx h  V .
x2
I ( v ) h 

d 
(3.4) 
x1
Example 3.6. Dirichlet integral. Let  be n-dimensional set with the boundary S. Consider an
integral
 n  v 2

 d ,
I (v)    

2
vf


x


i

1
 i

where f is a given function. This integral is called Dirichlet one. We consider it on the set V of
the smooth enough functions on  with zero value on the boundary S. Find the value
 n   (v   h) 2

 n  v  2

I (v   h)     
  2(v   h) f  d         2vf  d  

x
x
i

1


i



 
 i 1  i 
2
n
 n v h

 h 
2
2   
 fh  d      
 d .

x

x

x
i
i
i 

  i 1
 i 1 
Then we have
lim
I (v   h)  I (v )

 0
 n v
 2  
xi
  i 1
h
xi

 fh  d .

Using Green formula we get
v h
n
  x
 i 1
where  is Laplace operator, and
i
xi

d    vhd  


v
hdS ,
n
S
v
is the derivative of v with respect to the exterior normal to
n
the boundary S. So we can determine Gateaux derivative of Dirichlet integral is determine by the
equality

I (v ) h   vhd  h  V
(3.5)

because the function h is equal to zero on the set S.

Example 3.7. Discontinuous function. Consider the function of two variables, which is
determine by the equality
 y( x 2  y 2 ) / x, x  0,
f ( x, y)  
x  0.
0,
Find the difference
f ( h,  g )  f (0,0)   2 (h2  g 2 ) g / h
for all numbers h and g. After division by  and passing to the limit as   0 we obtain that
Gateaux derivative of the function f at zero point is zero, namely second order zero vector.
However we continue our analysis. Determine x  t 3 , y  t , where t is a parameter. Then we have
f ( x, y)  (t 4  1). Passing to the limit as t  0 we have ( x, y )  0 and f ( x, y )  1. But the
value of the function f at zero is zero. So this function is discontinuous at zero. Hence the
discontinuous function of two variables can be Gateaux differentiable.

Example 3.8. Absolute value. Consider the function f ( x)  x . We have
f ( h)  f (0)  h

.


Then we obtain
lim  h /   h , lim  h /    h .
 0
 0
The result depends from the method of passing to the limit, and the result is not linear with
respect to h. So the absolute value function is not Gateaux differentiable.

3.3. Gateaux derivatives for functional on unitary spaces
We determined that Gateaux derivative is definite by scalar product for the functions of many
variables. It is possible to extend this result for the general linear spaces with scalar product.
Definition 3.4. The value (u , v ) is the scalar product of the elements u and v of the linear
space V, if the maps u  (u , v ) and v  (u , v ) are linear functionals and (u , v )  (v, u ) for all
u , v  V ; besides (u , u )  0, and (u , u )  0, if and only if u is zero element of the space V, that is
u  v  v for all v. The linear space with a scalar product is called the unitary space.
n
The Euclid space R is the space with standard scalar product.
We can determine the convergence for the space with scalar product with following notions.
Definition 3.5. Let V be the unitary space. The value
v  ( v ,v )
is called the norm of the element v of V.
Definition 3.6. The sequence {vk } of the linear space V with scalar product converges to
the point v, if
lim vk  v  0.
k 
The linear space with this convergence is the linear topological space.
Definition 3.7. A functional I on the unitary space V is called Gateaux differentiable at the
point v if there exists an element I ( v ) of V such that
lim
 0
I (v   h)  I (v )

  I ( v ), h  h  V
holds. Besides I ( v ) is called Gateaux derivative of the functional I at the point v.
(3.6)
The set of real number R is the unitary space with scalar product
(u , v )  uv u , v  R.
Then the formula (3.2) for Gateaux derivative of a function of one variable can be transformed to
the equality (3.6).
Euclid space Rn is the unitary space with scalar product
n
n
(u , v )   ui vi u , v  R .
i 1
So the formula (3.2) for Gateaux derivative of a function of n variable can be transformed to the
equality (3.6).
The set C[ x1 , x2 ] of the continuous functions on the interval [ x1 , x2 ] with zero values at the
points x1 and x2 is the unitary spaces with scalar product
x2
(u, v)   u ( x)v( x)dx u, v  C[ x1 , x2 ].
x1
The analogical scalar product can be determine for the space of the smooth enough functions
with zero values at the boundary points of the given interval. So the equality (3.4) for the
derivative of Lagrange functional can be transformed to the equality
x2
 I (v ),h   


d 
F x, v ( x ), v( x ) 
F x, v ( x ), v( x )
v
dx v



 h( x)dx h  C[ x1 , x2 ].
x1
Then we find Gateaux derivative
I (v) 

d 
F x, v( x), v( x) 
F x, v ( x ), v( x ) .
v
dx v




The set C    of the continuous functions on the on the set  with zero values on the
boundary of this set is the unitary spaces with scalar product
(u, v)   u( x)v( x)d  u, v  C ().

The analogical scalar product can be determine for the space of the smooth enough functions
with zero values at the boundary points of the given set. So the equality (3.5) for the derivative
of Dirichlet integral can be transformed to the equality
 I (v ),h     vhd  h  C ().

Then we find Gateaux derivative
I ( v )  v.
Example 3.9. Square of the norm. Consider the functional
2
I (v)  v
on the unitary space V. Find the value
I (v   h)   v   h, v   h    v, v   2  v, h    2  h, h  .
Then we have
I (v   h)  I (v)  2  v, h    2 h .
2
After division by  and passing to the limit as   0 we get
 I '(v), h   2v, h  h V .
So I '(v)  2v.

Task
Find Gateaux derivatives for the given functions of two variables and integral functionals:
variant
function
f  f (v1 , v2 )
1
v14  v24
2
v v
functional
I  I (v )
1
 v (t )dt
3
0
6
1
2
2
 v( x)
2

 2v( x) dx

3
v14  2v22
1
v
4
(t ) dt
0
4
v13  v23
 v ( x)dx
3

Next step
We know that inverse problems can be transformed to the minimization problems. The easiest
minimization problem is the problem of the minimization for the function of one variable. It can
be solve with using of the stationary condition and gradient method. This technique is based on
the differentiation of the given function. Now we are able to differentiate general functionals.
Then we try to apply the known optimization methods to problems of minimization general
functionals.