Canada a drabek p fonda a (eds ) handbook of differential equations ordinary differential equations vol 2

H ANDBOOK
OF D IFFERENTIAL E QUATIONS
O RDINARY D IFFERENTIAL E QUATIONS
VOLUME II


This page intentionally left blank


H ANDBOOK
OF D IFFERENTIAL E QUATIONS
O RDINARY D IFFERENTIAL
E QUATIONS
VOLUME II
Edited by

A. CAÑADA
Department of Mathematical Analysis, Faculty of Sciences,
University of Granada, Granada, Spain

P. DRÁBEK
Department of Mathematics, Faculty of Applied Sciences,
University of West Bohemia, Pilsen, Czech Republic

A. FONDA
Department of Mathematical Sciences, Faculty of Sciences,
University of Trieste, Trieste, Italy

2005

NORTH
HOLLAND

Amsterdam • Boston • Heidelberg • London • New York • Oxford •
Paris • San Diego • San Francisco • Singapore • Sydney • Tokyo


ELSEVIER B.V.
Radarweg 29
P.O. Box 211, 1000 AE Amsterdam
The Netherlands

ELSEVIER Inc.
525 B Street, Suite 1900
San Diego, CA 92101-4495
USA

ELSEVIER Ltd
The Boulevard, Langford Lane
Kidlington, Oxford OX5 1GB
UK

ELSEVIER Ltd
84 Theobalds Road
London WC1X 8RR
UK

© 2005 Elsevier B.V. All rights reserved.
This work is protected under copyright by Elsevier B.V., and the following terms and conditions apply to its use:
Photocopying

Single photocopies of single chapters may be made for personal use as allowed by national copyright laws. Permission of the Publisher and payment of a fee is required for all other photocopying, including multiple or systematic copying, copying for advertising
or promotional purposes, resale, and all forms of document delivery. Special rates are available for educational institutions that wish
to make photocopies for non-profit educational classroom use.
Permissions may be sought directly from Elsevier’s Rights Department in Oxford, UK: phone (+44) 1865 843830, fax
(+44) 1865 853333, e-mail: Requests may also be completed on-line via the Elsevier homepage
( />In the USA, users may clear permissions and make payments through the Copyright Clearance Center, Inc., 222 Rosewood Drive,
Danvers, MA 01923, USA; phone: (+1) (978) 7508400, fax: (+1) (978) 7504744, and in the UK through the Copyright Licensing
Agency Rapid Clearance Service (CLARCS), 90 Tottenham Court Road, London W1P 0LP, UK; phone: (+44) 20 7631 5555, fax:
(+44) 20 7631 5500. Other countries may have a local reprographic rights agency for payments.
Derivative Works
Tables of contents may be reproduced for internal circulation, but permission of the Publisher is required for external resale or
distribution of such material. Permission of the Publisher is required for all other derivative works, including compilations and
translations.
Electronic Storage or Usage
Permission of the Publisher is required to store or use electronically any material contained in this work, including any chapter or
part of a chapter.
Except as outlined above, no part of this work may be reproduced, stored in a retrieval system or transmitted in any form or by any
means, electronic, mechanical, photocopying, recording or otherwise, without prior written permission of the Publisher.
Address permissions requests to: Elsevier’s Rights Department, at the fax and e-mail addresses noted above.
Notice
No responsibility is assumed by the Publisher for any injury and/or damage to persons or property as a matter of products liability,
negligence or otherwise, or from any use or operation of any methods, products, instructions or ideas contained in the material
herein. Because of rapid advances in the medical sciences, in particular, independent verification of diagnoses and drug dosages
should be made.

First edition 2005
Library of Congress Cataloging in Publication Data: A catalog record is available from the Library of Congress.
British Library Cataloguing in Publication Data: A catalogue record is available from the British Library.

ISBN 0 444 52027 9

Set ISBN: 0 444 51742 1
∞ The paper used in this publication meets the requirements of ANSI/NISO Z39.48-1992 (Permanence of Paper).

Printed in The Netherlands.


Preface
This handbook is the second volume in a series devoted to self contained and up-to-date
surveys in the theory of ordinary differential equations, written by leading researchers in
the area. All contributors have made an additional effort to achieve readability for mathematicians and scientists from other related fields, in order to make the chapters of the
volume accessible to a wide audience. These ideas faithfully reflect the spirit of this multivolume and the editors hope that it will become very useful for research, learning and
teaching. We express our deepest gratitude to all contributors to this volume for their clearly
written and elegant articles.
This volume consists of six chapters covering a variety of problems in ordinary differential equations. Both, pure mathematical research and real word applications are reflected
pretty well by the contributions to this volume. They are presented in alphabetical order
according to the name of the first author. The paper by Barbu and Lefter is dedicated to
the discussion of the first order necessary and sufficient conditions of optimality in control
problems governed by ordinary differential systems. The authors provide a complete analysis of the Pontriaghin maximum principle and dynamic programming equation. The paper
by Bartsch and Szulkin is a survey on the most recent advances in the search of periodic
and homoclinic solutions for Hamiltonian systems by the use of variational methods. After
developing some basic principles of critical point theory, the authors consider a variety of
situations where periodic solutions appear, and they show how to detect homoclinic solutions, including the so-called “multibump” solutions, as well. The contribution of Cârj˘a
and Vrabie deals with differential equations on closed sets. After some preliminaries on
Brezis–Browder ordering principle and Clarke’s tangent cone, the authors concentrate on
problems of viability and problems of invariance. Moreover, the case of Carathéodory solutions and differential inclusions are considered. The paper by Hirsch and Smith is dedicated
to the theory of monotone dynamical systems which occur in many biological, chemical,
physical and economic models. The authors give a unified presentation and a broad range
of the applicability of this theory like differential equations with delay, second order quasilinear parabolic problems, etc. The paper by López-Gómez analyzes the dynamics of the
positive solutions of a general class of planar periodic systems, including those of Lotka–
Volterra type and a more general class of models simulating symbiotic interactions within

global competitive environments. The mathematical analysis is focused on the study of
coexistence states and the problem of ascertaining the structure, multiplicity and stability
of these coexistence states in purely symbiotic and competitive environments. Finally, the
paper by Ntouyas is a survey on nonlocal initial and boundary value problems. Here, some
old and new results are established and the author shows how the nonlocal initial or boundv


vi

Preface

ary conditions generalize the classical ones, having many applications in physics and other
areas of applied mathematics.
We thank again the Editors at Elsevier for efficient collaboration.


List of Contributors
Barbu, V., “Al.I. Cuza” University, Ia¸si, Romania, and “Octav Mayer” Institute of Mathematics, Romanian Academy, Ia¸si, Romania (Ch. 1)
Bartsch, T., Universität Giessen, Giessen, Germany (Ch. 2)
Cârj˘a, O., “Al. I. Cuza” University, Ia¸si, Romania (Ch. 3)
Hirsch, M.W., University of California, Berkeley, CA (Ch. 4)
Lefter, C., “Al.I. Cuza” University, Ia¸si, Romania, and “Octav Mayer” Institute of Mathematics, Romanian Academy, Ia¸si, Romania (Ch. 1)
López-Gómez, J., Universidad Complutense de Madrid, Madrid, Spain (Ch. 5)
Ntouyas, S.K., University of Ioannina, Ioannina, Greece (Ch. 6)
Smith, H., Arizona State University, Tempe, AZ (Ch. 4)
Szulkin, A., Stockholm University, Stockholm, Sweden (Ch. 2)
Vrabie, I.I., “Al. I. Cuza” University, Ia¸si, Romania, and “Octav Mayer” Institute of Mathematics, Romanian Academy, Ia¸si, Romania (Ch. 3)

vii

This page intentionally left blank


Contents
Preface
List of Contributors
Contents of Volume 1

v
vii
xi

1. Optimal control of ordinary differential equations
V. Barbu and C. Lefter
2. Hamiltonian systems: periodic and homoclinic solutions by variational methods
T. Bartsch and A. Szulkin
3. Differential equations on closed sets
O. Cârj˘a and I.I. Vrabie
4. Monotone dynamical systems
M.W. Hirsch and H. Smith
5. Planar periodic systems of population dynamics
J. López-Gómez
6. Nonlocal initial and boundary value problems: a survey
S.K. Ntouyas
Author index
Subject index

1
77

147
239
359
461

559
565

ix


This page intentionally left blank


Contents of Volume 1
Preface
List of Contributors

v
vii

1. A survey of recent results for initial and boundary value problems singular in the
dependent variable
R.P. Agarwal and D. O’Regan
2. The lower and upper solutions method for boundary value problems
C. De Coster and P. Habets
3. Half-linear differential equations
O. Došlý
4. Radial solutions of quasilinear elliptic differential equations
J. Jacobsen and K. Schmitt

5. Integrability of polynomial differential systems
J. Llibre
6. Global results for the forced pendulum equation
J. Mawhin
7. Wa˙zewski method and Conley index
R. Srzednicki
Author Index
Subject Index

1
69
161
359
437
533
591

685
693

xi


This page intentionally left blank


CHAPTER 1

Optimal Control of Ordinary Differential Equations
Viorel Barbu and C˘at˘alin Lefter

University “Al.I. Cuza”, Ia¸si, Romania, and
Institute of Mathematics “Octav Mayer”, Romanian Academy, Romania

Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.1. The calculus of variations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2. General form of optimal control problems . . . . . . . . . . . . . . . . . . . . . .
2. Preliminaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.1. Elements of convex analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.2. Ekeland’s variational principle . . . . . . . . . . . . . . . . . . . . . . . . . . . .
2.3. Elements of differential geometry and exponential representation of flows . . . .
3. The Pontriaghin maximum principle . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.1. The main theorem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.2. Proof of the maximum principle . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.3. Convex optimal control problems . . . . . . . . . . . . . . . . . . . . . . . . . .
3.4. Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
3.5. Reachable sets and optimal control problems . . . . . . . . . . . . . . . . . . . .
3.6. Geometric form of Pontriaghin maximum principle . . . . . . . . . . . . . . . .
3.7. Free time optimal control problems . . . . . . . . . . . . . . . . . . . . . . . . .
4. The dynamic programming equation . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.1. Optimal feedback controllers and smooth solutions to Hamilton–Jacobi equation
4.2. Linear quadratic control problems . . . . . . . . . . . . . . . . . . . . . . . . . .
4.3. Viscosity solutions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
4.4. On the relation between the two approaches in optimal control theory . . . . . .
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

HANDBOOK OF DIFFERENTIAL EQUATIONS
Ordinary Differential Equations, volume 2
Edited by A. Cañada, P. Drábek and A. Fonda
© 2005 Elsevier B.V. All rights reserved

1

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.

3
4
9
10
10
15
16
29
29
32
38
47
55
57
59
62
62
65

68
72
74



Optimal control of ordinary differential equations

3

1. Introduction
The theory of control of differential equations has developed in several directions in close
relation with the practical applications of the theory. Its evolution has shown that its methods and tools are drawn from a large spectrum of mathematical branches such as ordinary
differential equations, real analysis, calculus of variations, mechanics, geometry. Without
being exhaustive we just mention, as subbranches of the control theory, the controllability,
the stabilizability, the observability, the optimization of differential systems and of stochastic equations or optimal control. For an introduction to these fields, and not only, see
[22,33,38], as well as [2,25] and [26] for a geometric point of view.
The purpose of this work is to discuss the first order necessary and sufficient conditions
of optimality in control problems governed by ordinary differential systems. We do not
treat the optimal control of partial differential equations although all basic questions of
the finite dimensional theory (existence of optimal control, maximum principle, dynamic
programming) remain valid but the treatment requires more sophisticated methods because
of the infinite dimensional nature of the problems (see [4,27,38]).
In Section 1 we present some aspects and ideas in the classical Calculus of variations that
lead later, in the fifties, to the modern theory of optimal control for differential equations.
Section 2 presents some preliminary material. It contains elements of convex analysis
and the generalized differential calculus for locally Lipschitz functionals, introduced by
F.H. Clarke [10]. This will be needed for the proof of the maximum principle of Pontriaghin, under general hypotheses, in Section 3.1. We then discuss the exponential representation of flows, introduced by A. Agrachev and R. Gamkrelidze in order to give a
geometric formulation to the maximum principle that we will describe in Sections 3.5, 3.6.
Section 3 is concerned with the Pontriaghin maximum principle for general Bolza problems. There are several proofs of this famous classical result and here, following F.H.

Clarke’s ideas (see [11]), we have adapted the simplest one relying on Ekeland’s variational principle. Though the maximum principle given here is not in its most general form,
it is however sufficiently general to cover most of significant applications. Some examples
are treated in detail in Section 3.4. Since geometric control theory became in last years an
important branch of mathematics (for an introduction to the theory see [2,26]), it is useful
and interesting to give a geometric formulation of optimal control problems and, consequently, a geometric form of the maximum principle. Free time optimal problems are also
considered as a special case.
In the last section we present the dynamic programming method in optimal control problems based on the partial differential equation of dynamic programming, or Bellman equation (see [7]). The central result of this chapter says that the value function is a viscosity
solution to Bellman equation and that, if a classical solution exists, then an optimal control, in feedback form, is obtained. Applications to linear quadratic problems are given.
We discuss also the relationship between the maximum principle and the Bellman equation and we will see in fact that the dynamic programming equation is the Hamilton–Jacobi
equation for the Hamiltonian system given by the maximum principle.


4

V. Barbu and C. Lefter

1.1. The calculus of variations
In this section we point out the fundamental lines of development in the Calculus of variations. We will not impose rigorous assumptions on the functions entering the described
problems, they will be as regular as needed. The main purpose is just to emphasize some
fundamental ideas that will be reencountered, in a metamorphosed form, in the theory of
optimal control for differential equations. For a rigorous presentation of the theory a large
literature may be cited, however we restrict for instance to [8,24] and to a very nice survey
of extremal problems in mathematics, including the problems of Calculus of variations,
in [34].
Let M be an n-dimensional manifold, and M0 , M1 be subsets (usually submanifolds)
of M. L : R × T M → R is the Lagrangean function, T M being the tangent bundle of M.
The generic problem of the classical Calculus of variations consists in finding a curve, y ∗ ,
which minimizes a certain integral
t1

J (y) =

L t, y(t), y ′ (t) dt

(1.1)

t0

in the space of curves
Y = y : [t0 , t1 ] → M; y(tj ) ∈ Mj , j = 1, 2, y continuous and piecewise C 1 .
The motivation for studying such problems comes from both geometry and classical mechanics.
E XAMPLES . 1. The brachistocrone. The classical brachistocrone problem proposed by
Johann Bernoulli in 1682, asks to find the curve, in a vertical plane, on which a material
point, moving without friction under the action of its weight, is reaching the lower end
of the curve in minimum time. More precisely, if the curve is joining two points y(t0 ) =
y0 , y(t1 ) = y1 , then the time necessary for the material point to reach y1 from y0 is
t1

T=

t0

2g y(t) − y0

−1/2

2

1 + y ′ (x) dt.

The curve with this property is a cycloid.
2. The minimal surface of revolution. One is searching for the curve y : [t0 , t1 ] → R,
y(t0 ) = y0 , y(t1 ) = y1 , which generates the surface of revolution of least area. The functional to be minimized is
t1

J (y) = 2π

t0

2

y(x) 1 + y ′ (t) dt.

The solution is the catenary.
3. Lagrangean mechanics. A mechanical system with a finite number of degrees
of freedom is mathematically modelled by a manifold M and a Lagrangean function


Optimal control of ordinary differential equations

5

L : R × T M → R (see [3]). The manifold M is the configuration space of the mechanical system. The points y ∈ M are generalized coordinates and the y ′ ∈ T M are generalized
speeds. The principle of least action of Maupertuis–d’Alembert–Lagrange states that the
trajectories of the mechanical system are extremal for the functional J defined in (1.1).
Consider the case of a system of N material points in the 3 dimensional space, moving
under the action of mutual attraction forces. In this case the configuration space is (R3 )N ,
while the Lagrangean is
L=T −U

(1.2)

where T is the kinetic energy
N

T=

i=1

1
mi |xi′ |2
2

and U is the potential energy
N

U (x1 , . . . , xN ) =

i=1

kmi mj
,
|xi − xj |

k is an universal constant.
To make things clear we consider the simplest problem in the Calculus of variations
when M = Rn and y0 , y1 are fixed.
We consider the space of variations Y = {h : [t0 , t1 ] → Rn ; h(t0 ) = h(t1 ) = 0, h ∈ C 1 };
if y ∗ is a minimum of J in Y , then the first variation
δJ (y ∗ )h :=

d
J (y ∗ + sh)
ds

s=0

= 0.

(1.3)

A curve that satisfies (1.3) is called extremal and this is only a necessary condition for a
curve to realize the infimum of J . One easily computes
t1

δJ (y)h =

t0

Ly t, y ∗ (t), (y ∗ )′ (t) · h + Ly ′ t, y ∗ (t), (y ∗ )′ (t) · h′ dt

where Ly , Ly ′ are the gradients of L with respect to y and y ′ , respectively. If y ∗ is C 2 , an
integration by parts in the previous formula gives
Ly t, y ∗ (t), (y ∗ )′ (t) −

d
Ly ′ t, y ∗ (t), (y ∗ )′ (t) = 0
dt

(1.4)

which are the Euler–Lagrange equations. It is a system of n differential equations of second
order.


6

V. Barbu and C. Lefter

It may be proved that if the Hessian matrix (Ly ′ y ′ ) > 0, then the regularity of L is inherited by the extremals, for instance if L ∈ C 2 then the extremals are C 2 and thus satisfy
the Euler–Lagrange system. The proof of this fact is based on the first of the Weierstrass–
Erdmann necessary conditions which state that, along each extremal, Ly ′ and the Hamiltonian defined below in (1.6) are continuous.
Another necessary condition for the extremal y ∗ to realize the infimum of J is that
(Ly ′ y ′ ) 0 along y ∗ . This is the Legendre necessary condition.
Suppose from now on that (Ly ′ y ′ ) is a nondegenerate matrix at any point (t, y, y ′ ). We
set
p = Ly ′ (t, y, y ′ ).

(1.5)

Since (Ly ′ y ′ ) is nondegenerate, formula (1.5) defines a change of coordinates (t, y, y ′ ) →
(t, y, p). From the geometric point of view it maps T M locally onto T ∗ M, the cotangent
bundle. In mechanics p is called the generalized momentum of the system and in most
applications its significance is of adjoint (or dual) variable. We consider the Hamiltonian
H (t, y, p) = (p, y ′ ) − L(t, y, y ′ ).

(1.6)

If, moreover, L is convex in y ′ then H = L∗ , the Legendre transform of L:
H (t, y, p) = sup (p, y ′ ) − L(t, y, y ′ ) .

y′

For example, if L is given by (1.2) then H = T + U and it is just the total energy of the
system. If we compute the differential of H along an extremal, taking into account the
Euler–Lagrange equations, we obtain
dH = −Lt dt −

d
Ly ′ dy + y ′ dp.
dt

Thus, through these transformations we obtain the Hamiltonian equations

∂H
′


 y = ∂p (t, y, p),
 ′
∂H

p = −
(t, y, p).
∂y

(1.7)

Solutions of the Hamiltonian system are in fact extremals corresponding to the Lagrangean
L(t, (y, p), (y ′ , p ′ )) = p · y ′ − H (t, y, p) in T ∗ M. The projections on M are extremals
for J . Roughly speaking, solving the Euler–Lagrange system is equivalent to solving the

Hamiltonian system of 2n differential equations of first order. From the mechanics point of
view these transforms give rise to the Hamiltonian mechanics which study the mechanical
phenomena in the phase space T ∗ M while in mathematics this is the start point for the
symplectic geometry (see for example [3,28]).


Optimal control of ordinary differential equations

7

Consider now the more general case of end points lying on two submanifolds M0 , M1 .
It may be shown that the first variation of J in y computed in an admissible variation h
(assume that also t0 , t1 are free) is
t1

δJ (y)h =

t0

Ly (t, y, y ′ ) · h + Ly ′ (t, y, y ′ ) · h′ dt + {pδy − H δt}

t1
t0

(1.8)

where
H δt

t1

t0

= H t1 , y(t1 ), p(t1 ) δt1 − H t0 , y(t0 ), p(t0 ) δt0 ,

pδy

t1
t0

= p(t1 )δy1 − p(t2 )δy0 .

It turns out that y ∗ ∈ C 2 is extremal for J if y ∗ satisfies the Euler–Lagrange equations (1.4)
and in addition
{pδy − H δt}

t1
t0

= 0.

(1.9)

These are transversality conditions. In case t0 , t1 are fixed, these become
p(t0 ) ⊥ M0 ,

p(t1 ) ⊥ M1 .

Since (Ly ′ y ′ ) is supposed to be nondegenerate, the Euler–Lagrange equations form a second order nondegenerate system of equations and this implies that the family of extremals
starting at moment t0 from a given point of y0 ∈ M cover a whole neighborhood V of
(t0 , y0 ) (we just vary the value of y ′ (t0 ) in the associated Cauchy problem and use some

result on the differentiability of the solution with respect to the initial data, coupled with
the inverse function theorem). We consider now the function S : V → R defined by
t

S(t, y) =

L s, x(s), x ′ (s) ds

t0

where the integral is computed along the extremal x(s) joining the points (t0 , y0 ) and (t, y).
It may be proved that S satisfies the first order nonlinear partial differential equation
St + H (t, y, Sy ) = 0.

(1.10)

This is the Hamilton–Jacobi equation. This is strongly related to the Hamiltonian system (1.7) which is the system of characteristics associated to the partial differential equation (1.10) (see [16]).
A partial differential equation is usually a more complicated mathematical object than
an ordinary differential system. Solving a first order partial differential system reduces
to solving the corresponding characteristic system. This is the method of characteristics
(see [16]).


8

V. Barbu and C. Lefter

However, this duality may be successfully used in a series of concrete situations to
integrate the Hamiltonian systems appearing in mechanics or in the calculus of variations. This result, belonging to Hamilton and Jacobi, states that if a general solution for
the Hamilton–Jacobi equation (1.10) is known, then the Hamiltonian system may be integrated (see [3,16,24]). More precisely, we assume that a general solution of (1.10) is

2S
∂S
S = S(t, y1 , . . . , yn , α1 , . . . , αn ) such that the matrix ( ∂y∂i ∂α
) is nondegenerated. Then ∂α
j
j
are prime integrals and a general solution of the Hamiltonian system (1.7) is given by the
2n system of implicit equations:
βi =

∂S
,
∂αi

pi =

∂S
.
∂yi

In fact S is a generating function for the symplectic transform (yi , pi ) → (βi , αi ) and in
the new coordinates the system (1.7) has a simple form for which the Hamiltonian function
is ≡ 0. A last remark is that a general solution to equation (1.10) may be found if variables
of H are separated (see [3,24]).
We considered previously first order necessary conditions. Suppose that (Ly ′ y ′ ) > 0. Let
us take now the second variation
d2
J (y + sh)
ds 2

δ 2 J (y)h :=

.
s=0

This is a quadratic form denoted by
Qy (h) =

t1

Ω y (t, h, h′ ) dt

t0

where the new Lagrangean
Ω y (t, h, h′ ) = Lyy (t, y, y ′ )h, h + 2 Lyy ′ (t, y, y ′ )h, h′
+ Ly ′ y ′ (t, y, y ′ )h′ , h′ .

Here (·, ·) denotes the scalar product in Rn and we assumed that the matrix (Lyy ′ ) is symmetric (for n = 1 this is trivial, in higher dimensions the hypothesis simplifies computations but may be omitted). Clearly, if y ∗ realizes a global minimum of J , then the quadratic
form Q(y ∗ ) 0. The positivity of Q is related to the notion of conjugate point. A point t
is conjugate to t0 along the extremal y ∗ if there exists a non trivial solution h : [t0 , t] → Rn ,
h(t0 ) = h(t) = 0 of the second Euler equation:
y∗

Ωh −

d y∗
Ω ′ = 0.
dt h

The Jacobi necessary condition states that if y ∗ realizes the infimum of J then the open
interval (t0 , t1 ) does not contain conjugate points to t0 . If y ∗ is just an extremal and the
closed interval [t0 , t1 ] does not contain conjugate points to t0 , then y ∗ is a local weak
minimum of J (in C 1 topology).


Optimal control of ordinary differential equations

9

1.2. General form of optimal control problems
We consider the controlled differential equation
y ′ (t) = f t, y(t), u(t) ,

t ∈ [0, T ].

(1.11)

The input function u : [0, T ] → Rm is called controller or control and y : [0, T ] → Rn is
the state of the system. We will assume that u ∈ U where U is the set of measurable, locally
integrable functions which satisfy the control constraints:
u(t) ∈ U (t) a.e. t ∈ [0, T ]

(1.12)

where U (t) ⊂ Rn are given closed subsets. The differential system (1.11) is called the state
system. We also consider a Lagrangean L and the cost functional
T

J (y, u) =

0

L t, y(t), u(t) dt + g y(0), y(T ) .

(1.13)

A pair (y, u) is said to be admissible pair if it satisfies (1.11), (1.12) and J (y, u) < +∞.
The optimal control problem we consider is
min J (y, u); y(0), y(T ) ∈ C, (y, u) verifies (1.11)

(1.14)

Here C ⊂ Rn × Rn is a given closed set.
A controller u∗ for which the minimum in (1.14) is attained is called optimal controller.
The corresponding states y ∗ are called optimal states while (y ∗ , u∗ ) will be referred as optimal pairs. By solution to (1.11) we mean an absolutely continuous function y : [0, T ] → R
(i.e., y ∈ AC([0, T ]; Rn ) which satisfies almost everywhere the system (1.11). In the special case f (t, y, u) ≡ u, problem (1.14) reduces to the classical problem of calculus of
variations that was discussed in Section 1.1. For different sets C we obtain different types
of control problems. For example, if C contains one element, that is the initial and final
states are given, we obtain a Lagrange problem. If the initial state of the system is given
and the final one is free, C = {y0 } × R, one obtains a Bolza problem. A Bolza problem
with the Lagrangean L ≡ 0 becomes a Mayer problem.
An optimal controller u∗ is said to be a bang-bang controller if u∗ ∈ ∂U (t) a.e.
t ∈ (0, T ) where ∂U stands for the topological boundary of U .
It should be said that the control constraints (1.12) as well as end point constraints
(y(0), y(T )) ∈ C can be implicitely incorporated into the cost functional J by redefining
L and g as
L(t, y, u) =

L(t, u) if u ∈ U (t),

+∞
otherwise,

g(y
˜ 1 , y2 ) =

g(y1 , y2 )
+∞

if (y1 , y2 ) ∈ C,
otherwise.


10

V. Barbu and C. Lefter

Moreover, integral (isoperimetric) constraints of the form
T

hi t, y(t), u(t) dt

αi ,

0

i = 1, . . . , l,

T
0

hi t, y(t), u(t) dt = αi ,

i = l + 1, . . . , m

can be implicitly inserted into problem (1.14) by redefining new state variables {z1 . . . , zm }
and extending the state system (1.11) to
 ′

 y (t) = f t, y(t), u(t) , t ∈ (0, T ),
z′ (t) = h t, y(t), u(t) ,


z(0) = 0, zi (T ) αi for i = 1, . . . , l, zi (T ) = αi for i = l + 1, . . . , m

where h = {hi }m
i=1 . For the new state variable X = (y, z) we have the end point constraints
X(0), X(T ) ∈ K,
where
K=

(y0 , 0, . . . , 0), (y1 , z) ∈ Rn+m × Rn+m , (y0 , y1 ) ∈ C,
zi

αi , i = 1, . . . , l, zi = αi , i = l + 1, . . . , m .

2. Preliminaries
2.1. Elements of convex analysis
Here we shall briefly recall some basic results pertaining convex analysis and generalized
gradients we are going to use in the formulation and in proof of the maximum principle.

Let X be a real Banach space with the norm · and dual X ∗ . Denote by (·, ·) the pairing
between X and X ∗ .
The function f : X → R = ]−∞, +∞] is said to be convex if
f λx + (1 − λ)y

λf (x) + (1 − λ)f (y),

0

λ

1, x, y ∈ X.

(2.1)

The set D(f ) = {x ∈ X; f (x) < ∞} is called the effective domain of f and
E(f ) = (x, λ) ∈ X × R; f (x)

λ

(2.2)

is called the epigraph of f . The function f is said to be lower semicontinuous (l.s.c.) if
lim inf f (x)
x→x0

f (x0 ).

The function f is said to be proper if f ≡ +∞.

Optimal control of ordinary differential equations

11

It is easily seen that a convex function is l.s.c. if and only if it is weakly lower semicontinuous. Indeed, f is l.s.c. if and only if every level set {x ∈ X; f (x) λ} is closed.
Moreover, the level sets are also convex, by the convexity of f ; the conclusion follows by
the coincidence of convex closed sets and weakly closed sets.
Note also that, by Weierstrass theorem, if X is a reflexive Banach space and if f is
convex, l.s.c. and lim x →∞ f (x) = +∞, then f attains its infimum on X.
We note without proof (see, e.g., [9,6]) the following result:
P ROPOSITION 2.1. Let f : X → R be a l.s.c. convex function. Then f is bounded from
below by an affine function and f is continuous on int D(f ).
Given a l.s.c. convex function f : X → R, the mapping ∂f : X → X ∗ defined by
∂f (x) = w ∈ X ∗ ; f (x)

f (u) + (w, x − u), ∀u ∈ X

(2.3)

is called the subdifferential of f . An element of ∂f (x) is called subgradient of f at x.
The mapping ∂f is generally multivalued. The set
D(∂f ) = x; ∂f (x) = φ
is the domain of ∂f . It is easily seen that x0 is a minimum point for f on X if and only if
0 ∈ ∂f (x0 ).
We note also, without proof, some fundamental properties of ∂f (see, e.g., [6,9,31]).
P ROPOSITION 2.2. Let f : X → R be convex and l.s.c. Then int D(f ) ⊂ D(∂f ).
Let C be a closed convex set and let IC (x) be the indicator function of C, i.e.,
IC (x) =

0,
+∞,

x ∈ C,
x∈
/ C.

Clearly, IC (x) is convex and l.s.c. Moreover, we have D(∂IC (x)) = C and
∂IC (x) = w ∈ X ∗ ; (w, x − u)

0, ∀u ∈ C .

∂IC (x) is precisely the normal cone to C at x, denoted NC (x).
If F : X → Y is a given function, X, Y Banach spaces, we set
F ′ (x, y) = lim

λ→0

F (x + λy) − F (x)
λ

called the directional derivative of F in direction y.
By definition F is Gâteaux differentiable in x if ∃DF (x) ∈ L(X, Y ) such that
F ′ (x, v) = DF (x)v,

∀v ∈ X.

(2.4)

12

V. Barbu and C. Lefter

In this case, DF is the Gâteaux derivative (differential) at x.
If f : X → R is convex and Gâteaux differentiable in x, then it is subdifferentiable at x
and ∂f (x) = ∇f (x).
In general, we have
P ROPOSITION 2.3. Let f : X → R be convex, l.s.c. and proper. Then, for each x0 ∈ D(∂f )
∂f (x0 ) = w ∈ X ∗ ; f ′ (x0 , u)

(w, u), ∀w ∈ X .

(2.5)

If f is continuous at x0 , then
f ′ (x0 , u) = sup (w, u); w ∈ ∂f (x0 ) ,

∀u ∈ X.

(2.6)

Given f : X → R, the function f ∗ : X ∗ → R
f ∗ (p) = sup (p, x) − f (x); x ∈ X
is called the conjugate of f , or the Legendre transform of f .
P ROPOSITION 2.4. Let f : X → R be convex, proper, l.s.c. Then the following conditions
are equivalent:
1. x ∗ ∈ ∂f (x),
2. f (x) + f ∗ (x ∗ ) = (x ∗ , x),
3. x ∈ ∂f ∗ (x ∗ ).

In particular, ∂f ∗ = (∂f )−1 and f = f ∗∗ . In general, ∂(f + g) ⊃ ∂f + ∂g and the
inclusion is strict. We have, however,
P ROPOSITION 2.5 (Rockafellar). Let f and g be l.s.c. and convex on D. Assume that
D(f ) ∩ int D(g) = φ. Then
∂(f + g) = ∂f + ∂g.

(2.7)

We shall assume now that X = H is a Hilbert space. Let f : H → R be convex, proper
and l.s.c. Then ∂f is maximal monotone. In other words,
(y1 − y2 , x1 − x2 )

0,

∀(xi , yi ) ∈ ∂f, i = 1, 2

R(I + λ∂f ) = H,

∀λ > 0.

(2.8)

and
(2.9)

R(I + λ∂f ) is the range of I + λ∂f .
The mapping
(∂f )λ = λ−1 I − (I + λ∂f )−1 ,

λ>0

(2.10)