Further Results on Subgradients of the Value Function to a Parametric Optimal Control Problem

J Optim Theory Appl
DOI 10.1007/s10957-011-9933-0

Further Results on Subgradients of the Value Function
to a Parametric Optimal Control Problem
N.H. Chieu · B.T. Kien · N.T. Toan

Received: 25 May 2011 / Accepted: 16 September 2011
© Springer Science+Business Media, LLC 2011

Abstract This paper studies the first-order behavior of the value function of a parametric optimal control problem with nonconvex cost functions and control constraints. By establishing an abstract result on the Fréchet subdifferential of the value
function of a parametric minimization problem, we derive a formula for computing
the Fréchet subdifferential of the value function to a parametric optimal control problem. The obtained results improve and extend some previous results.
Keywords Parametric optimal control · Marginal function · Value function ·
Fréchet normal cone · Fréchet subgradients · Fréchet subdifferential

1 Introduction
Value functions in optimal control problems can be interpreted as the value functions in certain parametric mathematical programming problems. The study of firstorder behavior of value functions is important in variational analysis and optimization. There are many papers dealing with differentiability properties and computation

Communicated by Boris Mordukhovich.
N.H. Chieu · N.T. Toan
Department of Mathematics, Vinh University, 182 Le Duan, Vinh City, Vietnam
N.H. Chieu
e-mail:
N.T. Toan
e-mail:
B.T. Kien ( )
Department of Information and Technology, Hanoi National University of Civil Engineering, 55 Giai
Phong, Hanoi, Vietnam
e-mail:

J Optim Theory Appl

of the Fréchet subdifferential of value functions in the literature (see, for instance,
[1–6]). We refer the reader to [7–12] for recent studies on sensitivity analysis of
the value function in parametric optimal control, which are related to the Hamilton–
Jacobi equation. In particular, a formula was obtained in [9] for the approximate
subdifferential of convex analysis of the value function to the case where the objective function was assumed to be convex and the problem may have no optimal path.
It is noted that if the objective function is convex then we can compute the subdifferential of the value function via subgradients of convex functions. However, the
situation will be more complicated if the objective function is nonconvex because
subdifferential calculus of convex functions fails to apply.
Recently, Toan and Kien [13] have obtained a formula for computing the Fréchet
subdifferential of the value function in the case where the objective function is nonconvex and the control constraint is absent. In this paper, we continue to develop
the approach of [13] to deal with the control-constrained problem. Namely, by using
some new techniques, we shall derive a formula for computing the Fréchet subdifferential of the value function, which extends and improves the main result of [13].

2 Parametric Optimal Control Problem
A variety of problems in optimal control can be posed in the following form:
Determine a control vector u ∈ Lp ([0, 1], Rm ) and a trajectory x ∈ W 1,p ([0, 1],
n
R ), 1 ≤ p < ∞, which minimize the cost
1

g x(1) +

L t, x(t), u(t), θ (t) dt

(1)

0

with the state equation
x(t)
˙ = A(t)x(t) + B(t)u(t) + T (t)θ (t)

a.e. t ∈ [0, 1],

(2)

the initial value
x(0) = α,

(3)

u ∈ U.

(4)

and the control constraint

Here W 1,p ([0, 1], Rn ) is the Sobolev space which consists of absolutely continuous
functions x : [0, 1] → Rn such that x˙ ∈ Lp ([0, 1], Rn ). Its norm is given by
x

1,p

= x(0) + x˙

p,

where
– x, u are the state variable and the control variable, respectively,
– (α, θ ) ∈ Rn × Lp ([0, 1], Rk ) are parameters,
¯ L : [0, 1] × Rn × Rm × Rk → R
¯ are given functions,
– g : Rn → R,


J Optim Theory Appl

– A(t) = (aij (t))n×n , B(t) = (bij (t))n×m and T (t) = (cij (t))n×k are matrix-valued
functions,
– U is a closed and convex set in Lp ([0, 1], Rm ).
Put
X := W 1,p [0, 1], Rn ,

U := Lp [0, 1], Rm ,

Θ := Lp [0, 1], Rk ,

W := Rn × Θ.

We now denote by V (w) the optimal value of the problem (1)–(4) corresponding
¯ is an extended-real-valued function
to parameter w = (α, θ ) ∈ W . Thus V : W → R
which is called the value function or the marginal function of (1)–(4).
Let us recall some notions on generalized differentiation related to our problem.
The notions and results of generalized differentiation using the dual-space approach
¯ an extended-realcan be found in [14–16]. Let Z be a Banach space, ϕ : Z → R
valued function, and z¯ ∈ Z be such that ϕ(¯z) is finite. The set

∗
ˆ z) := z∗ ∈ Z ∗ | lim inf ϕ(z) − ϕ(¯z) − z , z − z¯ ≥ 0
∂ϕ(¯
z→¯z
z − z¯

(5)

is called the Fréchet subdifferential of ϕ at z¯ . A given vector z∗ ∈ ∂ϕ(¯z) is called a
Fréchet subgradient of ϕ at z¯ . It is known that the Fréchet subdifferential reduces to
the classic Fréchet derivative for differentiable functions and to the subdifferential
ˆ
of convex analysis for convex functions. The set ∂ˆ + ϕ(¯z) := −∂(−ϕ)(¯
z) is called the
upper subdifferential of ϕ at z¯ . Let Ω be a nonempty subset of Z and z0 ∈ Ω. The
Fréchet normal cone to Ω at z0 is defined by setting
N (z0 ; Ω) := z∗ ∈ Z ∗ | lim sup
Ω
z−
→z0

z∗ , z − z0
≤0 .
z − z0

(6)

We say that a set-valued map F : Z ⇒ E, where E is a Banach space, admits a locally
upper Lipschitzian selection at (¯z, v)
¯ ∈ gph F , where

gph F := (z, v) ∈ Z × E|v ∈ F (z) ,
iff there is a single-valued mapping φ : Z → E such that there exist η > 0 and l > 0
satisfying
φ(z) − φ(¯z) ≤ l z − z¯

whenever z ∈ B(¯z, η),

φ(¯z) = v,
¯ and φ(z) ∈ F (z) for all z in a neighborhood of z¯ .
We now return back to the problem (1)–(4). For each w = (α, θ ) ∈ W, put
1

J (x, u, w) = g x(1) +

L t, x(t), u(t), θ (t) dt,

(7)

G(w) = z = (x, u) ∈ X × U |(2) and (3) aresatisfied

(8)

0


J Optim Theory Appl

and
K = X × U.

(9)

Then (1)–(4) can be reformulated in the form
V (w) :=

inf

z∈G(w)∩K

(10)

J (z, w).

Denote by S(w) the solution set of (1)–(4) corresponding to the parameter w ∈ W .
Throughout the paper, we assume that (x,
¯ u)
¯ ∈ S(w)
¯ and the following assumptions
hold:
¯ and g : Rn → R
¯ have the prop(H1) The functions L : [0, 1] × Rn × Rm × Rk → R
erties that
L(·, x, u, v) is measurable for all (x, u, v) ∈ Rn × Rm × Rk ,
L(t, ·, ·, ·) and g(·) are continuously differentiable for almost every t ∈ [0, 1],
and there exist constants c1 > 0, c2 > 0, r ≥ 0, a nonnegative function ω1 ∈
Lp ([0, 1], R) and constants 0 ≤ p1 ≤ p and 0 ≤ p2 ≤ p − 1 such that
|L(t, x, u, v)| ≤ c1 (ω1 (t) + |x|p1 + |u|p1 + |v|p1 ),
max |Lx (t, x, u, v)|, |Lu (t, x, u, v)|, |Lv (t, x, u, v)|
≤ c2 |x|p2 + |u|p2 + |v|p2 + r
for all (t, x, u, v) ∈ [0, 1] × Rn × Rm × Rk .

(H2) The matrix-valued functions A : [0, 1] → Mn,n (R), B : [0, 1] → Mn,m (R) and
T : [0, 1] → Mn,k (R) are measurable and essentially bounded.
(H3) The map [0, 1] t → |A(t)|p−1 is nonincreasing or there exists a constant
c3 > 0 such that
|T T (t)v| ≥ c3 |v|

∀ v ∈ Rn , a.e. t ∈ [0, 1].

We are now ready to state our main result.
Theorem 2.1 Suppose the value function V from (10) is finite at w¯ = (α,
¯ θ¯ ), int U =
∗
∗
n
q
∅, and (H1)–(H3) are fulfilled. Then, for a vector (α , θ ) ∈ R × L ([0, 1], Rk ) to
be a Fréchet subgradient of V at (α,
¯ θ¯ ) it is necessary that there exist functions y ∈
1,q
n
∗
q
¯ U) such that the following
W ([0, 1], R ) and u ∈ L ([0, 1], Rm ) with u∗ ∈ Nˆ (u,
conditions hold:
1

¯
+
α ∗ = g x(1)

0

¯
y(1) = −g x(1)
and

1

Lx t, x(t),
¯
u(t),
¯
θ¯ (t) dt −

AT (t)y(t) dt,

(11)

0

(transversality condition),

(12)


J Optim Theory Appl

y(t)
˙ + AT (t)y(t), B T (t)y(t) − u∗ (t), θ ∗ (t) + T T (t)y(t)
= ∇L t, x(t),

¯
u(t),
¯
θ¯ (t)

(13)

for a.e. t ∈ [0, 1]. Here ∇L(t, x(t),
¯
u(t),
¯
θ¯ (t)) stands for the gradient of L(t, ·, ·, ·) at
¯
(x(t),
¯
u(t),
¯
θ (t)).
ˆ (α,
The above conditions are also sufficient for (α ∗ , θ ∗ ) ∈ ∂V
¯ θ¯ ) if the solution
map S has a locally upper Lipschitzian selection at (w,
¯ x,
¯ u).
¯ By AT we denote the
transpose of A and by q the conjugate number of p, that is, 1 < q ≤ +∞ and 1/p +
1/q = 1.
When U = U or u¯ ∈ int U we obtain
Corollary 2.1 (A refinement of [13, Theorem 1.1]) Suppose the value function V
from (10) is finite at w¯ = (α,

¯ θ¯ ) and assumptions (H1)–(H3) are fulfilled. Then, for
∗
∗
n
q
a vector (α , θ ) ∈ R × L ([0, 1], Rk ) to be a Fréchet subgradient of V at (α,
¯ θ¯ ) it
1,q
n
is necessary that there exists a function y ∈ W ([0, 1], R ) such that the following
conditions hold:
1

α ∗ = g x(1)
¯
+
0

¯
y(1) = −g x(1)

1

¯
u(t),
¯
θ¯ (t) dt −
Lx t, x(t),

AT (t)y(t) dt,

(14)

0

(transversality condition),

(15)

and
y(t)
˙ + AT (t)y(t), B T (t)y(t), θ ∗ (t) + T T (t)y(t)
= ∇L t, x(t),
¯
u(t),
¯
θ¯ (t)

a.e. t ∈ [0, 1].

(16)

ˆ (α,
¯ θ¯ ) if the solution map
The above conditions are also sufficient for (α ∗ , θ ∗ ) ∈ ∂V
S has a locally upper Lipschitzian selection at (w,
¯ x,
¯ u).
¯
Note that in Theorem 2.1 and Corollary 2.1, except for assumptions (H1) and

(H2) which are similar to (A1) and (A2) in [13, Theorem 1.1], the assumption (H3)
is weaker than (A3) in [13, Theorem 1.1]. Here we do not require that the map s →
|A(s)|p−1 is nonincreasing and the condition
T T (t)v ≥ c3 |v|

∀ v ∈ Rn , a.e. t ∈ [0, 1]

occurs simultaneously. Besides, the condition
B T (t)u ≥ c|u|

∀ u ∈ Rn , a.e. t ∈ [0, 1]

for some constant c > 0 is also omitted. In the case of p = 1, where (x, u) ∈
W 1,1 ([0, 1], R n ) × L1 ([0, 1], R m ), assumption (H3) is automatically fulfilled.
In order to prove Theorem 2.1, we first reduce the control problem to a mathematical programming problem and then establish some formulas for computing the
Fréchet subdifferential of the value function. This procedure is presented in the next
section. The complete proof of Theorem 2.1 will be provided in Sect. 4.


J Optim Theory Appl

3 Reduction to a Mathematical Programming Problem
In this section, we suppose that X, W , and Z are Banach spaces with the dual spaces
X ∗ , W ∗ , and Z ∗ , respectively. Assume that M : Z → X and T : W → X are continuous linear mappings. Let M ∗ : X ∗ → Z ∗ and T ∗ : X ∗ → W ∗ be adjoint mappings
¯ be a function and Ω be a closed and
of M and T , respectively. Let f : Z × W → R
convex set in Z. For each w ∈ W, we put
H (w) := {z ∈ Z| Mz = T w}.
Consider the problem of computing the Fréchet subdifferential of the value function
h(w) :=

inf

z∈H (w)∩Ω

(17)

f (z, w).

This is an abstract model for (10).
We denote by S(w) the solution set of (17) corresponding to the parameter w ∈ Y
and assume that z¯ is a solution of the problem corresponding to the parameter w,
¯ that
is, z¯ ∈ S(w).
¯
The following result is an extension and improvement of [13, Theorem 2.1] which
gives a formula for computing the Fréchet subdifferential of h at w.
¯ Note that we do
not require that the spaces X, W and Z be reflexive. Besides, the proof below is quite
different from the proof of [13, Theorem 2.1]. Here we compute the normal cones
directly instead of using the separation theorem (see Lemma 3.1).
Theorem 3.1 Suppose the value function h from (17) is finite at w¯ ∈ domS and
(i) f is Fréchet differentiable at (¯z, w),
¯
(ii) the mapping M is surjective or there exists a constant c > 0 such that
T ∗x∗ ≥ c x∗

∀x ∗ ∈ X ∗ ,

(18)

(iii) Ω is a closed and convex set with intΩ = ∅.
Then one has
ˆ w)
∂h(
¯ ⊆

∇w f (¯z, w)
¯ + T ∗ (M ∗ )−1 ∇z f (¯z, w)
¯ + z∗

.

(19)

z∗ ∈N(¯z;Ω)

Moreover, if the solution map Sˆ has a locally upper Lipschitzian selection at (w,
¯ z¯ ),
then
ˆ w)
∂h(
¯ =

∇w f (¯z, w)
¯ + T ∗ (M ∗ )−1 ∇z f (¯z, w)
¯ + z∗

.

(20)

z∗ ∈N(¯z;Ω)

For the proof of this theorem, we need the following lemmas.
Lemma 3.1 Suppose that assumptions of Theorem 3.1 are satisfied. Then for each
(w,
¯ z¯ ) ∈ gphH one has
N (w,
¯ z¯ ); gphH =

−T ∗ x ∗ , M ∗ x ∗ |x ∗ ∈ X ∗ .

(21)


J Optim Theory Appl

Proof We now define a continuous linear mapping
Φ:

W × Z → X,

(w, z) → T w − Mz.

(22)

Then Q = gphH can be represented in the form
Q = (w, z) : Φ(w, z) = 0 = Φ −1 (0).

(23)

If M is surjective then Φ is surjective. If there exists a constant c > 0 such that (18)
is satisfied then [17, Theorem 4.15] implies that T is surjective. Hence Φ is also
surjective. Notice that Φ is a continuous linear mapping, so its derivative Φ (w, z) =
Φ for all (w, z) ∈ W × Z. Hence [15, Theorem 1.14] (see also [15, Corollary 1.15])
implies that
¯ z¯ ); {0} = Φ ∗ (X ∗ ).
N (w,
¯ z¯ ); Q = Φ ∗ N Φ(w,

(24)

Take any (w ∗ , z∗ ) ∈ Φ ∗ (X ∗ ). Then we have (w ∗ , z∗ ) = Φ ∗ (x ∗ ) for some x ∗ ∈ X ∗ .
By definition of adjoint mappings, we get
(w ∗ , z∗ ), (w, z) = x ∗ , Φ(w, z) = T ∗ x ∗ , w + −M ∗ x ∗ , z
for all (w, z) ∈ W × Z. It follows that w ∗ = T ∗ x ∗ and z∗ = −M ∗ x ∗ . Putting x0∗ =
−x ∗ , we get (w ∗ , z∗ ) = (−T ∗ x0∗ , M ∗ x0∗ ) for some x0∗ ∈ X ∗ . Hence (24) implies
N (w,
¯ z¯ ); Q =

−T ∗ x ∗ , M ∗ x ∗ |x ∗ ∈ X ∗ .

The proof of the lemma is complete.
In the following lemma, we set P = W × Ω.
Lemma 3.2 Assume that intΩ = ∅. Then one has
N (w,
¯ z¯ ); P ∩ Q = {0} × N (¯z; Ω) + Nˆ (w,
¯ z¯ ); Q .
Proof Since int Ω = ∅, we have int P = ∅. Thus P is a closed convex set with

nonempty interior. We now show that P and Q satisfy the normal qualification condition at (w,
¯ z¯ ), that is,
N (w,
¯ z¯ ); P ∩ −N (w,
¯ z¯ ); Q

= (0, 0) .

Take any (w ∗ , z∗ ) ∈ N ((w,
¯ z¯ ); P ) ∩ [−N ((w,
¯ z¯ ); Q)]. As N ((w,
¯ z¯ ); P ) = {0} ×
N(¯z; Ω), we have w ∗ = 0, z∗ ∈ N (¯z; Ω). Besides, w ∗ = T ∗ x ∗ , z∗ = −M ∗ x ∗ for
some x ∗ ∈ X ∗ . From the condition (ii), it follows that x ∗ = 0. Hence z∗ = 0. Thus
we obtain (w ∗ , z∗ ) = (0, 0) and so the normal qualification condition for P and Q
at (w,
¯ z¯ ) holds. According to [18, Proposition 1, Chapt. 4] and [18, Proposition 3,
Chapt. 4] we have
N (w,
¯ z¯ ); P ∩ Q = N (w,
¯ z¯ ); P + N (w,
¯ z¯ ); Q
= {0} × N (¯z; Ω) + N (w,
¯ z¯ ); Q .


J Optim Theory Appl

The lemma is proved.
Proof of Theorem 3.1 We shall use some arguments similar to those of [3].

ˆ w).
Take any w ∗ ∈ ∂h(
¯ Then for any γ > 0, there exists a neighborhood V˜ of w¯
such that
∀w ∈ V˜ .

¯ + γ w − w¯ ,
w ∗ , w − w¯ ≤ h(w) − h(w)
Hence

¯ + γ w − w¯
w ∗ , w − w¯ ≤ f (z, w) − f (¯z, w)

(25)

for all w ∈ V˜ and z ∈ H (w) ∩ Ω. Since f is Fréchet differentiable at (¯z, w),
¯ we have
¯ =
∂ˆ + f (¯z, w)

∇z f (¯z, w),
¯ ∇w f (¯z, w)
¯ .

Hence
¯ ∇w f (¯z, w)
¯
− ∇z f (¯z, w),

ˆ

= ∂(−f
)(¯z, w).
¯

Due to the Fréchet differentiability of f , we can choose U, V as neighborhoods of z¯
and w,
¯ respectively, with V ⊂ V˜ such that
¯ z − z¯ − ∇w f (¯z, w),
¯ w − w¯
−∇z f (¯z, w),
≤ f (¯z, w)
¯ − f (z, w) + γ

z − z¯ + w − w¯

(26)

for all (z, w) ∈ U × V . From (25) and (26), we obtain
¯ w − w¯ − ∇z f (¯z, w),
¯ z − z¯ ≤ 2γ
w ∗ − ∇w f (¯z, w),

z − z¯ + w − w¯

for all (w, z) ∈ (V × (Ω ∩ U )) ∩ Q. Since γ > 0 was chosen arbitrarily, it holds that
lim sup
P ∩Q
(w,z)−
−−→(w,¯
¯ z)

w ∗ − ∇w f (¯z, w),
¯ w − w¯ + −∇z f (¯z, w),
¯ z − z¯
≤ 0.
z − z¯ + w − w¯

This implies that
w ∗ − ∇w f (¯z, w),
¯ −∇z f (¯z, w)
¯ ∈ N (w,
¯ z¯ ); P ∩ Q .
By Lemma 3.2, we have
¯ −∇z f (¯z, w)
¯ ∈ {0} × N (¯z, Ω) + N (w,
¯ z¯ ); Q .
w ∗ − ∇w f (¯z, w),
Hence there exists z∗ ∈ N (¯z; Ω) such that
¯ z¯ ); Q .
¯ −∇z f (¯z, w)
¯ − z∗ ∈ N (w,
w ∗ − ∇w f (¯z, w),
By Lemma 3.1, there exists x ∗ ∈ X ∗ such that
¯ = −T ∗ x ∗
w ∗ − ∇w f (¯z, w)

and −∇z f (¯z, w)
¯ − z∗ = M ∗ x ∗ .

J Optim Theory Appl

This implies that
w ∗ = ∇w f (¯z, w)
¯ + T ∗ (−x ∗ )

and −x ∗ ∈ (M ∗ )−1 ∇z f (¯z, w)
¯ + z∗ .

Consequently,
w ∗ ∈ ∇w f (¯z, w)
¯ + T ∗ (M ∗ )−1 ∇z f (¯z, w)
¯ + z∗ ,
and so we obtain the first assertion.
In order to prove the second assertion, it is sufficient to show that
ˆ w)
∂h(
¯ ⊃

∇w f (¯z, w)
¯ + T ∗ (M ∗ )−1 ∇z f (¯z, w)
¯ + z∗

.

(27)

ˆ w).
\∂h(
¯

(28)

z∗ ∈N (¯z;Ω)

On the contrary, suppose that there exists w ∗ ∈ W ∗ such that
w∗ ∈

∇w f (¯z, w)
¯ + T ∗ (M ∗ )−1 ∇z f (¯z, w)
¯ + z∗
z∗ ∈N(¯z;Ω)

Then we can find a γ¯ > 0 and a sequence wk −→ ω¯ such that
w ∗ , wk − w¯ > h(wk ) − h(w)
¯ + γ¯ wk − w¯ , ∀k.

(29)

Let φ be an upper Lipschitzian selection of the solution map S. Putting zk = φ(wk ),
we have zk ∈ S(wk ) and zk − z¯ ≤ l wk − w¯ for k > 0 sufficiently large. Hence
(29) implies
w ∗ , wk − w¯ > h(wk ) − h(w)
¯ + γ¯ wk − w¯
¯ + γ¯ wk − w¯
= f (zk , wk ) − f (¯z, w)
¯ zk − z¯ + ∇w f (¯z, w),
¯ wk − w¯
= ∇z f (¯z, w),
+ 0 zk − z¯ + wk − w¯

+ γ¯ wk − w¯

¯ zk − z¯ + ∇w f (¯z, w),
¯ wk − w¯
≥ ∇z f (¯z, w),
+ 0 zk − z¯ + wk − w¯

+

γ¯
γ¯
wk − w¯ +
zk − z¯ .
2
2l

γ¯
Putting γˆ = min{ γ2¯ , 2l
}, we get from the above that

w ∗ − ∇w f (¯z, w),
¯ −∇z f (¯z, w)
¯ , (wk − w,
¯ zk − z¯ )
> 0 zk − z¯ + wk − w¯

+ γˆ

zk − z¯ + wk − w¯ .

Consequently,
lim sup
P ∩Q
(w,z)−
−−→(w,¯
¯ z)

(w ∗ − ∇w f (¯z, w),
¯ −∇z f (¯z, w)),
¯ (w − w,
¯ z − z¯ )
≥ γˆ .
z − z¯ + w − w¯


J Optim Theory Appl

This means that
¯ −∇z f (¯z, w)
¯ ∈
/ N (w,
¯ z¯ ); P ∩ Q .
w ∗ − ∇w f (¯z, w),
By Lemmas 3.1 and 3.2, we have
¯ −∇z f (¯z, w)
¯ − z∗
w ∗ − ∇w f (¯z, w),
−T ∗ x ∗ , M ∗ x ∗ |x ∗ ∈ X ∗ ,

∈
/ N (w,
¯ z¯ ); Q =

(30)

for all z∗ ∈ N (¯z; Ω). From (28) we can find a vector z∗ ∈ N (¯z; Ω) such that
¯ ∈ T ∗ (M ∗ )−1 ∇z f (¯z, w)
¯ + z∗ .
w ∗ − ∇w f (¯z, w)
It follows that there exists x ∗ ∈ (M ∗ )−1 (∇z f (¯z, w)
¯ + z∗ ) such that
¯ = T ∗ (x ∗ ).
w ∗ − ∇w f (¯z, w)
Hence
¯ = −T ∗ (−x ∗ )
w ∗ − ∇w f (¯z, w)

and −∇z f (¯z, w)
¯ − z∗ = M ∗ (−x ∗ ).

Consequently,
¯ −∇z f (¯z, w)
¯ − u∗
w ∗ − ∇w f (¯z, w),
∈ N (w,
¯ z¯ ); Q = (−T ∗ v ∗ , M ∗ v ∗ )|v ∗ ∈ X ∗ ,
which contradicts (30). Hence (27) is valid, and the proof of the theorem is complete.

4 Proof of the Main Result

To prove Theorem 2.1, we first formulate problem (1)–(4) in the form to which Theorem 3.1 can be applied to. Consider the linear mappings A : X → X, B : U → X,
M : X × U → X, and T : W → X defined by setting
(·)

Ax := x −

A(τ )x(τ ) dτ,

(31)

0
(·)

Bu := −

B(τ )u(τ ) dτ,

(32)

0

M(x, u) := Ax + Bu,

(33)

(·)

T (α, θ ) := α +

T (τ )θ (τ ) dτ.

0

(34)


J Optim Theory Appl

Under (H2) and (H3), (8) implies
(·)

G(w) = (x, u) ∈ X × U | x = α +

(·)

Ax dτ +

0
(·)

= (x, u) ∈ X × U | x −

0
(·)

Ax dτ −

0

(·)

Bu dτ +

T θ dτ
0
(·)

Bu dτ = α +

0

T θ dτ
0

= (x, u) ∈ X × U | M(x, u) = T (w) .

(35)

Since 1 ≤ p < ∞, we have Lp ([0, 1], Rn )∗ = Lq ([0, 1], Rn ), where q is the conjugate number of p. Besides, Lp ([0, 1], Rn ) is pared with Lq ([0, 1], Rn ) by the formula
1

x∗, x =

x ∗ (t)x(t) dt

0

x∗

∈ Lq ([0, 1], Rn )

∈ Lp ([0, 1], Rn ).

and x
for all
Also, we have W 1,p ([0, 1], Rn )∗ = Rn × Lq ([0, 1], Rn ) and W 1,p ([0, 1], Rn ) is
pared with Rn × Lq ([0, 1], Rn ) by the formula
1

(a, v), x = a, x(0) +

v(t)x(t)
˙ dt

0

for all (a, v) ∈ Rn × Lq ([0, 1], Rn ) and x ∈ W 1,p ([0, 1], Rn ) (see [18, p. 21]).
In the case of p = 2, W 1,2 ([0, 1], Rn ) becomes a Hilbert space with the inner
product given by
1

x, y = x(0), y(0) +

x(t)
˙ y(t)
˙ dt

∀x, y ∈ W 1,2 [0, 1], Rn .

0

We shall need two lemmas from [13].
Lemma 4.1 ([13, Lemma 2.3]) Suppose M∗ and T ∗ are adjoint mappings of M
and T , respectively. Then the following assertions are true:
(a) The mappings M and T are continuous.
(b) T ∗ (a, v) = (a, T T v) for all (a, v) ∈ Rn × Lq ([0, 1], Rk ).
(c) M∗ (a, v) = (A∗ (a, v), B ∗ (a, v)), where B ∗ (a, v) = −B T v and
1

A∗ (a, v) = a −

(·)

AT (t)v(t) dt; v +

0

0

0

for all (a, v) ∈ Rn × Lq ([0, 1], Rn ).
Recall that K = X × U and
G(w) = (x, u) ∈ X × U | M(x, u) = T (w) .
Put Z = X × U . Our problem can be written in the form
V (w) :=

inf

1

AT (τ )v(τ ) dτ −

z∈G(w)∩K

J (z, w)

AT (t)v(t) dt


J Optim Theory Appl

with z = (x, u) ∈ Z, w = (α, θ ) ∈ W and G(w) = {z ∈ Z : M(z) = T (w)}, where
M : Z → X and T : W → X are defined by (33) and (34), respectively.
Note that, by [13, Lemma 2.4], the mapping A is surjective. Hence M is also
surjective. Combining this fact with [13, Lemma 3.1], we have the following lemma.
Lemma 4.2 Suppose that assumptions (H1), (H2) and (H3) are valid. Then the following are fulfilled:
(a) The mapping M is surjective or there exists a constant c > 0 such that
T ∗x∗ ≥ c x∗ ,

∀x ∗ ∈ X ∗ .

(b) The functional J is Fréchet differentiable at (¯z, w)
¯ and ∇J (¯z, w)
¯ is given by
¯
u(t),
¯
θ¯ (t) ,
¯ = 0, Lθ t, x(t),
∇w J (¯z, w)

¯ = Jx (x,
¯ u,
¯ θ¯ ), Ju (x,
¯ u,
¯ θ¯ )
∇z J (¯z, w)
with
¯ u,
¯ θ¯ ) = Lu (·, x,
¯ u,
¯ θ¯ )
Ju (x,
and
1

¯ u,
¯ θ¯ ) = g x(1) +
Jx (x,

¯
u(t),
¯
θ¯ (t) dt,
Lx t, x(t),

0

g x(1) +

1

¯
u(t),
¯
θ¯ (t) dt .
Lx t, x(t),

(·)

We now return to the proof of Theorem 2.1, our main result.
By Lemma 4.2, all the assumptions of Theorem 3.1 are fulfilled. According to
Theorem 3.1, there exists a function z∗ ∈ Nˆ (¯z; K) such that
ˆ (w)
¯ + T ∗ (M∗ )−1 ∇z J (¯z, w)
¯ + z∗ .
∂V
¯ ⊂ ∇w J (¯z, w)

(36)

ˆ (w).
Note that z∗ = (0, u∗ ) for some u∗ ∈ N (u;
¯ U). We now take (α ∗ , θ ∗ ) ∈ ∂V
¯ By
(36),
(α ∗ , θ ∗ ) − ∇w J (¯z, w)
¯ ∈ T ∗ (M∗ )−1 ∇z J (¯z, w)
¯ + z∗ .
This is equivalent to
¯ ∈ T ∗ (M∗ )−1 ∇z J (¯z, w)

¯ + z∗ .
α ∗ , θ ∗ − Jθ (¯z, w)
Hence there exists (a, v) ∈ Rn × Lq ([0, 1], Rn ) such that
α ∗ , θ ∗ − Jθ (¯z, w)
¯ = T ∗ (a, v)

and ∇z J (¯z, w)
¯ + z∗ = M∗ (a, v).

By Lemma 4.1,
(37) ⇔

¯ = T T (·)v(·),
α ∗ = a; θ ∗ − Jθ (¯z, w)
¯ u,
¯ w),
¯ Ju (x,
¯ u,
¯ w)
¯ + u∗ ) = (A∗ (a, v), B ∗ (a, v)).
(Jx (x,

(37)


J Optim Theory Appl

⇔

⎧ ∗

α = a,
⎪
⎪
⎪ θ ∗ = L (·, x(·),
⎪
¯
u(·),
¯
θ¯ (·)) + T T (·)v(·),
⎪
θ
⎪
⎪
1
⎨ g (x(1))
¯
+ L (t, x(t),
¯
u(t),
¯
θ¯ (t)) dt = a −

1 T
0 x
0 A (t)v(t) dt,
1
¯
+ (·) Lx (τ, x(τ
¯ ), u(τ
¯ ), θ¯ (τ )) dτ

g (x(1))
⎪
⎪
⎪
⎪
(·)
1
⎪
= v(·) + 0 AT (τ )v(τ ) dτ − 0 AT (t)v(t) dt,
⎪
⎪
⎩
Lu (·, x(·),
¯
u(·),
¯
θ¯ (·)) + u∗ = −B T (·)v(·).

⎧ ∗
α = a,
⎪
⎪
⎪ θ ∗ = Lθ (·, x(·),
⎪
¯
u(·),
¯
θ¯ (·)) + T T (·)v(·),
⎨
1

1
¯
+ 0 Lx (t, x(t),
¯
u(t),
¯
θ¯ (t)) dt = a − 0 AT (t)v(t) dt,
⇔ g (x(1))
⎪
(·)
(·)
⎪
⎪
¯
− 1 Lx (τ, x(τ
¯ ), u(τ
¯ ), θ¯ (τ )) dτ = v(·) + 1 AT (τ )v(τ ) dτ,
⎪
⎩ g (x(1))
Lu (·, x(·),
¯
u(·),
¯
θ¯ (·)) + u∗ = −B T (·)v(·).
⎧
v ∈ W 1,q ([0, 1], Rn ),
⎪
⎪
⎪
⎪

θ ∗ − T T (·)v(·) = Lθ (·, x(·),
¯
u(·),
¯
θ¯ (·)),
⎪
⎪
⎨ ∗
1
1
¯
+ 0 Lx (t, x(t),
¯
u(t),
¯
θ¯ (t)) dt + 0 AT (t)v(t) dt,
α = g (x(1))
⇔
(38)
⎪
¯
v(1) = g (x(1)),
⎪
⎪
⎪
⎪
−v(·)
˙ − AT (·)v(·) = Lx (·, x(·),
¯
u(·),

¯
w(·)),
¯
⎪
⎩
−B T (·)v(·) = Lu (·, x(·),
¯
u(·),
¯
θ¯ (·)) + u∗ .
For y := −v we get
⎧
1
1
⎪
α ∗ = g (x(1))
¯
+ 0 Lx (t, x(t),
¯
u(t),
¯
θ¯ (t)) dt − 0 AT (t)y(t) dt,
⎪
⎨
¯
y(1) = −g (x(1)),
(38) ⇔
⎪
(y(t)
˙ + AT (t)y(t), B T (t)y(t) − u∗ (t), θ ∗ (t) + T T (t)y(t))

⎪
⎩
= ∇L(t, x(t),
¯
u(t),
¯
θ¯ (t)),
for a.e. t ∈ [0, 1]. This is the first assertion of the theorem. Using the second conclusion of Theorem 3.1, we also obtain the second assertion of the theorem. The proof
is complete.

5 An Example
We will illustrate the obtained result by a concrete problem. Put
X := W 1,2 [0, 1], R2 ,
Θ := L2 [0, 1], R2 ,

U := L2 [0, 1], R2 ,
W := R2 × Θ.

Consider the problem
1

J (x, u, θ ) = −x2 (1) +
0

u21 +

1
+ u22 + θ12 + θ22 dt −→ inf
1 + u21

J Optim Theory Appl

⎧
x˙1 = 2x1 + θ1 ,
⎪
⎪
⎪
⎪
⎪
⎨ x˙2 = x2 + u2 + θ2 ,
subject to x1 (0) = α1 ,
⎪
x2 (0) = α2 ,
⎪
⎪
⎪
⎪
⎩ u ≤ 1 e2 −1 .
2

2

(39)

2

Let (α,
¯ θ¯ ) = ((1, 1), (0, 0)). The following assertions are valid:
(i) The pair (x,

¯ u),
¯ where
x¯ = e2t , 1 +

1
u¯ = 0, e−t+1 ,
2

e t e −t
,
e − e
4
4

is a solution of the problem corresponding to (α,
¯ θ¯ ).
∗
∗
∗
ˆ
¯
(ii) If (α , θ ) ∈ ∂V (α,
¯ θ ), then α = (0, −e) and θ ∗ = (0, −e1−t ).
Indeed, for (α,
¯ θ¯ ) = ((1, 1), (0, 0)) the problem becomes
1

J0 (x, u) = −x2 (1) +

u21 +

0

⎧
⎪
⎨ x˙1 = 2x1 ,
x1 (0) = 1,
⎪
⎩ u ≤1

subject to

2

1
+ u22 dt −→ inf
1 + u21

x˙2 = x2 + u2 ,
x2 (0) = 1,

(40)

e2 −1
2 .

2

By direct computation, we see that the pair (x,
¯ u)

¯ satisfies (40). Moreover, u¯ ∈ ∂U ,
where ∂U is the boundary of U and U = B(0, ) ⊂ L2 ([0, 1], R2 ) with
Besides,

=

1
2

e2 −1
2 .

9
e2
−e+ .
8
8
We now claim that (x,
¯ u)
¯ is a solution of the control problem under consideration. In
fact, for any (x, u) satisfying (40) we have
J0 (x,
¯ u)
¯ =−

1

J0 (x, u) = −x2 (1) +
0

1

≥ −x2 (1) +
0

1

= −x2 (1) +

u21 +

1
+ u22 dt
1 + u21

1 + u22 dt
1 + (x˙2 − x2 )2 dt

0
1

=

(x˙2 − x2 )2 − x˙2 dt.

(41)

0

Consider the variational problem

1

Jˆ(x2 ) :=
0

(x˙2 − x2 )2 − x˙2 dt −→ inf.

(42)


J Optim Theory Appl

Since Jˆ is a convex function, by solving the Euler equation we obtain that xˆ2 (t) =
cet + (1 − c)e−t is a solution of (42), where c is determined by c = ae−1
and a =
e2 −1
x2 (1). Hence
e2 − 1
1
Jˆ(x2 ) ≥ Jˆ(xˆ2 ) = 1 + 2 2 (c − 1)2 + (c − 1) − e(c − 1) − e.
e
e

(43)

Combining (41) with (43) and putting r = c − 1, we obtain
J0 (x, u) ≥ 1 + 2
≥−

e2 − 1 2 1

r + r − er − e
e
e2

e2
9
¯ u).
¯
− e + = J0 (x,
8
8

Hence (x,
¯ u)
¯ is a solution of the problem corresponding to (α,
¯ θ¯ ). Assertion (i) is
proved.
It remains to prove (ii). From (39), we get
A=

2
0

0
,
1

B=

0 0

,
0 1

T=

1 0
.
0 1

It is easy to see that (H2) and (H3) are fulfilled. Since
L(t, x, u, θ ) = u21 +

1
+ u22 + θ12 + θ22 ,
1 + u21

we have
L(t, x, u, θ ) ≤ |u|2 + |θ |2 + 1,
Lu (t, x, u, θ ) ≤ 2 |u| + 1 ;

Lθ (t, x, u, θ ) ≤ 2|θ |

for all (x, u, θ ) ∈ R2 × R2 × R2 and a.e. t ∈ [0, 1]. Thus condition (H1) is also valid.
ˆ (α,
Hence all the assumptions of Theorem 2.1 are fulfilled. Let (α ∗ , θ ∗ ) ∈ ∂V
¯ θ¯ ). By
Theorem 2.1, there exists y = (y1 , y2 ) ∈ W 1,2 ([0, 1], R2 ) such that
⎧
⎪
⎪ y˙1 = −2y1 ,

⎨
y˙2 = −y2 ,
y˙ + AT y = ∇x L(t, x,
¯ u,
¯ θ¯ ),
⇔
y1 (1) = 0,
y(1) = −g (x(1))
⎪
⎪
⎩
y2 (1) = 1.
This implies that (y1 , y2 ) = (y¯1 , y¯2 ) = (0, e1−t ). By 13, we have
θ ∗ = −T T y + ∇θ L(t, x,
¯ u,
¯ θ¯ ) = −T T y.
It follows that θ ∗ (t) = (0, −e1−t ). On the other hand, from (13) and (11) we get
u∗ (t) = B T (t)y(t) − Lu t, x(t),
¯
u(t),
¯
θ¯ (t)

(44)


J Optim Theory Appl

and
1

¯
+
α ∗ = g x(1)
0

1

¯
u(t),
¯
θ¯ (t) dt −
Lx t, x(t),

AT (t)y(t) dt.

(45)

0

Substituting y = y¯ into (44), we get u∗ = 0 which satisfies the condition u∗ ∈
N(u,
¯ U). Substituting y = y¯ into (45), we obtain α ∗ = (0, −e).

6 Concluding Remarks
In summary, we have established a formula for computing the Fréchet subdifferential of the value function to a parametric optimal control problem which is governed
by linear differential equations and satisfies conditions (H1)–(H3). Note that when
p = 1, where (x, u) ∈ W 1,1 ([0, 1], R n ) × L1 ([0, 1], R m ), assumption (H3) is automatically fulfilled. In this case, our result covers classical results for the case with
only one parameter α.
It is known that in optimal control we often require that u ∈ L∞ for which the

pointwise constraint u ∈ U a.e. is easy to process. However, in our case, we have
to deal with dual spaces (Lp )∗ , so it will become more complicated when control
variable u ∈ Ł∞ . For this reason, we required that u ∈ Lp , 1 ≤ p < +∞ instead of
u ∈ L∞ . Thus for the case of p = +∞, our problem is unsolved.
Acknowledgements This research was supported in part by the project “Joint research and training
on Variational Analysis and Optimization Theory, with oriented applications in some technological areas
(Vietnam–USA)”. The authors wish to thank the anonymous referees for their suggestions and comments.

References
1. Clarke, F.H.: Method of Dynamic and Nonsmooth Optimization. SIAM, Philadelphia (1989)
2. Clarke, F.H.: Optimization and Nonsmooth Analysis. SIAM, Philadelphia (1990)
3. Mordukhovich, B.S., Nam, N.M., Yen, N.D.: Subgradients of marginal functions in parametric mathematical programming. Math. Program. 116, 369–396 (2009)
4. Mordukhovich, B.S., Nam, N.M.: Variational stability and marginal functions via generalized differentiation. Math. Oper. Res. 30, 800–816 (2005)
5. Moussaoui, M., Seeger, A.: Sensitivity analysis of optimal value functions of convex parametric programs with possibly empty solution sets. SIAM J. Optim. 4, 659–675 (1994)
6. Penot, J.-P.: Differentiability properties of optimal value functions. Can. J. Math. 56, 825–842 (2004)
7. Rockafellar, R.T.: Hamilton–Jacobi theory and parametric analysis in fully convex problems of optimal control. J. Glob. Optim. 248, 419–431 (2004)
8. Cernea, A., Frankowska, H.: A connection between the maximum principle and dynamic programming for constrained control problems. SIAM J. Control Optim. 44, 673–703 (2005)
9. Moussaoui, M., Seeger, A.: Epsilon-maximum principle of Pontryagin type and perturbation analysis
of convex optimal control problems. SIAM J. Control Optim. 34, 407–427 (1996)
10. Rockafellar, R.T., Wolenski, P.R.: Convexity in Hamilton–Jacobi theory I: Dynamics and duality.
SIAM J. Control Optim. 39, 1323–1350 (2000)
11. Rockafellar, R.T., Wolenski, P.R.: Convexity in Hamilton–Jacobi theory II: Envelope representation.
SIAM J. Control Optim. 39, 1351–1372 (2000)
12. Seeger, A.: Subgradient of optimal-value function in dynamic programming: the case of convex system without optimal paths. Math. Oper. Res. 21, 555–575 (1996)
13. Toan, N.T., Kien, B.T.: Subgradients of the value function to a parametric optimal control problem,
set-valued and variational. Analysis 18, 183–203 (2010)


J Optim Theory Appl
14. Borwein, J.M., Zhu, Q.J.: Techniques of Variational Analysis. Springer, New York (2005)

15. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation I, Basis Theory. Springer,
Berlin (2006)
16. Mordukhovich, B.S.: Variational Analysis and Generalized Differentiation II, Applications. Springer,
Berlin (2006)
17. Rudin, W.: Functional Analysis. McGraw-Hill, New York (1991)
18. Ioffe, A.D., Tihomirov, V.M.: Theory of Extremal Problems. North-Holland, Amsterdam (1979)