Xấp xỉ, ổn định và điều kiện tối ưu trong tối ưu không trơn

VIETNAM NATIONAL UNIVERSITY - HO CHI MINH CITY
UNIVERSITY OF SCIENCE

HUYNH THI HONG DIEM

APPROXIMATIONS, STABILITY AND
OPTIMALITY CONDITIONS
IN NONSMOOTH OPTIMIZATION

PhD THESIS IN MATHEMATICS

Hochiminh City - 2015


VIETNAM NATIONAL UNIVERSITY HO CHI MINH CITY
UNIVERSITY OF SCIENCE

HUYNH THI HONG DIEM

APPROXIMATIONS, STABILITY AND
OPTIMALITY CONDITIONS
IN NONSMOOTH OPTIMIZATION

Major: Optimization Theory
Code: 62 46 20 01

First examiner: Associate Prof. Dr. NGUYEN DINH HUY
Second examiner: Associate Prof. Dr. NGUYEN NGOC HAI
Third examiner: Dr. NGUYEN DINH TUAN
First independent examiner: Prof. DSc. NGUYEN DONG YEN
Second independent examiner: Dr. DINH NGOC QUY

SCIENTIFIC SUPERVISOR: Prof. DSc. PHAN QUOC KHANH

Hochiminh City - 2015


Declarization of originality
I hereby declare that this thesis, done under the supervision of Professor Phan Quoc
Khanh, is entirely the result of my own work. To the best of my knowledge, this study
and its findings have never been published by any other researchers. There is no part
of this thesis which overlaps parts of other works submitted for the award of any degree
or diploma. I also obtained the consent of Dr. Le Thanh Tung, co-author of the joint
paper [D4] referred to in Chapter 5, to let me include in my thesis some of the results
of this joint paper, which were not included in his thesis defended three years before
their publication.

Hochiminh City, November, 2015
The author

Huynh Thi Hong Diem

i


Acknowledgements
The completion of this doctoral dissertation would not have been possible without the
support of many people. I would like to express my sincere gratitude to all of them.
First, I want to express my deepest gratitude to my supervisor, Professor Phan
Quoc Khanh, for his valuable guidance, scholarly inputs, and consistent encouragement
I received throughout my research work. From deciding on the research topic in the

beginning to the process of actual writing of the thesis, he offered his unreserved help
and guided me through every step of my work. He provided me with inspiring and
insightful guidance, without which this study would never have been brought to a
successful completion.
Second, I am very pleased to extend my thanks to the reviewers of this thesis. Their
comments, remarks and questions have truly improved the quality of this manuscript.
I would like to thank the professors who agreed to be on the jury judging my thesis
defense. I would like to express my thankfulness to the members of the Group of
Optimization in Southern Vietnam. Their discussions during the seminar activities
have provided me with valuable encouragements and suggestions. In particular, I would
like to appreciate Dr. Le Thanh Tung who extended his support in a very special way
during the work on our joint paper, and I benefited a lot from his scholarly interactions
and suggestions. My thanks also go to the University of Science, Vietnam National
University Hochiminh City, and the College of Can tho for their support during my
work to complete the PhD program.
Last but not least, I owe a lot to my mother and my aunts, who support, encourage and help me at every stage of my personal and academic life, and long to see
this achievement come true. They always make sure I am provided with a carefree

ii


environment where I can devote myself entirely to my study (by, for example, taking
good care of my two children from their infant days so that I can focus on my PhD
study). I really appreciate so much my children sacrified by agreeing to stay with their
grandmother missing their busy mother’s warmth and care.

Hochiminh City, November, 2015
The author

Huynh Thi Hong Diem

iii


Contents
Certificate of originality

i

Acknowledgements

ii

List of symbols and notations

vi

Preface

ix

1. Basic notations and preliminary facts

1

2. Variational convergence of finite-valued bifunctions

7

2.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

7

2.2. Epi-convergence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

2.2.1. Epi-convergence . . . . . . . . . . . . . . . . . . . . . . . . . . .

9

2.2.2. Legendre-Fenchel transform and its continuity . . . . . . . . . .

10

2.3. Epi/hypo convergence and lopsided convergence, geometric characterizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

12

2.4. A characterization by e/h-convergence of proper bifunctions . . . . . .

23

2.5. A characterization by continuity of partial Legendre-Fenchel transform

25

2.6. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

32

3. Variational properties of epi/hypo convergence and approximations
of optimization problems

33

3.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

33

3.2. Variational properties of epi-convergence . . . . . . . . . . . . . . . . .

35

iv


3.3. Variational properties of epi/hypo convergence . . . . . . . . . . . . . .

37

3.4. Approximations of equilibrium problems . . . . . . . . . . . . . . . . .

43

3.5. Approximations of multi-objective optimization . . . . . . . . . . . . .

47

3.6. Approximations of Nash equilibria . . . . . . . . . . . . . . . . . . . . .

50

3.7. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

53

4. Variational convergence of bifunctions on nonrectangular domains
and approximations of quasiequilibrium problems

55

4.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

55

4.2. Variational convergence of bifunctions on nonrectangular domains . . .

56

4.3. Variational properties of epi/hypo convergence . . . . . . . . . . . . . .

62

4.4. Approximations of quasiequilibrium problems . . . . . . . . . . . . . .

69

4.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

72

5. Higher-order sensitivity analysis in nonsmooth vector optimization

74

5.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

74

5.2. Higher-order radial-contingent derivatives . . . . . . . . . . . . . . . . .

76

5.3. Properties of higher-order contingent-type derivatives . . . . . . . . . .

87

5.4. Higher-order contingent-type derivatives of perturbation maps . . . . .

92

5.5. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100
6. Optimality conditions for a class of relaxed quasiconvex minimax
problems

101

6.1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101
6.2. Optimality conditions for minimax problem (P) . . . . . . . . . . . . . 103

6.3. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
General conclusions

112

List of the author’s papers related to the thesis

114

List of the author’s conference reports related to the thesis

115

References

116
v


List of symbols and notations
N

the set of the natural numbers

Q

the set of the rational numbers

R

the set of the real numbers

¯ = R ∪ {−∞, ∞}
R

the set of the extended straight line

Rn

an n-dimensional normed space

X∗

the dual/conjugate space of a normed space X

f∗

the Legendre-Fenchel conjugate of a numerical function f

clA

the closure of a set A

intA

the interior of a set A

bdA

the boundary of a set A

coA

the convex hull of a set A

dist(x, A)

the distance from x to a subset A.

∀

for all

∃

there exists

✷

the end of a proof

U(¯
x)

the collection of the neighbourhoods of x¯

fcn(Rn )

the collection of the extended real-valued unifunctions

pfcn(Rn )

the collection of the proper functions of fcn(Rn )

biv(Rn × Rm )

the collection of the extended real-valued bifunctions

e/h

epi/hypo

fv-fcn(Rn )

the collection of the finite-valued unifunctions

fv-biv(Rn × Rm )

the collection of the finite-valued bifunctions on rectangles

e

f ν → f or f = e-limν f ν

f ν epi converge to f

vi


e-lsν f ν

the upper epi-limit of functions f ν

e-liν f ν

the lower epi-limit of functions f ν

usc

upper semicontinuous

lsc

lower semicontinuous

coneA

the conical hull of a set A

limsupx →x f (x )

infδ>0 supx ∈B(x,δ) f (x ) for f : X → R

liminfx →x f (x )

supδ>0 infx ∈B(x,δ) f (x ) for f : X → R

argminf

the set of the minimizers of a function f

argmaxf

the set of the maximizers of a function f

infA

the infimum value of a numerical set A

supA

the supremum value of a numerical set A

domf

the domain of a numerical function f

epif

the epigraph of a numerical function f

hypof

the hypograph of a numerical function f

domF

the domain of a set-valued map

F+

the profile map of a set-valued map

gphF

the graph of F

epiF :=epiF+
Limsupν K ν

the upper/outer limit of a sequence of set-valued maps K ν

Liminfν K ν

the lower/inner limit of a sequence of set-valued maps K ν

MinC A

the collection of the local (Pareto) minimal points of A

WMinC A

the collection of the local weak minimal points of A

HeC A

the collection of the Henig-proper minimal points of A

QMinC A

the collection of the local Q-minimal poinst of A

m

D F (x0 , y0 , u1 , v1 , ..., um−1 , vm−1 ) the mth-order contingent derivative of F at (x0 , y0 )
wrt (u1 , v1 ), ..., (um−1 , vm−1 ) ∈ X × Y
Dm F (x0 , y0 )

the mth-order contingent derivative of F at (x0 , y0 )

m
DR
F (x0 , y0 )

the mth-order radial derivative of F at (x0 , y0 )

DSm F (x0 , y0 )

the mth-order radial-contingent derivative of F at (x0 , y0 )

Dbm F (x0 , y0 )

the mth-order adjacent-type derivative of F at (x0 , y0 )

vii


Dlm F (x0 , y0 )

the mth-order strong adjacent-type derivative of F at (x0 , y0 )

L(X, Y )

the space of the bounded linear maps from X to Y

Lf (¯
x)

the sublevel set of f at x¯

L<
x)
f (¯

the strict sublevel set of f at x¯

x)
Laf (¯

the adjusted sublevel set of f at x¯

∂ < f (¯
x)

the lower subdifferential or Plastria subdifferential

∂ ≤ f (¯
x)

the infradifferential or Gutierrez subdifferential

∂ ∗ f (¯
x)

the Greenberg-Pierskalla subdifferential

∂ ν f (¯
x), ∂ f (¯
x)

the normal-cone subdifferentials

∂ a f (¯
x)

the adjusted subdifferential

viii


Preface
Like in all the other areas of applied mathematics, most optimization problems in
practice are nonsmooth, i.e., the functions and maps involved in the problem data do
not have derivatives in the classical sense of Fr´echet and Gateaux. There are two ways
of dealing with such problems: using derivative-free objects or developing generalized
derivatives to replace the classical ones as tools of study. In this thesis, we follow both
approaches, depending on the situation under our consideration. So, the terminology
“nonsmooth optimization” in this thesis is understood more widely than in a number of
publications where this terminology is used only for studies with generalized derivatives.
Optimization is a very broad mathematical area, including various topics such as

the existence of solutions, stability of solutions, well-posedness of problems, optimality
conditions, duality, numerical methods, etc. Our study focus only on stability and
optimality conditions. First, regarding stability, there are qualitative and quantitative considerations of stability. To specify the topics, besides the general meaning,
the word “stability” sometimes stands specifically for qualitative stability, while “sensitivity analysis” designates quantitative stability. However, the splitting is rather
relative (and some authors use the general word “stability” for both). In (qualitative)
stability, one investigates various types of semicontinuity, continuity, and also calmness or Holder calmness, Lipschitz or Holder continuity of solution sets with respect
to parameters if no (or few) results on estimates/computations of constants/modulus
and (Holder) degrees are discussed. Quantitative stability (i.e., sensitivity analysis) is
related to estimates/computations of quantities reflecting stability properties. So, usually, in sensitivity analysis various (generalized) derivatives or derivative-like objects
(with respect to perturbation parameters) of marginal functions, i.e., optimal-valued

ix


functions, or optimal solutions are computed or estimated. Recently, many stability
investigations in optimization-related problems include also variational convergence of
solutions of approximating problems to that of the original problem under consideration. Usually, by variational convergence one means types of convergence which
preserve, to some extent, the so-called variational properties such as being optimal
values, optimal solutions, minimax values, saddle points, etc, and one does not define
exactly the notion “variational convergence”. Hence, variational convergence is important in optimization and related to how to approximate or estimate optimization
problems. In this sense, a study of approximations of a problem can be considered a
study of a kind of stability, but not in the sense that the problem is subject to arbitrary
small perturbations or stochastic perturbations with known probability distributions.
(In such a study, some authors still use “stability” instead of “approximations”.) In
this thesis, we study qualitative stability in terms of approximations in this sense, and
hence we first investigate variational convergence. (Note also that variational convergence is an important part of variational analysis, a modern field including analytic
investigations of optimization problems and mathematical analysis in relation with optimization.) Then, we move on to approximations of various important and typical
models in optimization. We also contribute to conceptional development in this topic
when proposing new definitions of epi/hypo convergence and lopsided convergence of
bifunctions on nonrectangular domains and apply them to dealing with approximations

of quasivariational problems, i.e., constrained optimization-related problems with constraints depending on the decision variables. To the best of our knowledge, our results
on variational convergence of bifunctions on nonrectangular domain are the first ones
of the topic in the literature.
Note that the aforementioned results for nonsmooth optimization are derivativefree. But, our remaining results in the second part of the thesis are in terms of generalized derivatives. The second contribution to stability research is devoted to sensitivity
analysis in the typical sense of computing approximately derivatives of solution maps
or estimating bounds for these derivatives. Regarding optimization models, we consider a general set-valued vector problems and their Pareto and weak solutions. Since

x


higher-order considerations for sensitivity, like for optimality conditions and many other
topics in optimization, are of great importance, we aim to deal with this subject in
the present thesis. Our tools of generalized derivatives are different from the existing
results on the subject. Namely, we propose a notion of higher-order radial-contingent
derivative and develop some calculus rules. This kind of derivative of set-valued maps
combines the ideas of the well-known (higher-order) contingent derivative (see [60]) and
the radial derivatives, which were developed and successfully used recently in establishing optimality conditions in [60, 2]. This combination makes the radial-contingent
derivative bigger than the contingent-type derivatives (in the sense of the comparison
of set-valued maps) and hence leads to better results in research on optimality conditions and sensitivity analysis. Furthermore, unlike the radial derivative which captures
global properties of a map, the radial-contingent derivative reflects local features of a
map, and is more suitable in research. While the radial-contingent derivative appears
mainly in our assumptions, the conclusions of our results are in terms of contingent-type
derivatives. This derivative is different from the well-known (higher-order) contingent
derivative and has also appeared in the literature under the name “upper Studniarski
derivative”.
The third contribution of the thesis includes new results on optimality conditions.
The study of nonsmooth optimality conditions is one of the strongest points of our
Group of Optimization in Southern Vietnam. Many senior colleagues of mine have
already got significant results in this field. Hence, I choose a relatively specific issue:
optimality conditions in a minimax problem with relaxed quasiconvex data. Our minimax problem is not a general minimax model, but is simply a special case where the

maximization is with respect to a finite index set. (However, we have made contributions to a research area with a very small number of papers in the literature.) Note
that quasiconvex problems constitute the most important class of generalized convex
problems, since such problems are often met in practice and we still can apply the
powerful tools of convex analysis in suitable ways to study them. It is also understood
that here the problem data are nonsmooth and thanks to convex analysis one can
employ particular kinds of generalized derivatives and subdifferentials designed specif-

xi


ically for quasiconvex functions. We are relatively successful in reasonably employing
the derivative-like objects to get, for minimax problems, new results which even improve the recent existing ones when applied to the special case of minimization problem
(i.e., the case where the aforementioned index set is a singleton).
The layout of the thesis is as follows. We begin, in Chapter 1, with several basic
concepts, definitions and preliminary facts for our later use. In the next three chapters
we deal with qualitative stability in the sense of approximations via variational convergence. Namely, first, Chapter 2 contains the results of paper [D2] of the author and
focuses on studies of variational convergence of finite-valued bifunctions because many
optimization-related problems can be reformulated as finding minsup points and/or
maxinf points of such bifunctions. Here we discuss basic definitions and characterizations of epi/hypo convergence and lopsided convergence. The content of Chapter 3
is taken from paper [D3] which is devoted to variational properties of these kinds of
variational convergence and applications to approximations of optimization problems.
In Chapter 4, we develop new notions of variational convergence of bifunctions on nonrectangular domains and its variational properties with applications to quasivariational
problems. The results are selected from those of paper [D5]. Chapter 5 begins studies
with the aid of generalized derivatives following paper [D4]. Here we consider sensitivity analysis for nonsmooth set-valued vector optimization with set constraints. Namely,
we compute approximately contingent-type derivatives of perturbation maps and weak
perturbation maps of parametric problems under this setting. Finally, Chapter 6 includes the results of paper [D1] about optimality conditions for relaxed quasiconvex
minimax problems. Here we apply various kinds of cone subdifferentials of relaxed
quasiconvex functions based on convex analysis tools.

xii

Chapter 1
Basic notations and preliminary
facts
In this thesis, we generally use only standard notations. The notations N, Rn , Rn+ ,
¯ stand for the set of the natural numbers, an n-dimensional normed space, its
and R
¯ := R ∪ {−∞, ∞}, respectively (resp).
positive orthant, and the extended-real-line R
X, Y, Z denote normed spaces. (Rn )∗ and X ∗ stand for the dual/conjugate space of Rn
and X, resp, and x∗ , x is the value of the linear function x∗ ∈ X ∗ at x ∈ X. For a
subset A ⊂ Rn , intA, clA, and bdA stand for the interior, closure and boundary, resp,
of A. riA denotes the relative interior of A, i.e., the interior of A in the affine hull of A
defined by affA := {(1 − λ)a1 + λa2 | λ ∈ R, a1 , a2 ∈ A}. coneA stands for the conical
hull of A (called also the cone generated by A), i.e., coneA := {λx|x ∈ A, λ ∈ IR+ }. A
convex set B is called a base of a convex cone C if 0 ∈ clB and C = {tb|t ∈ R+ , b ∈ B}.
Clearly C has a compact base B if and only if C ∩ bdB is compact. The distance from
x ∈ X to A is dist(x, A) := inf{ x − y |y ∈ A}. The normal cone at x to A, denoted
by N (A, x), is defined by
N (A, x) := {x∗ ∈ X ∗ |∀u ∈ A, x∗ , u − x ≤ 0}.
If x ∈ clA, we adopt that N (A, x) = ∅. The contingent cone of A at x ∈ X, denoted
by T (A, x) and also TA (x), is the following cone
T (A, x) := {v ∈ X|∃(rn )

0, ∃(vn ) → v, ∀n, x + rn vn ∈ A}.
1


To see relationships between N (A, x) and T (A, x), recall that the polar cones of cones

B ⊂ X and D ⊂ X ∗ are
B − := {x∗ ∈ X ∗ |∀x ∈ B, x∗ , x ≤ 0},
D− := {x ∈ X|∀x∗ ∈ D, x∗ , x ≤ 0}.
Clearly N (A, x) = [clcone(A−x)]− . Setting, in the definition of T (A, x), xn = x+rn vn ,
we see that
T (A, x) = {v : ∃(rn ) → 0, ∃(xn ) ⊂ A → x, lim

xn − x
= v} ⊂ clcone(A − x).
rn

Hence, T (A, x)− ⊃ N (A, x). Furthermore, if v ∈ T (A, x), i.e., v is of the form lim
xn −x
,
rn

and x∗ ∈ N (A, x), then x∗ , v ≤ 0. Therefore, T (A, x) ⊂ N (A, x)− . Moreover,

if A is convex then the above containments become equalities. B(x, r) is the open ball
of radius r and centered at x. For a point x¯ ∈ X, U(¯
x) designates the collection of the
neighborhoods of x¯. α

α
¯ (α

α
¯ , resp) means α > α
¯ (α < α
¯ , resp) and tends to α

¯.

¯ its domain, epigraph, hypograph, and graph are
For a function ψ : Rn → R,
defined by domψ := {x ∈ Rn | ψ(x) < ∞}, epiψ := {(x, r) ∈ Rn × R| ψ(x) ≤ r},
hypoψ := {(x, r) ∈ Rn × R| ψ(x) ≥ r}, gphψ := {(x, r) ∈ Rn × R| ψ(x) = r}, resp.
liminfψ and limsupψ designate the lower and upper limits of ψ as x tends to x¯, defined,
resp, by
liminf x→¯x ψ(x) := limδ

0 [inf x∈B(¯
x,δ) ψ(x)]

= supδ>0 [inf x∈B(¯x,δ) ψ(x)],

limsupx→¯x ψ(x) := limδ

0 [supx∈B(¯
x,δ) ψ(x)]

= inf δ>0 [supx∈B(¯x,δ) ψ(x)].

We adopt the notation

argminA ψ(x) :=



{x ∈ Rn | ψ(x) = inf A ψ(x)} if infA ψ(x) < ∞,


∅

if infA ψ(x) = ∞,

¯ is said to be lower semiconand similarly for argmax. A function ψ : Rn → R
tinuous (lsc) at x¯ if liminfx→¯x ψ(x) ≥ ψ(¯
x), and upper semicontinuous (usc) at x¯ if
limsupx→¯x ψ(x) ≤ ψ(¯
x)). It is known that ψ is lsc (usc) at x¯ if and only if the epigraph
(hypograpgh, resp) of ψ is closed at (¯
x, ψ(¯
x)). Hence, a lower semicontinuous function
2


is called also a (lower) closed (a similar terminology is used for a usc function). Note
that in Chapters 2-4 and 6 we usually use f , g, h, ψ to denote numerical functions, and
K, F , G to denote numerical bifunctions (i.e., functions of two variables) according
to the traditions of publications of the topic in the literature. However, in Chapter 5,
F , G, H stand for set-valued maps. We are sure that these differences do not cause
confusions in reading the thesis.
For a sequence of subsets {Aν }ν∈N in Rn , the lower/inner limit and upper/outer
limit are defined by
Liminf ν Aν := {x ∈ Rn | ∃xν → x with xν ∈ Aν },
Limsupν Aν := {x ∈ Rn | ∃νl (a subsequence), ∃xνl → x with xνl ∈ Aνl }.
If Liminfν Aν = Limsupν Aν , one says that Aν tends to A or A = Limν Aν (in the
Painlev´e-Kuratowski sense).
For a set-valued map H : X ⇒ Y , the domain, graph, and epigraph of H are
defined by, resp,
domH := {x ∈ X|H(x) = ∅}, gphH := {(x, y) ∈ X × Y |y ∈ H(x)},

epiH := {(x, y) ∈ X × Y |y ∈ H(x) + C},
where C is the (partial) ordering cone of Y . The profile map (known also as the
epigraphical map) of H is H + C (defined by (H + C)(x) := H(x) + C). Recall
concepts of optimality/efficiency in vector optimization, for a0 ∈ A ⊂ Y .
(i) a0 is called a local (Pareto) minimal/efficient point of A (with respect to C), and
denoted by a0 ∈ MinC A, if there exists U ∈ U(a0 ) such that
(A ∩ U − a0 ) ∩ (−C \ C) = ∅.
(ii) Supposing that intC = ∅, a0 is said to be a local weak minimal/efficient point of
A, denoted by a ∈ WMinC A, if there exists U ∈ U(a0 ) such that
(A ∩ U − a0 ) ∩ (−intC) = ∅.
(iii) Assuming that C is pointed, a0 is termed a Henig-proper minimal/efficient point
of A, denoted by a0 ∈ HeC A, if there exist a convex cone K
3

Y with C \ {0} ⊂


intK and U ∈ U(a0 ) such that
(A ∩ U − a0 ) ∩ (−K) = {0}.
(iv) Let Q ⊂ Y be a nonempty open cone, different from Y . One calls a0 a local Qminimal/efficient point of A (see [38]), denoted by a0 ∈ QMinC A, if there exists
U ∈ U(a0 ) such that
(A ∩ U − a0 ) ∩ (−Q) = ∅.
If U = Y , the word “local” is omitted, i.e., we have the corresponding global notions.
Note that the notion of Q-minimal solutions contains as special cases many kinds of
solutions in vector optimization (see [3, 38]). We mention the concept of Henig-proper
efficiency above only as an example for many other definitions of properness in vector
optimization. For comprehensive expositions including comparisons, of these notions,
see [44, 34, 61].
Recall now the two kinds of higher-order derivatives which we are most concerned
with in the sequel. Let F : X ⇒ Y , u ∈ X, m ∈ N, and (x0 , y0 ) ∈ gphF .

(i) ([78]) The mth-order contingent-type derivative of F at (x0 , y0 ) is defined by
Dm F (x0 , y0 )(u) := {v ∈ Y | ∃tn

0, ∃(un , vn ) → (u, v), y0 + tm
n vn ∈ F (x0 + tn un )}.

−1
Setting (xn , yn ) := (x0 + tn un , y0 + tm
n vn ) and γn = tn , we have

Dm F (x0 , y0 )(u) := {v ∈ Y | ∃γn > 0, ∃(xn , yn ) ∈ gphF : (xn , yn ) → (x0 , y0 ),
(γn (xn − x0 ), γnm (yn − y0 )) → (u, v)}.
m
(ii) ([3]) The mth-order radial derivative of F at (x0 , y0 ) is DR
F (x0 , y0 ) defined by
m
DR
F (x0 , y0 )(u) := {v ∈ Y | ∃tn > 0 , ∃(un , vn ) → (u, v), y0 + tm
n vn ∈ F (x0 + tn un )}.
−1
Setting (xn , yn ) := (x0 + tn un , y0 + tm
n vn ) and γn = tn , we have
m
DR
F (x0 , y0 )(u) = {v ∈ Y | ∃γn > 0, ∃(xn , yn ) ∈ gphF, (γn (xn −x0 ), γnm (yn −y0 )) → (u, v)}.

Note that Dm F (x0 , y0 ) is also known as the upper Studniarski derivative (see, e.g.,
[79]). Since D1 F (x0 , y0 ) is the well-known contingent derivative, we choose the term
4

“contingent-type” to reflect the similarity to the well-known mth-order contingent
derivative of F at (x0 , y0 ) wrt (u1 , v1 ), ..., (um−1 , vm−1 ) ∈ X × Y defined as (see [12],
Chapter 5)
m

D F (x0 , y0 , u1 , v1 , ..., um−1 , vm−1 )(u) := {v ∈ Y | ∃tn

0, ∃(un , vn ) → (u, v),

m−1
y0 + tn v1 + · · · + tm−1
vm−1 + tm
um−1 + tm
n
n vn ∈ F (x0 + tn u1 + · · · + tn
n un )}.
1

Of course, D F (x0 , y0 ) = D1 F (x0 , y0 ). An important geometric feature of the mthorder contingent derivative of F at (x0 , y0 ) ∈ gphF is that its graph is the mth-order
contingent set of gphF at (x0 , y0 ) (see [12], Chapter 5). For the sake of simplicity,
recall only for the first order that the contingent cone of A ⊂ X at x0 ∈ clA is
TA (x0 ) := {u ∈ X| ∃tn

0, ∃un → u, x0 + tn un ∈ A}.

1

Then, gphD F (x0 , y0 ) = TgphF (x0 , y0 ).
m

In [3], DR
F (x0 , y0 ) is called the mth-order outer radial derivative. There are the

corresponding lower/inner objects obtained by replacing “∃tn , ∃un ” by “∀tn , ∀un ”.
Since we consider only the upper/outer objects, we omit these adjectives.
We recall now the definitions of subdifferentials, needed in Chapter 6. The lower
subdifferential or Plastria subdifferential in [67] is defined by
∂ < f (¯
x) := x∗ ∈ X ∗ |∀x ∈ L<
x), f (x) − f (¯
x) ≥ x∗ , x − x¯
f (¯

.

The infradifferential or Guti´errez subdifferential in [36] is
∂ ≤ f (¯
x) := {x∗ ∈ X ∗ |∀x ∈ Lf (¯
x), f (x) − f (¯
x) ≥ x∗ , x − x¯ } .
The Greenberg-Pierskalla subdifferential in [33], which is akin to the normal cone, is
defined by
∂ ∗ f (¯
x) := x∗ ∈ X ∗ |∀x ∈ L<
x), x∗ , x − x¯ < 0 .
f (¯
So, we say that it is a kind of normal-cone subdifferentials. The star subdifferentials
defined in [18, 66] is the following normal-cone subdifferentials
∂ ν f (¯
x) := N (Lf (¯

x), x¯),
5


∂ f (¯
x) := N (L<
x), x¯).
f (¯
The adjusted sublevel set of f at x¯ [13] is
Laf (¯
x) = Lf (¯
x) ∩ clB(L<
x), ρx¯ )
f (¯
x) = Lf (¯
x) otherwise, where B(A, ρ) :=
if x¯ is not a global minimizer of f and Laf (¯
{x ∈ X| dist(x, A) < ρ} and ρx := dist(x, L<
f (x)). The adjusted subdifferential in [13]
is
∂ a f (¯
x) := N (Laf (¯
x), x¯).
It is obvious that
∂ < f (¯
x) ⊂ ∂ ∗ f (¯
x) ⊂ ∂ f (¯
x),
∂ ≤ f (¯
x) ⊂ ∂ ν f (¯

x) ⊂ ∂ a f (¯
x) ⊂ ∂ f (¯
x).
For details about the calculus of these subdifferentials the reader is referred to [13,65].
Although they are defined for arbitrary functions (finite at x¯), they possess good properties only under additional conditions. In the literature the sublevel sets are usually
assumed to be convex, i.e., the functions are quasiconvex. In this paper, like [45], we
relax remarkably this assumption to the convexity only at x¯.
By the above chains of inclusions of the subdifferentials, it is clear that IR+ ∂ < f (¯
x) ⊂
∂ f (¯
x) and IR+ ∂ ≤ f (¯
x) ⊂ ∂ ν f (¯
x). Therefore, the following definitions are natural. A
function f is said to be a Plastria function at x¯ if its strict sublevel set L<
x) is convex
f (¯
and
IR+ ∂ < f (¯
x) = ∂ f (¯
x),
and to be a Guti´errez function at x¯ if Lf (¯
x) is convex and
IR+ ∂ ≤ f (¯
x) = ∂ ν f (¯
x).
In Chapter 6, we consider optimality conditions for the following minimax problem
(P)

minx∈X max1≤i≤k fi (x),
gj (x) ≤ 0, j = 1, ...m,

¯ gj : X → R
¯ for i ∈ I := {1, ..., k} and
where X is a normed space, fi : X → R,
j ∈ J := {1, ...m}.
6


Chapter 2
Variational convergence of
finite-valued bifunctions
2.1

Introduction

For dealing with approximations of optimization problems in terms of variational convergence in the first part of the thesis about stability, in this first chapter we study
variational convergence. Namely, we recall main facts about epi-convergence of univariate functions (unifunctions or simply functions), the principal variational convergence
and develop a theory of epi/hypo convergence of finite-valued bivariate functions (bifunctions). The former was introduced independently in [85, 86] and [84]. For details
of this convergence and applications, see the books [5, 12, 72]. The latter was proposed
in [8] and rigorously treated in [11]. In [9] a modified and stronger form than epi/hypo
convergence, called lopsided convergence (lop-convergence) was given its airing. All
the aforementioned works dealt with extended real-valued functions or bifunctions. In
[42], lop-convergence was not considered for extended-real-valued bifunctions, whose
class is denoted by biv(Rn × Rm ), but for finite-valued bifunctions defined on subsets
of the form C × D ⊂ Rn × Rm , denoted now by fv-biv(Rn × Rm ). The motivation
is that important bifunctions met in applications like Lagrangians in constraint optimization, payoff functions in zero-sum games, or Hamiltonians in variational calculus
and optimal control are finite-valued bifunctions defined on product sets. For detailed
7

applications, see [72, 83] for epi-convergence and [42, 43, 57, 59] for lop-convergence.
Being stronger than epi/hypo convergence, lop-convergence enjoys many good variational properties. However, this kind of convergence is one-sided and one is interested
in the convergence of either minsup points or maxinf points, but not in both. But,
a variational convergence notion for bifunctions should be aimed first at the convergence of saddle points, which correspond to not only minsup points but also maxinf
points. So, it seems that we had better not make a distinction between minsup and
maxinf problems in many cases, especially in considerations related to duality or dual
problems. Hence, in this chapter we revisit epi/hypo convergence. Note that applications of epi/hypo convergence were discussed in [10, 15, 75-77, 87], but only for
extended real-valued bifunctions. Because of the importance of finite-valued bifunctions in applications, our main concern will be with the class fv-biv(Rn × Rm ). In
Section 2.2, we recall basic facts about epi-convergence and develop Legendre-Fenchel
conjugates and skew-conjugates of finite-valued unifunctions. The next three sections
are devoted to three full characterizations of epi/hypo convergence of bifunctions in
fv-biv(Rn × Rm ), with some comparison with lop-convergence. These characterizations
are sometimes complicated, mainly because limits are not unique, but form entire sets
of bifunctions called equivalence classes (for explanations of these classes, see Theorem
2.3.1 and Remark 2.3.3). Via these characterizations, we will see that some properties
such as closedness, convexity-concavity, or relations between epi/hypo convergence and
lop-convergence are affected by this non-uniqueness and we should be careful when arguing on these properties. Not paying attention to the non-uniqueness and equivalence
classes may lead to mistakes in dealing with the mentioned convergence, as can be
spotted in several existing papers.
Full characterizations of epi/hypo convergence are important not only for understanding the essence of this notion, but also for applications because we can use them
as both the strongest necessary conditions and the weakest sufficient conditions to
consider if a given sequence of bifunctions epi/hypo converges or not. Of course, variational properties of epi/hypo convergence are even more often needed. We will discuss
this concern in Chapter 3.

8


2.2

Epi-convergence

In this section we first recall basic facts about epi-convergence in the class fv-fcn(Rn )
of the finite-valued unifunctions. Then, we develop the Legendre-Fenchel conjugation
and skew-conjugation for elements of fv-fcn(Rn ), in the same way as Rockafellar [69]
studied for fcn(Rn ), i.e., the class of the extended real-valued unifunctions.

2.2.1

Epi-convergence

In [42], the epi-convergence defined in [84-86] for the class fcn(Rn ) was adjusted for
finite-valued unifunctions as follows.
Definition 2.2.1 (epi-convergence, [42]) A sequence {f ν : C ν → R}ν∈N is called to
e

epi-converge to f : C → R, denoted by f ν → f or f = e-limν f ν , if
(a) for all xν ∈ C ν → x, liminfν f ν (xν ) ≥ f (x) when x ∈ C and f ν (xν ) → ∞ when
x∈
/ C;
(b) for all x ∈ C, there exists xν ∈ C ν → x such that limsupν f ν (xν ) ≤ f (x).
e

Note first that irrespective of concerning fv-fcn(Rn ) or fcn(Rn ), f ν → f if and
P −K

only if epif ν −→ epif in the set convergence sense of Painlev´e-Kuratowski. This also
applies to following upper and lower epi-limits defined in terms of epigraphs.
Definition 2.2.2 (lower and upper epi-limits) Let f ν , f ∈ fv-fcn(Rn ).
(i) f is the lower epi-limit of the functions f ν , denoted by f = e-liν f ν , if epif is the
outer set-limit of epif ν .

(ii) f is the upper epi-limit of the functions f ν , denoted by f = e-lsν f ν , if epif is
the inner set-limit of epif ν .
Of course, e-liν f ν ≤ e-lsν f ν and the epi-limit exists if and only if e-liν f ν = e-lsν f ν =
e-limν f ν .
Proposition 2.2.1 (properties of epi-limits) The lower and upper epi-limits and the
epi-limit if it exists, of a sequence {f ν }ν∈N in fv-fcn(Rn ) or fcn(Rn ) are lsc. Moreover,
if the functions f ν are convex, so is the upper epi-limit and also the epi-limit if it
9


exists. In particular, this implies that the collection of lsc convex functions is closed
under epi-convergence.
We can identify fv-fcn(Rn ) with a subclass of the so-called proper functions of
fcn(Rn ), denoted by pfcn(Rn ), as follows.

A bijection η between fv-fcn(Rn ) and

pfcn(Rn ) is defined by
(ηf )(x) =



f (x) if x ∈ C,

∞

otherwise.

e

e

Then, epif = epi(ηf ). Hence, f ν → f if and only if ηf ν → ηf . Hypo-convergence
e

is defined “symmetrically”: f ν is said to hypo-converge to f if −f ν → −f . Then,
all the aforementioned facts for epi-convergence are rephrased for hypo-convergence by
replacing epi, liminf, limsup, inf, argmin, lsc, ≤, etc, resp, by hypo, limsup, liminf, sup,
argmax, usc, ≥, etc.

2.2.2

Legendre-Fenchel transform and its continuity

In this subsection we develop Legendre-Fenchel conjugation and skew-conjugation for
fv-fcn(Rn ). First recall the Legendre-Fenchel conjugate (known also as Fenchel or
Young-Fenchel conjugate; the word “conjugate” is also often replaced by “transform”)
of f ∈ fcn(Rn ) is (see [30]), for u ∈ (Rn )∗ ,
f ∗ (u) := supx∈Rn { u, x − f (x)} = supx∈domf { u, x − f (x)}.
In the special case where f has a one-to-one gradient

f , the Legendre-Fenchel trans-

form comes down to the classical Legendre transform of the calculus of variations:
f ∗ (u) = u, ( f )−1 u − f (( f )−1 u).
The skew-conjugate of a convex function f ∈ fcn(Rn ) is defined in [69] as
g(u) := inf x∈Rn {f (x) − x, u } = −f ∗ (u).
Then, g is a (upper) closed (i.e., usc) concave function and
(clf )(x) = supu∈Rn {g(u) + u, x }.
10

(2.1)


Starting from a concave function g ∈ fcn(Rn ), its skew-conjugate is a (lower) closed
(i.e., lsc) convex function f ∈ fcn(Rn ) defined in [69] by
f (x) := supu∈Rn {g(u) + u, x } = (−g)∗ (x).

(2.2)

Clearly the skew-conjugate of f is in turn clg. Thus, (2.1) and (2.2) set up an one-to-one
correspondence between the closed convex functions and the closed concave functions,
all in fcn(Rn ).
Consider now a closed convex function f ∈ fv-fcn(Rn ). We compute
fˆ∗ (u) := supx∈Rn { u, x − (ηf )(x)} = (ηf )∗ .
We define the Legendre-Fenchel conjugate of f as
f ∗ := fˆ∗ |domfˆ∗ = (ηf )∗ |dom(ηf )∗
with domf ∗ := domfˆ∗ = dom(ηf )∗ . For the closed convex function ηf ∈ fcn(Rn ) it is
known that
(ηf )(x) = supu∈Rn { x, u − (ηf )∗ (u)}
= supu∈domf ∗ { x, u − f ∗ (u)}.
Since ηf |dom(ηf ) = f , with the convention that when taking a conjugation we always
restrict the resulting function on its domain to obtain a function of fv-fcn(Rn ), we
see a one-to-one correspondence between f and f ∗ . For the closed convex elements
of fv-fcn(Rn ) the Legendre-Fenchel transform is also bicontinuous with respect to epiconvergence. Indeed, since epif = epi(ηf ) for any f ∈ fv-fcn(Rn ), applying [68, 69] to
the involved proper functions, one has
e

e

e

e

[f ν → f ] ⇔ [ηf ν → ηf ] ⇔ [(ηf ν )∗ → (ηf )∗ ] ⇔ [(f ν )∗ → f ∗ ].
Now we move on to defining the skew-conjugate of a closed convex function f ∈
fv-fcn(Rn ). We see that
gˆ(u) := inf x∈Cf {f (x) − x, u }
= inf x∈Rn {(ηf )(x) − x, u } = −(ηf )∗ (u).
11