Duality for sets and functions

Chapter 3. Duality for sets and functions
Chapter 3.
Duality for sets and functions
tvnguyen (University of Science) Convex Optimization 46 / 108
Chapter 3. Duality for sets and functions
Dual representation of convex sets
Several basic geometrical objects in IR
n
can be described by using linear
forms. For example
A closed hyperplane H can be written
H = {x ∈ IR
n
| p, x = α}
for some p ∈ IR
n
, p = 0, and α ∈ IR.
Similarly, a closed half-space H can be written
H = {x ∈ IR
n
| p, x ≤ α}
We will show that arbitrary closed convex sets in IR
n
can be described by
using only linear forms. This is what we call a dual representation.
This theory is based on the Hahn-Banach theorem.
tvnguyen (University of Science) Convex Optimization 47 / 108
Chapter 3. Duality for sets and functions
Closest point theorem
A well-known geometric fact is that, given a closed convex set A and a
point x ∈ A, there exists a unique point y ∈ A¸ with minimum distance

from x.
Theorem. (Closest Point Theorem) Let A be a nonempty, closed
convex set in IR
n
and x ∈ A. Then, there exists a unique point y ∈ A
with minimum distance from x. Furthermore, y is the minimizing point,
or closest point to x, if and only if x − y, z − y ≤ 0 for all z ∈ A.
A
z
yx
tvnguyen (University of Science) Convex Optimization 48 / 108
Chapter 3. Duality for sets and functions
Separation of convex sets
Almost all optimality conditions and duality relationships use some sort of
separation or support of convex sets.
Definition. (Separation of Sets) Let S1 and S2 be nonempty sets in
IR
n
. A hyperplane H = {x|p, x = α} separates S1 and S2 if p, x ≥ α
for each x ∈ S1 and p, x ≤ α for each x ∈ S2.
If, in addition, p, x ≥ α + ε for each x ∈ S1 and p, x ≤ α for each
x ∈ S2, where ε is a positive scalar, then the hyperplane H is said to
strongly separate the sets S1 and S2.
Notice that strong separation implies separation of sets.
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx

xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxx
Separation
Strong separation
S
1
S
1
S
2
S
2
H
H
tvnguyen (University of Science) Convex Optimization 49 / 108
Chapter 3. Duality for sets and functions
The following is the most fundamental separation theorem.
Theorem. (Separation Theorem) Let A be a nonempty closed convex
set in IR
n
and x ∈ A. Then, there exists a nonzero vector p and a scalar
α such that p, x > α and p, x ≤ α for each z ∈ A.
A

z
yx
Here p = x − y.
tvnguyen (University of Science) Convex Optimization 50 / 108
Chapter 3. Duality for sets and functions
A direct consequence
Proposition. Let A be a nonempty closed convex set in IR
n
. Then A is
equal to the intersection of all closed half-spaces that contain it :
A = ∩ { H
p≤α
| A ⊂ H
p≤α
}
where H
p≤α
= {x ∈ IR
n
| p, x ≤ α}
In such a representation of a closed convex set, it is natural to look for the
simplest representation.
Observe :
(a) α

≥ α and A ⊂ H
p≤α
⇒ A ⊂ H
p≤α


(b) fixing p = 0 and making α vary gives rise to parallel hyperplanes
tvnguyen (University of Science) Convex Optimization 51 / 108
Chapter 3. Duality for sets and functions
Support function
Question. For a given p ∈ IR
n
,p = 0, such that A ⊂ H
p≤α
for some
α ∈ IR, what is the intersection of all the parallel half-spaces containing A.
Proposition. For a given p ∈ IR
n
,p = 0, such that A ⊂ H
p≤α
for some
α ∈ IR, the intersection of all the parallel half-spaces containing A is the
closed half-space H
p≤σ
A
(p)
, where
σ
A
(p) = sup {p, x | x ∈ A}.
Definition. Let A be a nonempty subset of IR
n
. The support function
of A is defined by
σ
A

: IR
n
→ IR ∪ {+∞} σ
A
(p) = sup
x∈A
< p, x >
tvnguyen (University of Science) Convex Optimization 52 / 108
Chapter 3. Duality for sets and functions
Support function
As a direct consequence of the definition we obtain the dual representation
of a closed convex set.
Proposition. Let A be a closed convex set in IR
n
. Then A is
completely determined by its support function, i.e.,
A = {x| p, x ≤ σ
A
(p), ∀p ∈ IR
n
}
Here are some properties of the support function
Proposition. The support function σ
A
: IR
n
→ IR ∪ {+∞} of a closed
convex nonempty subset A is a function which is proper, closed, convex,
and positively homogeneous of degree 1. Its epigraph is a closed convex
cone in IR

n
× IR. Furthermore, dom σ
A
= IR
n
when A is bounded.
tvnguyen (University of Science) Convex Optimization 53 / 108