Subdifferential calculus for convex functions

Chapter 4. Subdifferential calculus for convex functions
Chapter 4.
Subdifferential calculus for convex functions
tvnguyen (University of Science) Convex Optimization 66 / 108
Chapter 4. Subdifferential calculus for convex functions
Directional derivative
Definition. Let f : IR
n
→ IR ∪ {+∞}, x ∈ domf and d ∈ IR
n
. The
directional derivative of f at x in the direction d is
f

(x, d) = lim
t↓0
f (x + td) − f (x)
t
if the limit exists (-∞ and ∞ are allowed)
Proposition. Let f : IR
n
→ IR ∪ {+∞} be a convex function, and let
x ∈ dom f . For each d, the difference quotient in the definition of
f

(x, d) is a non-decreasing function of t > 0, so that
f

(x, d) = inf
t>0
f (x + t d) − f (x)

t
If f is differentiable at x, then f

(x, d) = ∇f (x), d.
tvnguyen (University of Science) Convex Optimization 67 / 108
Chapter 4. Subdifferential calculus for convex functions
Subdifferential. Motivation
To obtain the simplest possible dual representation of a closed convex set
C, we have introduced the notion of supporting hyperplane.
When taking C the epigraph of a closed convex proper function f , the
corresponding notion is the exact minoration : an affine function
l : IR
n
→ IR is an exact minorant of f at x if l ≤ f and l(x) = f (x)
Setting l(y) = x
∗
, y + α, this becomes
f (y) ≥ x
∗
, y + α ∀y ∈ IR
n
, and f (x) = x
∗
, x + α
which is equivalent to (α = f (x) − x
∗
, x)
∀y ∈ IR
n
f (y) ≥ f (x) + x

∗
, y − x
This leads to the following definition.
tvnguyen (University of Science) Convex Optimization 68 / 108
Chapter 4. Subdifferential calculus for convex functions
Subgradient. Subdifferential
Definition. Let f : IR
n
→ IR ∪ {+∞} be a convex function. A vector
x
∗
is said to be a subgradient of f at x if
∀y ∈ IR
n
f (y) ≥ f (x) + x
∗
, y − x.
The set of subgradients of f at x is called the subdifferential of f at x
and is denoted by ∂f (x).
Proposition. The subdifferential ∂f (x) is a closed convex set. It may
be empty.
tvnguyen (University of Science) Convex Optimization 69 / 108
Chapter 4. Subdifferential calculus for convex functions
Examples
f (x) = |x|
∂f (0) = [−1, 1], ∂f (x) = {1} if x > 0, ∂f (x) = −1 if x < 0
f (x) = e
x
− 1 if x ≥ 0 and 0 if x < 0
∂f (0) = [0, 1], ∂f (x) = {e

x
} if x > 0, ∂f (x) = 0 if x < 0
x
|x|
x
e
x
− 1
0
tvnguyen (University of Science) Convex Optimization 70 / 108
Chapter 4. Subdifferential calculus for convex functions
Proposition. Let f : IR
n
→ IR ∪ {+∞} be a convex function and let
x ∈ dom f . Then x
∗
is a subgradient of f at x if and only if
f

(x, d) ≥ x
∗
, d, ∀d.
In fact, the closure of f

(x, d) as a convex function of d is the support
function of the closed convex set ∂f (x).
Proposition. Let f : IR
n
→ IR ∪ {+∞} be a convex function. Then
∂f (x) is empty when x ∈ domf , and nonempty when x ∈ ri(domf ). In

that case f

(x, y) is closed and proper as a function of y , and
f

(x, d) = sup{< x
∗
, d > |x
∗
∈ ∂f (x)} = δ
∗
(d|∂f (x)).
Finally, ∂f (x) is a non-empty bounded set if and only if x ∈ int(domf ),
in which case f

(x, d) is finite for every d.
tvnguyen (University of Science) Convex Optimization 71 / 108