PROJECTION ONTO NONNEGATIVE ORTHANT, RECTANGULAR BOX AND POLYHEDRON

Projection onto nonnegative orthant, rectangular box and polyhedron

Andersen Ang Content
proj onto nonnegative orthant
ECS, Uni. Southampton, UK proj onto box
alternating projection algorithm
Homepage angms.science active set method
interior point method
Version: April 2, 2023 dual proximal gradient method
First draft: March 19, 2020

Projection onto nonnegative orthant

▶ The Euclidean projection of a given point y ∈ Rn onto a (non-empty & compact) set
S ⊆ Rn, denoted as projS(y), is a function Rn → Rn that output a point xˆ by solving
xˆ = projS(y) = argmin ∥x − y∥2.

x∈S

Such optimization always has a unique solution, details here.

▶ Question: What if S is the nonnegative orthant?
projS(y) = argmin ∥x − y∥2,

x≥0

where x ≥ 0 means x is inside the nonnegative orthant S = x | xi ≥ 0 ∀i .

▶ This problem has a trivial solution xˆ = [y]+, where xˆi = [yi]+ with [ · ]+ = max{ · , 0 }.

2 / 13

[y]+ = argmin ∥x − y∥2

x≥0

Among all the feasible value in the nonnegative orthant, 0 is the closest to negative value.

Example
▶ The orange point y = 2 is projected to the

−2
blue point 2 as the blue point is where the

0
blue circle just touch the nonnegative orthant
(violet region).
▶ Here the radius of the circle (=2) is the optimal
cost value.

3 / 13

Projection onto a box

A generalization of the nonnegativity is the box constraint

PS(y) = argmin ∥x − y∥2,

l≤x≤u



 li yi ≤ li

with the close form solution [PS(y)]i = yi li < yi < ui

ui yi ≥ ui

An example li = 1, ui = 2 and a point at (5, 5).

4 / 13

Projection onto a polyhedron

▶ projnonnegative orthant and projbox are special cases of projpolyhedron
Polyhedron S = x | Ax ≤ b .

where A ∈ Rm×n is full rank, and b ∈ Rm.
▶ Note that it does not matter Ax ≥ b or Ax ≤ b: we can absorb the negative sign into A

or b.

5 / 13

Example 1 : m = n = 2

orange point y = [2, −2]
blue point xˆ = [12/5, −6/5]
Red region x + 2y ≥ 0
Blue region 2x + y ≥ 0
a1 [2, 1]
a2 [1, 2]

A= 2 1 b= 0

12 0

▶ y fulfill Ax ≥ b for the first row of A, but not the second row.
▶ The optimal cost value = 45 , which is the distance between xˆ and y, and it is also the

radius of the blue circle.

▶ We can see in this example that, proj to nonnegative orthant is simply the vector a1, a2

are rotated to the direction of the standard basis.

6 / 13

Example 2 : m = 3, n = 2

orange point y = [2, −2]
blue point xˆ = [2, −1]
Red region x + 2y ≥ 0
Blue region 2x + y ≥ 0
Green region x−y ≤ 3

 2 1 0
A =  1 2 b= 0 

−1 1 −3

▶ The boundary of the feasible region is highlighted in yellow.

7 / 13

Solving projection onto polyhedron

▶ Unlike the projection onto nonnegative orthant or projection onto a box, the problem

argmin ∥x − y∥2,

Ax≤b

has no simple close form solution in general.

▶ To solve it, we use an iterative optimization algorithms, such as
1. Alternating projection algorithm
2. Active set methods
3. Interior point methods
4. (Dual) Proximal gradient method
5. Douglas-Rachford splitting algorithm on the dual

We talk very briefly on the first four approaches.

8 / 13

Alternating projection algorithm

▶ The constraint Ax ≤ b describe a (non-empty) polyhedron as an intersection of m
inequality constraint, each of the constraint is a problem pf projection onto halfspace

argmin 1 ∥x − y∥22,
⟨a,x⟩≤b 2

with the close form solution as

 y ⟨a, x⟩ ≤ b
⟨a, x⟩ > b




xˆ = ⟨a, x⟩ − b
y − 2a
∥a∥2

▶ Alternating projection algorithm: cycle through the projection onto halfspace from i = 1
to r and repeats a few times gives the projection onto Ax ≤ b.

▶ Drawback of this approach: possibly slow if m is large.

9 / 13

Active set

▶ Assume we know a set I : {i ∈ [m] | ⟨ai, x⟩ = bi}1 that at the solution the equality is

satisfied. Let AI be the submatrix of A with rows in I. Then we can solve the following

problem instead argmin 1 ∥x − y∥22 s.t. AIx = bI.
x2

The optimality conditions can be expressed as a matrix-vector equation as

I A⊤I x= 0
AI 0 z bI

▶ The idea of active set method is to construct a set J to approximate I. In each iteration,
it solves the above equation, and correct J . Eventually J approach to I and the sol. of
the matrix-vector equation gives the sol. of the original problem.

1[n] means the set {1, 2, · · · , n} and ai is the i-th row of A

10 / 13

Interior point method

▶ First, transform Ax ≤ b to Ax + s = b, where s ≥ 0 is a slack variable.

▶ Construct a log barrier on s with parameter µ > 0, and put it into the cost function, we
obtain a typical problem in the interior point method
argmin 1 ∥x − y∥22 − µ log |si| s.t. Ax + s = b.
x2 i
This problem is convex.

▶ Starting from (x0, µ0), compute an approximate solution x, then decrease µ. As µ → 0,
xk → x∗.

▶ Drawback of interior point method: not scalable, not for problem with large size. 11 / 13

(Dual) proximal gradient method

▶ The dual of the problem is

min 1 ∥A⊤z − A⊤y∥22 + ⟨b, z⟩ s.t. z ≥ 0.
z2

The key is that nonnegative constraint is easy to handle, so you keep the nonnegative
constraint.
▶ Proximal update of this problem is simple. See here for details of (unaccelerated) proximal
gradient.
▶ In fact, dual proximal gradient method in this case is exactly the dual projected gradient
method.
▶ Nesterov’s acceleration can be used.
▶ See here for the discussion of dual proximal gradient method, and also how the dual
problem is constructed.

12 / 13

Last page - summary

Discussed: solving projection onto a polyhedron
▶ Polyhedron set : Ax ≤ b, A full rank
▶ Brief overview of solving projection problem by alternating projection algorithm, active set

method, interior point method and dual proximal gradient method.
▶ Projection onto nonnegative orthant and box as special cases with simple close form

solution.

Not discussed
▶ If A not full rank.
▶ Details of the methods.

▶ Solving the projection problem using Douglas-Rachford splitting.

End of document

13 / 13