Economic growth and economic development 272

Introduction to Modern Economic Growth
form that the constraint correspondence could take, since it does not depend on
k (t)).

Problem A1, also referred to as the sequence problem, is one of choosing an infinite sequence {x (t)}∞
t=0 from some (vector) space of infinite sequences (for example,
∞
⊂ L∞ , where L∞ is the vector space of infinite sequences that
{x (t)}∞
t=0 ∈ X

are bounded with the k·k∞ norm, which we will denote throughout by the simpler

notation k·k). Sequence problems sometimes have nice features, but their solutions

are often difficult to characterize both analytically and numerically.

The basic idea of dynamic programming is to turn the sequence problem into
a functional equation. That is, it is to transform the problem into one of finding a
function rather than a sequence. The relevant functional equation can be written
as follows:
Problem A2
(6.1)

:

V (x) =

sup [U(x, y) + βV (y)] , for all x ∈ X,

y∈G(x)

where V : X → R is a real-valued function. Intuitively, instead of explicitly choosing

the sequence {x (t)}∞
t=0 , in (6.1), we choose a policy, which determines what the
control vector x (t + 1) should be for a given value of the state vector x (t). Since

instantaneous payoff function U (·, ·) does not depend on time, there is no reason for

this policy to be time-dependent either, and we denote the control vector by y and
the state vector by x. Then the problem can be written as making the right choice
of y for any value of x. Mathematically, this corresponds to maximizing V (x) for
any x ∈ X. The only subtlety in (6.1) is the presence of the V (·) on the right hand

side, which will be explained below. This is also the reason why (6.1) is also called
the recursive formulation–the function V (x) appears both on the left and the right
hand sides of equation (6.1) and is thus defined recursively.

The functional equation in Problem A2 is also called the Bellman equation, after
Richard Bellman, who was the first to introduce the dynamic programming formulation, though this formulation was anticipated by the economist Lloyd Shapley in
his study of stochastic games.
258