Đánh giá độ phúc tạp : Dynamic programming 01

Dynamic programming
From the D&C Theorem, we can see that a recursive algorithm is likely to be polynomial if
the sum of the sizes of the subproblems is bounded by kn. (Using the variables of that
Theorem, k = a/c). If, however, the obvious division of a paroblem of size n results in n
problems of size n-1 then the recursive algorithm is likely to have exponential growth.
Dynamic programming can be thought of as being the reverse of recursion. Recursion is a
top-down mechanism -- we take a problem, split it up, and solve the smaller problems that
are created. Dynamic programming is a bottom-up mechanism -- we solve all possible
small problems and then combine them to obtain solutions for bigger problems.
The reason that this may be better is that, using recursion, it is possible that we may solve
the same small subproblem many times. Using dynamic programming, we solve it once.
Evaluation of the product of n matrices
We wish to determine the value of the product ∏
i = 1 to n
M
i
, where M
i
has r
i-1
rows and r
i
columns.
The order of which matrices are multiplied together first can significantly affect the time
required.
To multiply M×N, where matrix M is p×q and matrix N is q×r, takes pqr operations if we
use the "normal" matrix multiplication algorithm. Note that the matrices have a common
dimension value of q. This makes the matrices have the property of compatibility, without
which it would not be possible for them to be multiplied. Also note that, while matrix
multiplication is associative, matrix multiplication is not commutative. That is, N×M
might not equal M×N and, in fact, N×M might not even be defined because of a lack of

compatibility.
EXAMPLE:
Calculate M = M
1
× M
2
× M
3
× M
4
,
where the dimensions of the matrices are
M
1
: 10,20 M
2
: 20,50 M
3
: 50,1 M
4
: 1,100
Calculating M = M
1
× ( M
2
× ( M
3
× M
4
) ) requires 125000 operations.

Calculating M = ( M
1
× ( M
2
× M
3
) ) × M
4
requires 2200 operations.
We could figure out how many operations each possible order will take and then use the
one having the minimum number of operations, but there are there are an exponential
number of orderings. Any of the n-1 multiplications could be first and then any of the
remaining n-2 multiplications could be next and so on, leading to a total of (n-1)!
orderings.
We can find the best order in time O(n
3
) by using dynamic programming.
If m
i ,j
is the minimum cost of evaluating the product M
i
× ... × M
j
then:
m
i, j
= 0, if i = j, and
m
i, j
= MIN

i ≤ k < j
{ m
i,k
+ m
k+1,j
+ r
i-1
r
k
r
j
}, if i < j.
The algorithm:
for i := 1 to n do
m
i,i
:= 0
for length := 1 to n-1 do
for i := 1 to n-length do
j := i + length
m
i,j
= MIN
i≤k<j
{m
i,k
+ m
k+1,j
+ r
i-1

r
k
r
j
}
In the above listing, length refers to the number of matrix multiplications in a subproblem.
An alternative approach would be to use size (equal to length+1) as the number of
matrices in the subproblem.
For the example given above, we would calculate:
m
1,1
= 0, m
2,2
= 0, m
3,3
= 0, m
4,4
= 0
m
1,2
= 10000, m
2,3
= 1000, m
3,4
= 5000
m
1,3
= 1200, m
2,4
= 3000

m
1,4
= 2200
Longest common substring problem
Given two sequences of letters, such as A = HELLO and B = ALOHA,
find the longest contiguous sequence appearing in both.
One solution: (assume strings have lengths m and n)
For each of the m starting points of A, check for the longest common string starting at each
of the n starting points of B.
The checks could average Θ(m) time → a total of Θ(m
2
n) time.
Dynamic programming solution:
Let L
i, j
= maximum length of common strings that end at A[i] & B[j]. Then,
A[i] = B[j] → L
i, j
= 1 + L
i-1, j-1

A[i] ≠ B[j] → L
i, j
= 0
LONGEST COMMON SUBSTRING(A,m,B,n)
for i := 0 to m do L
i,0
:= 0
for j := 0 to n do L
0,j

:= 0
len := 0
answer := <0,0>
for i := 1 to m do
for j := 1 to n do
if A
i
≠ B
j
then
L
i,j
:= 0
else
L
i,j
:= 1 + L
i-1,j-1
if L
i,j
> len then
len := L
i,j
answer = <i,j>
Example:
A L O H A
H 0 0 0 1 0
E 0 0 0 0 0
L 0 1 0 0 0
L 0 1 0 0 0

O 0 0 2 0 0
Longest common subsequence
String C is a subsequence of string A if string C can be obtained by deleting 0 or more
symbols from string A.
Example:
houseboat
houseboat
ousbo is a subsequence of houseboat
String C is a common subsequence of strings A and B if C is a subsequence of A and also C
is a subsequence of B.
Example:
houseboat
computer
houseboat
computer
out is a common subsequence of houseboat and computer
String C is a longest common subsequence (LCS) of strings A and B if C is a common
subsequence of A & B and there is no other common subsequence of A & B that has greater
length.
Let L
i, j
be the length of an LCS of A[1...i] & B[1...j], i.e., the prefixes of strings A & B of
lengths i and j.
L
i, j
= L
i-1, j-1
+ 1, if A
i
= B

i

L
i, j
= Max{ L
i-1, j
, L
i, j-1
}, if A
i
≠ B
i

LONGEST COMMON SUBSEQUENCE(A,m,B,n)
for i := 0 to m do L
i,0
:= 0
for j := 0 to n do L
0,j
:= 0
for i := 1 to m do
for j := 1 to n do
if A
i
= B
j
then
L
i,j
:= 1 + L

i-1,j-1
else
L
i,j
:= Max{ L
i-1,j
, L
i,j-1
}
length := L
m,n
Dan Hirschberg
Computer Science
University of California, Irvine, CA 92697-3425
dan at ics.uci.edu
Last modified: Oct 28, 2003