slide cấu trúc dữ liệu và giải thuật bản đầy đủ (hay) ở file đính kèm

Data structures and Algorithms
Basic definitions and notations
Pham Quang Dung

Hanoi, 2012

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

1 / 46


Outline

1

First example

2

Algorithms and Complexity

3

Big-Oh notation

4

Pseudo code

5

Analysis of algorithms

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

2 / 46


First example

Find the longest subsequence of a given sequence of numbers
Given a sequence s = a1 , . . . , an
a subsequence is s(i, j) = ai , . . . , aj , 1 ≤ i ≤ j ≤ n
weight w (s(i, j)) =
j

ak
k=i

Problem : find the subsequence having largest weight
Example
sequence : -2, 11, -4, 13, -5, 2
The largest weight subsequence is 11, -4, 13 having weight 20

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

3 / 46


Direct algorithm
Scan all possible subsequences

n
2

=

n2 +n
2

Compute and keep the largest weight subsequence
C++ code :

Analyzing the complexity by counting the number of basic operations
n3
6

+

n2
2

+

n

3

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

4 / 46


Direct algorithm
Faster algorithm

j
k=i

Observation :

a[k] = a[j] +

j−1
k=i

a[k]

C++ code :

Complexity :

Pham Quang Dung ()

n2
2

+

n
2

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

5 / 46


Recursive algorithm
Divide the sequence into 2 subsequences at the middle s = s1 :: s2
The largest subsequence might
be in s1 or
be in s2 or
start at some position of s1 and end at some position of s2

C++ code :

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

6 / 46


Recursive algorithm

Count the number of addition (”+”) operation T (n)
T (n) =

0
if n = 1
T ( n2 ) + T ( n2 ) + n if n > 1

By induction : T (n) = n logn

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

7 / 46


Dynamic programming

General principle
Division : divide the initial problem into smaller similar problems
(subproblems)
Storing solutions to subproblems : store the solution to subproblems
into memory
Aggregation : establish the solution to the initial problem by
aggregating solutions to subproblems stored in the memory

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

8 / 46


Dynamic programming

Largest subsequence
Division :
Let si be the weight of the largest subsequence of a1 , . . . , ai ending at
ai

Aggregation :
s1 = a1
si = max{si−1 + ai , ai }, ∀i = 2, . . . , n
Solution to the original problem is max{s1 , . . . , sn }

Number of basic operations is n (best algorithm)

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

9 / 46


Comparison between algorithms

operations
logn
nlogn

n2
n3
operations
logn
nlogn
n2
n3

n = 10
time
n = 100
time
−8
3.32
3.3×10 sec.
6.64
6×10−8 sec.
−7
33.2
3.3×10 sec.
664
6.6×10−6 sec.
100
10−6 sec.
10000
10−4 sec.
3
−5
6
10

10 sec.
10
10−2 sec.
n = 104
time
n = 106
time
−6
13.3
10 sec.
19.9
< 10−5 sec.
5
−3
7
1.33×10
×10 sec. 1.99×10
2×10−1 sec.
108
1 sec.
101 2
2.77 hours
1
10 2
2.7 hours
101 8
115 days

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

10 / 46


Outline

1

First example

2

Algorithms and Complexity

3

Big-Oh notation

4

Pseudo code

5

Analysis of algorithms

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

11 / 46


Algorithms and complexity

Definition
Informally, an algorithm is any well-defined computational procedure that
takes a set of values, as input, and produces a set of values, as output
Input
Output
Precision
Finiteness
Uniqueness
Generality

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

12 / 46


Complexity
Criteria for evaluating the complexity of an algorithm
Memory
CPU time

Definition
The size of input is defined to be the number of necessary bits for

representing it

Definition
The basic operations are the operations which can be performed in a
bounded time, and do not depend on the size of the input
We evaluate the complexity of an algorithm in term of the number of basic
operations it performs
Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

13 / 46


Complexity

Worst-case time complexity :
The longest execution time the algorithm takes given any input of size
n
Used to compare the efficiency of algorithms

Average-case time complexity : execution time the algorithm takes on
a random input
Best-case time complexity : The smallest execution time the
algorithm takes given any input of size n

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

14 / 46


Outline

1

First example

2

Algorithms and Complexity

3

Big-Oh notation

4

Pseudo code

5

Analysis of algorithms

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

15 / 46

Big-Oh notation
Given a fucntion g (n), we denote :
Θ(g (n)) = {f (n) : ∃c1 , c2 , n0 s.t. 0 ≤ c1 g (n) ≤ f (n) ≤ c2 g (n), ∀n ≥
n0 }
Example :
10n2 − 3n = Θ(n2 )

source : http ://www.personal.kent.edu/ rmuhamma/Algorithms/MyAlgorithms/intro.htm
Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

16 / 46


Big-Oh notation
Given a fucntion g (n), we denote :
O(g (n)) = {f (n) : ∃c, n0 > 0 s.t. f (n) ≤ cg (n), ∀n ≥ n0 }
Example :
2n2 = O(n3 )

source : http ://www.personal.kent.edu/ rmuhamma/Algorithms/MyAlgorithms/intro.htm
Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

17 / 46

Big-Oh notation
Given a fucntion g (n), we denote :
Ω(g (n)) = {f (n) : ∃c, n0 > 0 s.t. cg (n) ≤ f (n), ∀n ≥ n0 }
Example :
5n2 = Ω(n)

source : http ://www.personal.kent.edu/ rmuhamma/Algorithms/MyAlgorithms/intro.htm

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

18 / 46


Big-Oh notation

When we say ”the time complexity is O(f (n))” : the time complexity
in the worst case is O(f (n))
When we say ”the time complexity is Ω(f (n))” : the time complexity
in the best case is Ω(f (n))

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

19 / 46

Little-o notation

Given a fucntion g (n), we denote :
o(g (n)) = {f (n) : ∀ constant c > 0, ∃n0 > 0 s.t. 0 ≤ f (n) <
cg (n), ∀n ≥ n0 }
Example :
5n2 = o(n3 )

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

20 / 46


Little-o notation

Given a fucntion g (n), we denote :
ω(g (n)) = {f (n) : ∀ constant c > 0, ∃n0 > 0 s.t. 0 ≤ cg (n) <
f (n), ∀n ≥ n0 }
Example :
5n2 = ω(n1.5 )

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

21 / 46

Big-Oh notation
f (n)
< ∞ ⇒ f (n) = O(g (n))
n→∞ g (n)
lim

f (n)
> 0 ⇒ f (n) = Ω(g (n))
g (n)

lim

n→∞

0 < lim

n→∞

lim

f (n)
= 0 ⇒ f (n) = o(g (n))
g (n)

lim

f (n)
= ∞ ⇒ f (n) = ω(g (n))
g (n)

n→∞

n→∞

Pham Quang Dung ()

f (n)
< ∞ ⇒ f (n) = Θ(g (n))
g (n)

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

22 / 46


Big-Oh notation

O(lgn) = O(lnn)
lg a n = o(nb ) where a, b are constant
n! = o(nn )
n! = ω(2n )
logn! = Θ(nlogn)

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

23 / 46

Big-Oh notation

Example
A = log3 (n2 ) vs. B = log2 (n3 )
A = 2log3 n = 2lnn/ln3 where we denote lnx = loge (x)
B = 3log2 n = 3lnn/ln2
A
B

= constant ⇒ A = Θ(B)

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

24 / 46


Big-Oh notation

Example
A = nlg 4 vs. B = 3lgn where we denote lgx = log2 x
B = 3lgn = nlg 3 (logb a = loga b)
A
B

= nlg (4/3) → ∞

A = ω(b)

Pham Quang Dung ()

Data structures and Algorithms Basic definitions and notations Hanoi, 2012

25 / 46