Digital Video Compression Fundamentals and Standards docx

Digital Video Compression
Digital Video Compression
Fundamentals and Standards
Fundamentals and Standards
Speaker: Wei-Yi Wei
Speaker: Wei-Yi Wei
Advisor: Prof. Jian-Jung Ding
Advisor: Prof. Jian-Jung Ding
Digital Image and Signal Processing
Digital Image and Signal Processing
Lab
Lab
GICE, National Taiwan University
GICE, National Taiwan University
2008/12/26
Digital Video Compression Fundamentals an
d Standards
2
Outline
Outline
Introduction
Introduction
Video Compression Standards
Video Compression Standards
Simulation Reference Software
Simulation Reference Software
Future Work and Conclusions
Future Work and Conclusions
2008/12/26
Digital Video Compression Fundamentals an
d Standards

3
Outline
Outline
Introduction
Introduction
Video Compression Standards
Video Compression Standards
Simulation Reference Software
Simulation Reference Software
Future Work and Conclusions
Future Work and Conclusions
2008/12/26
Digital Video Compression Fundamentals an
d Standards
4
Introduction (1/2)
Introduction (1/2)
Why video compression technique is important ?
Why video compression technique is important ?
One movie video without compression
One movie video without compression
720 x 480 pixels per frame
720 x 480 pixels per frame
30 frames per second
30 frames per second
Total 90 minutes
Total 90 minutes
Full color
Full color
The total quantity of data = 167.96 G Bytes !!

2008/12/26
Digital Video Compression Fundamentals an
d Standards
5
Introduction (2/2)
Introduction (2/2)
What is the difference between video compression and image
What is the difference between video compression and image
compression?
compression?
Temporal Redundancy
Temporal Redundancy
Coding method to remove redundancy
Coding method to remove redundancy
Intraframe Coding
Intraframe Coding
Remove spatial redundancy
Remove spatial redundancy
Interframe Coding
Interframe Coding
Remove temporal redundancy
Remove temporal redundancy
2008/12/26
Digital Video Compression Fundamentals an
d Standards
6
The most intuitive method to remove
The most intuitive method to remove
Spatiotemporal redundancy
Spatiotemporal redundancy

3-Dimensional DCT
3-Dimensional DCT
Remove spatiotemporal correlation
Remove spatiotemporal correlation
Good for low motion video
Good for low motion video
Bad for high motion video
Bad for high motion video
1 1 1
3
0 0 0
8 (2 1) (2 1) (2 1)
( , , ) ( ) ( ) ( ) ( , , ) cos cos cos
2 2 2
N N N
t x y
x u y v t w
F x y t C u C v C w x y t
N N N N
π π π
− − −
= = =
+ + +
     
= Ψ
     
     
∑∑∑
for 0, , 1 , 0, , 1 and 0, , 1
1/ 2 for 0

where 8 and ( )
1 otherwise
u N v N w N
k
N C k
= − = − = −

=

= =



2008/12/26
Digital Video Compression Fundamentals an
d Standards
7
The most popular method to remove
The most popular method to remove
temporal redundancy
temporal redundancy
The Block-Matching Algorithm
The Block-Matching Algorithm
2008/12/26
Digital Video Compression Fundamentals an
d Standards
8
Matching Function
Matching Function
The dissimilarity between two blocks and

The dissimilarity between two blocks and
The matching criteria
The matching criteria
Mean square error (MSE)
Mean square error (MSE)


High precision is needed
High precision is needed
Mean absolute difference (MAD)
Mean absolute difference (MAD)


Low precision is enough
Low precision is enough
2
( , ) ( - )M u v u v=
( , ) | - |M u v u v=
1
1 , 1
( , ) [ ( , ), ( , )]
p q
n n x y
Vy Vx
D s t M x y x V y V
−
= =
= Ψ Ψ + +
∑ ∑
n

ψ
1n−
ψ
( , )D s t
2008/12/26
Digital Video Compression Fundamentals an
d Standards
9
The Exhaustive Block-Matching
Algorithm
Reference Frame Current Frame
Search
Range
Motion
Vector
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
11 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
12 22 33 44 55 66 77 88
11 23 34 44 55 66 77 88
11 23 34 44 55 66 77 88
11 23 34 44 55 66 77 88

11 23 34 44 55 66 77 88
1 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0
1 0 0 0 0 0 0 0
0 1 1 0 0 0 0 0
0 1 1 0 0 0 0 0
0 1 1 0 0 0 0 0
0 1 1 0 0 0 0 0
|A|=12
2008/12/26
Digital Video Compression Fundamentals an
d Standards
10
Fast Block-Matching Algorithms
Fast Block-Matching Algorithms
EBMA needs Intensive computation
EBMA needs Intensive computation


Fast Algorithm is needed
Fast Algorithm is needed
Find the possible local optimal
Find the possible local optimal
[ ] [ ] [ ]
720 480
128 128 16 16 30 170 G instructions/sec
16 16
 
× × × × × × =

 
 
Reference Frame Current Frame
2008/12/26
Digital Video Compression Fundamentals an
d Standards
11
Fast Block-Matching Algorithms
Fast Block-Matching Algorithms
The characteristics of fast algorithm
The characteristics of fast algorithm
Not accurate as EBMA
Not accurate as EBMA
Save large computation
Save large computation
Two famous fast algorithm
Two famous fast algorithm
2-D logarithm Search Method
2-D logarithm Search Method
Three Steps Search Method
Three Steps Search Method
2008/12/26
Digital Video Compression Fundamentals an
d Standards
12
2-D logarithm Search Method
2-D logarithm Search Method
1 1
1
1

1 2
2
2
3
3
4
44
4 4
1
2 3
4
2008/12/26
Digital Video Compression Fundamentals an
d Standards
13
Three Step Search Method
Three Step Search Method
11 1
1
1
2 2
2
2
11
11
2
2
2
2
3 3 3

3 3
3 3 3
1 2
3
2008/12/26
Digital Video Compression Fundamentals an
d Standards
14
Multiresolution Motion Estimation (1/3)
Multiresolution Motion Estimation (1/3)
The number of levels is
The number of levels is
L
L
l
l
-th level images of the target frames
-th level images of the target frames
where is set of pixels at level
where is set of pixels at level
L
L


At the
At the
l
l
-th level, the MV is
-th level, the MV is

At the
At the
l
l
-th level, the estimated MV is
-th level, the estimated MV is
Determine update such that
Determine update such that
error
error
is minimized
is minimized
The new motion vector is
The new motion vector is
,
( ), , 1, 2,
t l l
tΨ ∈Λ =x x
l
Λ
%
1
( ) ( ( ))
l
l
U
−
=d x d x
( )xd
%

2, 1,
| ( ( ) ( )) ( )) |
p
l
l l l
x l
error
∈Λ
= ψ + + −ψ
∑
x d x q x x
( )
l
q x
%
( ) ( ) ( )
l
l l
= +d x d x q x
2008/12/26
Digital Video Compression Fundamentals an
d Standards
15
Multiresolution Motion Estimation (2/3)
Multiresolution Motion Estimation (2/3)


Variable block size method
Variable block size method
2008/12/26

Digital Video Compression Fundamentals an
d Standards
16
Multiresolution Motion Estimation (3/3)
Multiresolution Motion Estimation (3/3)
2008/12/26
Digital Video Compression Fundamentals an
d Standards
17
Outline
Outline
Introduction
Introduction
Video Compression Standards
Video Compression Standards
Simulation Reference Software
Simulation Reference Software
Future Work and Conclusions
Future Work and Conclusions
2008/12/26
Digital Video Compression Fundamentals an
d Standards
18
The Development of Video
The Development of Video
Compression Standards
Compression Standards
2008/12/26
Digital Video Compression Fundamentals an
d Standards

19
The MPEG-1 Standard
The MPEG-1 Standard
Group of Pictures
Motion Estimation
Motion Estimation
Motion Compensation
Motion Compensation
Differential Coding
Differential Coding
DCT
DCT
Quantization
Quantization
Entropy Coding
Entropy Coding
2008/12/26
Digital Video Compression Fundamentals an
d Standards
20
Group of Pictures (1/2)
Group of Pictures (1/2)
I-frame (Intracoded Frame)
I-frame (Intracoded Frame)
Coded in one frame such as DCT.
Coded in one frame such as DCT.
This type of frame do not need previous frame
This type of frame do not need previous frame
P-frame (Predictive Frame)
P-frame (Predictive Frame)

One directional motion prediction from a previous frame
One directional motion prediction from a previous frame
The reference can be either I-frame or P-frame
The reference can be either I-frame or P-frame
Generally referred to as inter-frame
Generally referred to as inter-frame
B-frame (Bi-directional predictive frame)
B-frame (Bi-directional predictive frame)
Bi-directional motion prediction from a previous or future frame
Bi-directional motion prediction from a previous or future frame
The reference can be either I-frame or P-frame
The reference can be either I-frame or P-frame
Generally referred to as inter-frame
Generally referred to as inter-frame
2008/12/26
Digital Video Compression Fundamentals an
d Standards
21
Group of Pictures (2/2)
Group of Pictures (2/2)
The distance between two nearest P-frame or P-frame and I-frame
The distance between two nearest P-frame or P-frame and I-frame
denoted by
denoted by
M
M
The distance between two nearest I-frames
The distance between two nearest I-frames
denoted by
denoted by

N
N
I B B P B B P B B I
GOP
GOP
Bidirectional Motion
Compensation
Forward Motion
Compensation
N=9
M=3
2008/12/26
Digital Video Compression Fundamentals an
d Standards
22
The MPEG-1 Encoder (1/4)
The MPEG-1 Encoder (1/4)
DCT Q
Entropy
Coding
Q
-1
IDCT
Motion
Compensation
Frame
Memory
Motion
Estimation
DCT Q

Entropy
Coding
DCT Q
Q
-1
IDCT
Motion
Compensation
Frame
Memory
Motion
Estimation
Intra-frame
Entropy
Coding
Residue
Motion Vector
Inter-frame
2008/12/26
Digital Video Compression Fundamentals an
d Standards
23
The MPEG-1 Encoder (2/4)
The MPEG-1 Encoder (2/4)
Differential Coding
Differential Coding

is the input image
is the input image


is the predictive image
is the predictive image
DCT
DCT
^
( ) ( ) ( )D t t t= ψ − ψ
( )tψ
^
( )tψ
1 1
0 0
2 (2 1) (2 1)
( , ) ( ) ( ) ( , )cos cos
2 2
for 0, , 1 and 0, , 1
1/ 2 for 0
where 8 and ( )
1 otherwise
N N
x y
x u y v
F u v C u C v f x y
N N N
u N v N
k
N C k

π π
− −
= =
+ +
   
=
   
   
= − = −

=

= =



∑∑
2008/12/26
Digital Video Compression Fundamentals an
d Standards
24
The MPEG-1 Encoder (3/4)
The MPEG-1 Encoder (3/4)
Quantization
Quantization
Intra quantization matrix
Intra quantization matrix
Inter quantization matrix
Inter quantization matrix
intra

8 16 19 22 26 27 29 34
16 16 22 24 27 29 34 37
19 22 26 27 29 34 34 38
22 22 26 27 29 34 37 40
22 26 27 29 32 35 40 48
26 27 29 32 35 40 48 58
26 27 29 34 38 46 56 69
27 29 35 38 46 56 69 83
Q
 
 ÷
 ÷
 ÷
 ÷
 ÷
=
 ÷
 ÷
 ÷
 ÷
 ÷
 ÷
 
intra
16 17 18 19 20 21 22 23
17 18 19 20 21 22 23 24
18 19 20 21 22 23 24 25
19 20 21 22 23 24 26 27
20 21 22 23 25 26 27 28
21 22 23 24 26 27 28 30

22 23 24 26 27 28 30 31
23 24 35 27 28 30 31 33
Q
 
 ÷
 ÷
 ÷
 ÷
 ÷
=
 ÷
 ÷
 ÷
 ÷
 ÷
 ÷
 
2008/12/26
Digital Video Compression Fundamentals an
d Standards
25
The MPEG-1 Encoder (4/4)
The MPEG-1 Encoder (4/4)
Motion Compensation
Motion Compensation
Exploit motion vector and the previous reconstructed frame to generate
Exploit motion vector and the previous reconstructed frame to generate
the predictive frame
the predictive frame

is the compensated image
is the compensated image


is the previous image
is the previous image


is the motion vector
is the motion vector
¶
( , ) ( ( , ), ( , )),( , ) ( , )
n n x y
x y x v p q y v p q x y MB p qψ = ψ + + ∈
¶
( , )
n
x yψ
( , )
n
x yψ
( ( , ), ( , ))
x y
v v p q v p q=
Reference frame
Reference frame
Target frame

Target frame