Mã nén Lecture_3 Block

Data Compression
Lecture 3
Block Coding
Alexander Kolesnikov

Shortcoming of Huffman codes
Alphabet: a, b.
p
a
=p=0.99, p
b
=q=0.01
1) Entropy
H
1
=-(p*log
2
(p)+q*log
2
(q))=0.081 bits/pel
2) Huffman code: p
a
=’0’, p
b
=’1’
Bitrate R
1
= 1*p+1*q = p+q = 1 bit/pel!
Make a new alphabet blocking symbols!

Block coding: n=2
New alphabet: ’A’=’aa’, ’B’=’ab’, ’C’=’ba’, ’D’=’bb’
p
A
=p
2
=0.9801, p
B
=pq=0.0099, p
C
=pq=0.0099, p
D
= q
2
=0.0001
1) Entropy: H
2
=-(0.9801*log
2
(0.9801) + 0.0099*log
2
(0.0099) +
+ 0.0099*log
2
(0.0099) + 0.0001*log
2
(0.0001))=
=(0.0284+0.0659+0.0659+0.0013)/2= 0.081 bits/pel
Why H
2

=H
1
?
2) Huffman code: c
A
=’0’, c
B
=’10’, c
C
=’110’, c
D
=’111’
L
A
=1, L
B
=2, L
C
=2, L
D
=3
Bitrate R
2
= (1*p
A
+2*p
B
+3*p
C
+3*p

D
)/2=0.515 bits/pel

Block coding: n=3
’A’=’aaa’ -> p
A
=p
3
’B=’aab’, ’C’=’aba’, ’D’=’baa’ -> p
B
= p
C
=p
D
=p
2
q
’E’=’abb’, ’F’=’bab’, ’G’=’bba’ -> p
E
= p
F
=p
G
=pq
2

’H’=’bbb’ -> p
H
=q
3

Huffman code:
c
A
=’0’, c
B
=’10’, c
C
=’110’, c
D
=’1110’
c
E
=’111100, c
B
=’111101’, c
G
=’111110’, c
H
=’111111’
Entropy H
3
?
Bitrate:
R
3
= (1*p
A
+2*p
B
+3*p

C
+4*p
D
+6*(p
E
+p
F
+p
G
+p
H
))/3=
= 0.353 bits/pel

Block coding: n→ ∞
p
a
=p=0.99, p
b
=q=0.01
Entropy H
n
=0.081 bits/pel
Bitrate for Hufman coder:
n= 1: R
1
= 1.0 bit 2 symbols in alphabet
n= 2: R
2
= 0.515 bits 4 symbols in alphabet

n= 3: R
3
= 0.353 bits 8 symbols in alphabet
If block size n → ∞? H
n
≤ R
n
< H
n
+1/n
Problem - alphabet size and Huffman table size grows
exponentially with number of symbols n blocked.
∑∑
==
+≤≤
N
i
nn
N
i
nnn
n
BpBp
n
RBpBp
n
1
2
1
*

2
1
)(log)(
1
)(log)(
1