Các phép toán cơ bản trong xử lý ảnh

1
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
1/27
Twodimensionalsystems
&
Mathematicalpreliminaries
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
2/27
1. Notationsanddefinitions
2. Linearsystemsandshiftinvariance
3. FourierTransform
4. ZTransformorLaurentseries
5. Matrixtheoryandresults
6. BlockmatricesandKroneckerproducts
7. Randomsignals
8. Someresultsfromestimationtheory
9. Someresultsfrominformationtheory
2
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction

3/27
1. Notationsanddefinitions
Ø1Dand2Dfunctions
ü1D:
),(x
d
 )(n
d

),(ng
),(xf
ü2D:
),,( yx
d

),( nm
d

),,( nmg),,( yxf
ØSeparableformsof2Dfunctions
üDirac:
üKronecker:
ürect(x,y),sinc(x,y),comb(x,y)
)()(),( yxyx
d d d
× =
)()(),( nmnm
d d d
× =
ØSpecialfunctions

üDiracdelta:
üShifting:
üScaling:
üKroneckerdelta:
üShifting:
üRectangle:
üSignum:
üSinc:
üComb:
üTriagle:
1)(lim;0,0)(
0
= ¹ =
ò
+
-
®

e
e
e
d d
 dxxxx
)(')'()'( xfdxxxxf = -
ò
+
-

e
e

d

a
x
ax
)(
)(

d
d
=
î
í
ì
=
¹
=
01
00
)(
n
n
x
d

)()()( nfmnmf = -
å
¥
¥ -

d

ï
î
ï
í
ì
>
£
=
2/10
2/11
)(
x
x
xrect
ï
î
ï
í
ì
< -
=
>
=
01
00
01
)(
x

x
x
xsign
å
¥
¥ -
- = )()( nxxcom b
d

x
x
xc

p
p
sin
)(sin =
ï
î
ï
í
ì
>
£ -
=
10
11
)(
x
xx

xtri
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
4/27
2. Linearsystemsandshiftinvariance
Ø2Dlinearsystems
[]
× H
[ ]
),(),( nmxnmy H =
),( nmx
üLinearsuperpositionproperty:
üImpulseresponse:
üImpulseresponseiscalledPSF:Inputandoutputarepositivequantities
üIngeneral:Impulseresponsecantakenegativeorcomplexvalues
üRegionofsupport(RoS)ofimpulseresponse
üFiniteimpulseresponse(FIR)andinfiniteimpulseresponse(IIR):When
RoSisfiniteorinfinite
[ ]
)','()', ';,( nnmmnmnmh - - H =
d

[ ] [ ] [ ]
),(),(),(),(),(),(
221122112211
nmyanmyanmxanmxanmxanmxa + = H + H = + H
3

HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
5/27
ỉOutputofalinearsystem:
[ ]
ỳ
ỷ
ự
ờ
ở
ộ
- - H = H =
ồồ
)','()','(),(),(
' '
nnmmnmxnmxnmy
m n

d

[ ]
ồồ ồồ
= - - H = ị
' '' '
)',',()','()','()','(),(
m nm n
nmnmhnmxnnmmnmxnmy

d
ỉSpatiallyinvariantorshiftinvariantsystem:
[ ]
)0,0,(),( nmhnm = H
d

[ ]
)0,0','()','()',',( nnmmhnnmmnmnmh - - = - - H =
d

)','()',',( nnmmhnmnmh - - = ị
ỹTheshapeofimpulseresponsedoesnotchangeastheimpulse
responsemovesaboutthe(m,n)plan
ỹDiscreteconvolution:
),(*),()','()','(),(
' '
nmxnmhnmxnnmmhnmy
m n
= - - =
ồồ
ỹContinuousconvolution:
'')','()','(),(*),(),( dydxxxfyyxxhyxfyxhyxg
ũ ũ
Ơ
Ơ -
Ơ
Ơ -
- - = =
ốExp.2.1
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications

2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
6/27
3. Fouriertransform
ỉFTofa1Dfunction:f(x)
[ ]
[ ]
ũ
ũ
Ơ +
Ơ -
- -
+Ơ
Ơ -
-
= =
= =

x x x
x
px
px

deFFxf
dxexfxfF
xj
xj
211

2
)()()(:
)()()(:
ỉFTofa2Dfunction:f(x,y)
[ ]
[ ]
ũ ũ
ũ ũ
Ơ +
Ơ -
Ơ +
Ơ -
+
- -
+Ơ
Ơ -
+Ơ
Ơ -
+ -
= =
= =
21
)(2
2121
11
)(2
21
21
21
),(),(),(:

), (),(),(:

x x x x x x
x x
x x p
x x p

ddeFFyxf
dxdyeyxfyxfF
yxj
yxj
4
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
7/27
Performingachangeofvariables:
ỉPropertiesofFT
ỹSpatialfrequencies
ỹUniqueness
ỹSeparability
ũ ũ ũ ũ
+Ơ
Ơ -
-
+Ơ
Ơ -
-

+Ơ
Ơ -
+Ơ
Ơ -
+ -
ỳ
ỷ
ự
ờ
ở
ộ
= = dyedxeyxfdxdyeyxfF
yjxjyxj
2121
22)(2
21
),(),(),(

px px x x p
x x
ỹFrequencyresponseandeigenfunctionsofshiftinvariantsystems
),( yxh
FH
F
ũ ũ
+Ơ
Ơ -
+Ơ
Ơ -
+

- - = '')','(),(
)''(2
21
yddxeyyxxhyxg
yxj
x x p

',' yyyxxx - = - =
)(2
21
:where
yxj
e

x x p
+
= F
)(2
21
21
),(),(
yxj
eyxg

x x p
x x

+
H = ị
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications

2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
8/27
ỹConvolutiontheorem
ỹInnerproductpreservation
),(F),(),G(),(*),(),(
212121

x x x x x x
ì H = ị = yxfyxhyxg
ỹCorrelationbetween2realfunctions
ũ ũ
+Ơ
Ơ -
+Ơ
Ơ -
+ + = ã = '')','()','(),(),(),( dydxyyxxfyxhyxfyxhyxc
Performingachangeofvariables:
),F(),(),(C),(),(),(
212121

x x x x x x
ì - - H = ị ã - - = yxfyxhyxc
ũ ũ ũ ũ
+Ơ
Ơ -
+Ơ
Ơ -

+Ơ
Ơ -
+Ơ
Ơ -
= =
2121
*
21
*
),(H),(F),(),(
x x x x x x
dddxdyyxhyxfI
Settingh=fốParsevalenergyconservationformula
ũ ũ ũ ũ
+Ơ
Ơ -
+Ơ
Ơ -
+Ơ
Ơ -
+Ơ
Ơ -
=
21
2
21
2
),(F),(
x x x x
dddxdyyxf

5
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
9/27
üFouriertransformpairs
comb(x,y)
tri(x,y)
rect(x,y)
1
),( yxf ),(F
21

x x

),( yx
d

yljxkj
ee

p p
 22 ± ±
2010
22
x p x p
 yjxj
ee

± ±
),(
21
lk m m
x x d

),(
00
yyxx ± ±
d

)(
22
yx
e
+ -
p

)(
2
2
2
1
y
e
+ -
x p

),(s
21

x x
inc
),(s
21
2

x x
inc
),(
21

x x
comb
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
10/27
Innerproduct
Spatialcorrelation
Multiplication
Convolution
Modulation
Shifting
Scaling
Separability
Conjungation
Linearity

Rotation
Propertyof2DFT FouriertransformFunction
),F(
21
)(2
2010

x x
x x p
 yxj
e
+ ±
)()(
21
yfxf
),( yxf ± ±
),(F),(H),(G
212121

x x x x x x
× =
),(),(
2211
yxfayxfa +
),(F),(F
21222111

x x x x
 aa +
),(

*
yxf ),(F
21
*

x x
- -
)( F)(F
2211

x x

),( yxf
),( byaxf
[ ]
abba /)/,/(F
21

x x

),(
00
yyxxf ± ±
),(
)(2
21
yxfe
yxj
h h p
+ ±

),(F
2211

h x h x
m m
),(),(),( yxfyxhyxg * =
),(),(),( yxfyxhyxg × =
),(F),(H),( G
212121

x x x x x x
* =
),(),(),( yxfyxhyxc · = ),(F),(H),(C
212121

x x x x x x
× - - =
ò ò
+¥
¥ -
+¥
¥ -
= dxdyyxhyxfI ),(),(
*
ò ò
+¥
¥ -
+¥
¥ -
2121

*
21
),(H),(F
x x x x x x
 dd
),(F
2
2
1
1

x x
m m
),(F
21

x x
6
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
11/27
Theevaluationofatand yieldsFTof
4. ZTransformorLaurentseries
ỉFouriertransformofsequences(Fourierseries):Selfreading
ỉGeneralizationofFTseries:Ztransform
ỹFor2Dsequencex(m,n):
wherez

1
,z
2
arecomplexvariables
ồ ồ
+Ơ
-Ơ =
+Ơ
-Ơ =
- -
=
m n
nm
zznmxzzX
2121
),(),(
ỹRegionofconverge(RoC):thisseriesconvergesuniformlyinthisregion
ỹZtransformofaLSIsystemiscalledtransferfunction
),(
),(
),(
),(),(),(
21
21
21
212121
zzX
zzY
zzH
zzXzzHzzY

= ị
=
ỉInverseZtransform:
11where,),(
)2(
1
),(
21
2
1
1
2
1
121
2
= = =
ũũ
- -
zzdzdzzzzzX
j
nmx
nm

p

),(
21
zzX
1
1

w
j
ez =
2
2

w
j
ez =
),( nmx
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
12/27
ỉPropertiesof2DZtransform
Multiplication
Convolution
Modulation
Shifting
Separability
Conjungation
Linearity
Rotation
Property FouriertransformFunction
),(
2121
00

zzXzz
nm
)()(
21
nxmx
),( nmx - -
),(),(
2121
zzFzzH ì
),(),(
2211
nmxanmxa +
),(),(
21222111
zzX azzXa +
),(
*
nmX
),(
*
2
*
1
*
zzX
)( F)(F
2211

x x

),( yxf
),(
00
nnmmx
),( nmxba
nm
),(),( nmxnmh *
),(),( nmynmx
),(
1
2
1
1
- -
zzX
),(F
21

x x

ữ
ứ
ử
ỗ
ố
ổ
b
z
a
z

X
21
,
ũ ũ
ữ
ữ
ứ
ử
ỗ
ỗ
ố
ổ
ữ
ữ
ứ
ử
ỗ
ỗ
ố
ổ
1 2
'
2
2
'
1
1
'
2
'

1
'
2
2
'
1
1
2
),(,
2
1
C C
z
dz
z
dz
zzY
z
z
z
z
X
j
p
7
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction

13/27
ỉCausality
ỹCausal:Impulseresponseforanditstransferfunction
musthaveaonesidedLaurentseries
0)( =nh 0 <n
ồ
Ơ
=
-
=
0
)()(
n
n
znhzH
ỹAnticausal:Impulseresponseforanditstransferfunction
musthaveaonesidedLaurentseries
0)( =nh
0 n
ỹNoncausal:Neithercausaloranticausal
ỉStability:Outputremainsuniformlyboundedforanyboundedinput
Ơ <
ồ
Ơ
=0
)(
n
nh
ỉCausalandstablesystem:polesofH(z)mustlieinsidetheunitcircle
ỉ2Dcase: RoCofmustincludetheunitcircles

ồồ
Ơ <
m n
nmh ),(
),(
21
zzH
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
14/27
5. Matrixtheoryandresults
ỉVectorsandmatrices
ỹColumnvectorofsizeN:
NnnuU á = = 1),(
ỹRowvectorofsizeM:
MmmuU á = = 1),(
ỹMatrixAofsizeMxNcontainingMrows,Ncolumns
ỳ
ỳ
ỳ
ỳ
ỷ
ự
ờ
ờ
ờ
ờ

ở
ộ
=
),()2,(),1,(
),2()2,2(),1,2(
),1()2,1(),1,1(
NMaMa Ma
Naaa
Naaa
A
L
L
L
L
ỹIndexnotation:
{ }
1,0),,( - Ê Ê =

NnmnmaA
NN
ỹAnimageisusuallyvisualizedasamatrix
ốExp.2.2
8
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
15/27
ØRowandcolumnordering

üRoworderedvector(rowstacking)
[ ]
T
T
NMxMxNxxNxxxx ),( ,),1,(),2( ,),1,2(),,1( ,),2,1(),1,1( L =
[ ]
T
T
NMxMxMxxMxxxx ),( ,),,1()2,( ,),2,1(),1,( ,),1,2(),1,1( L =
üColumnorderedvector(columnstacking)
ØMatrixtheorydefinitions
{ }
),( nmaA =
üMatrix:
üTranspose:
{ }
),( mnaA
T
=
{ }
),(
**
nmaA =
üComplexconjungate:
{ }
)( nmI - =
d
üConjungatetranspose:
üIdentitymatrix:
üNullmatrix:

{ }
0 =O
{ }
),(
**
mnaA
T
=
üMatrixaddition:
{ }
),(),( nmbnmaBA + = +
:A,B:Samedimension
üScalarmultiplication:
{ }
),( nmaA
a a
=
üMatrixmultiplication:
å
=
=
K
k
nkbkmanmc
1
),(),(),(
A:MxK,B:KxN,C:MxN
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0

1/2012
Introduction
16/27
üVectorinnerproduct:
å
= = )()(,
**
nynxYXYX
T
:Scalarquantity,ifequal0
èXandYareorthogonal
üVectorouterproduct:
{ }
)()( nymxXY
T
=
:X:Mx1,Y:Nx1,XY
T
:MxN
üSymmetric:
T
AA =
ü Hermitian:
T
AA
*
=
:RealsymmetricmatrixisHermitan.Eigenvaluesarereal
üDeterminant:
A

üRankofA:Numberofindependentrowsorcolumns
üInversematrix:
IAAAA = =
- - 11
:Squarematrixonly
üSingular:A
1
doesnotexistand
0 =A
üEigenvalues:allrootsof
k

l

0 = - IA
k

l
üEigenvectors:allsolutionsof
k
F 0, ¹ F F = F
kkkk
A
l
9
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction

17/27
ØTransposeandconjungaterules
[ ]
[ ]
[ ] [ ]
[ ]
**
*
1
1
*
*
.4
.3
.2
.1
BAAB
AA
ABAB
AA
T
T
TT
T
TT
=
=
=
=
-

-
ØToeplitzandcirculantmatrices
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ë
é
=
-
-
+ - -
+ - -
0121
12
2101
110
,,,
,,
,
tttt
tt
tttt
ttt

A
N
N
N
L
L
L
L
ØCirculantmatrixC
ú
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê
ê
ê
ë
é
=
-
- -
-
0121

2
2101
1210
,,,
,,
,,
cccc 
c
cccc
cccc
C
N
NN
N
L
L
L
L
CisalsoToeplitzandc(m,n)=c((mn)moduloN)
èExp.2.3
èExp.2.4
t(i,j)=t
ij
:Constantelementsalongthe
maindiagonalandsubdiagonal
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction

18/27
whereandareeigenvaluesandeigenvectorsofR
üOtherform,whichisthesetofeigenvalueequations
ØOrthogonalandunitarymatrices
üOrthogonalmatrix:
üUnitarymatrix:
IAAAAAA
TTT
= = =
- ***1
or
èExp.2.5a
ØDiagonalforms
üIfRisHermitianmatrix,thereexistsaunitarymatrixΦsuchthat
whereΛisadiagonalmatrixcontainingeigenvaluesofR
L = F RΦ
*T
L = ΦRΦ
Nk
kk
,,2,1,ΦRΦ
k
L = =
l

{ }
k

l


k
Φ
èExp.2.5b
IAAAAAA
TTT
= = =
-
or
1
10
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
19/27
Ø isblockToeplitzifisToeplitzor
6. BlockmatricesandKroneckerproducts
ØBlockmatricesofsize:eachelementisamatrixitself
ú
ú
ú
ú
ú
û
ù
ê
ê
ê
ê

ê
ë
é
= À
nmmm
n
n
AAA
AAA
AAA
,2,1,
,22,21,2
,12 ,11,1
,
,
,
L
L
L
L
wherearematrices
ji
A
,
)(, jiji
AA
-
=
Ø isblockcirculantifiscirculant
nmAA

njiji
= =
-
,
)ulomod)((,
èExp.2.6
èExp.2.7
ØKroneckerproducts:A:M
1
xM
2
,B:N
1
xN
2
:
ØSeparableoperations:selfreading
{ }
BnmaBA ),( = Ä
qp ´
À
ji
A
,
À
ji
A
,
nm´
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications

2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
20/27
ü isanNx1vector
7. Randomsignals
ØDefinitions:givenasequenceofrandomvariablesu(n)
üMean:
üVariance:
üCovariance:
üCrosscovariance:
üAutocorrelation:
üCrosscorrelation:
[ ]
)()()( nuEnn
u
= =
m m

[
]
2
2
2
)()()()( nnuEnn
u

m s s
- = =

[ ] [ ]
[
]
{
}
)'()'()()()',()'(),(
**
nnunnuEnnrnunuCov
u

m m
- - = =
[ ] [ ]
[
]
{
}
)'()'()()()',()'(),(
*
*
nnvnnuEnnrnvnuCov
vuuv

m m
- - = =
[
]
)'()()',()()()',()',(
**
nnnnrnunuEnnanna

uu

m m
- = = =
[ ]
)'()()',()()()',(
*
*
nnnnrnvnuEnna
vuuvuv

m m
- = =
ØForvectorofsizeNx1:u
[ ]
{ }
)(nE
m
= = μu
ü isanNxNmatrix
[ ]
[
]
{ }
)',())((
**
nnrECov
T
= = = - - = RRμuμuu
u

ü isanNxNmatrix
[ ]
[
]
{ }
)',())((,
*
*
nnrECov
uv
T
= = - - =
uvvu
Rμvμuvu
11
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
21/27
ỉGaussianrandomprocess:thejointprobabilitydensityofanyfinitesub
sequenceisaGaussiandistribution
ỉStationaryprocess
ỉGaussianornormaldistribution:forstandardnormaldistribution
ù
ỵ
ù
ý
ỹ

ù
ợ
ù
ớ
ỡ
- -
=
2
2
2
2
2
1
)(

s
m
ps

u
u
eup
1and0
2
= =
s m
ỹu(n)isstrictsensestationary:jointdensityofanypartial
sequenceisthesameasthatoftheshifted
sequenceforanyintegermandanylengthk
{ }

kllu Ê Ê1),(
{ }
klmlu Ê Ê + 1),(
ỹu(n)iswidesensestationary:
[ ]
constnuE = =
m
)(
[
]
)'()',()'()(
*
nnrnnrnunuE - = =
ỹSymmetry:
ỹNonnegativity:
',)',()',(
*
nnnnrnnr " =
nnxnxnnrnx
n n
" ạ
ồồ
,)(,0)'()',() (
'
*
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction

22/27
ỉMarkovprocess
ỹMarkovporpthorderMarkov
ỹGaussianMarkovpsequence
[ ] [ ]
npnununuEnununuE " - - = - - )(,),1(|)()2(),1(|)( L L
ỹCovariancefunctionofafirstorderstationaryMarkovsequenceu(n)
nnr
n
" < = ,1)(
r r
ỹToeplitzcovariancematrix
ỳ
ỳ
ỳ
ỳ
ỳ
ỳ
ỷ
ự
ờ
ờ
ờ
ờ
ờ
ờ
ở
ộ
=
-

-
1,,,
1,
,,1
1
12

r r
r
r
r r r
L
L
L
L
N
N
R
[ ] [ ]
npnununuprobnununuprob " - - = - - )(,),1(|)()2() ,1(|)( L L
12
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
23/27
ỉOrthogonalityandindependence
ỹxandyareindependent:
nypxpyxp

yxyx
" < = ,1)()(),(
,

r
ỹx(n)andy(n)randomsequencesareindependent:x(n)andy(n)are
independentforeverynandn
ỹxandyareorthogonal:
[ ]
0
*
=xyE
ỹxandyareuncorrelated:or
[ ]
[ ]
( )
[ ]
( )
**
EEE yxxy =
[
]
0))((E
*
= - -
yx
yx
m m
ỉKarhunenLoevetransform(KLT)
ỹ isacomplexrandomsequencewith

{ }
Nnnx Ê Ê1),( R
ỹ isanNxNunitarymatrix,whichreducestoitsdiagonalform
R
ỹKLTof:
x
xy
*T
=
ỹPropertyofKLT:
[ ] [ ]
{ }
Rxxyy = = =
TT ****
EE
[ ]
)()()(E
*
lklyky
k
- = ị
d l
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
24/27
8. Someresultsfromestimationtheory
ỉMeansquareestimates(MSE)

ỹ isarealrandomsequenceandxisarealrandomvariable
{ }
Nnny Ê Ê1),(
ỹ iscalledtheoptimummeansquareestimateofxiftheMSEisminimized

x
ỳ
ỷ
ự
ờ
ở
ộ
- =

2
2
)( xxE

e
s
Thatmeans:
( ) ( )
[ ]

x x x
ũ
+Ơ
Ơ -

= = = dpNyyyxEyxEx

yx
)()(,),2(),1(||
|
L
ỹIfxandy(n)areindependent,thensimplyismeanofxbecause:

x
( )
[ ]
( )
xEyxEExE = =
ỳ
ỷ
ự
ờ
ở
ộ

|
ỹForzeromeanGaussianrandomvariables,becomeslineariny(n)

x
ồ
=

=
N
n
nynx
1

)()(
a
13
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
25/27
ØTheorthogonalprinciples
üFromMSEtheory:errorvectorisorthogonaltoeveryrandomvariable
i.e forany
( )
)(,),2(),1()( Nyyygyg L =
0)()(
2
=
ú
û
ù
ê
ë
é
-
Ù
ygxxE
üOrthogonalityprincipleisusefulinlinearestimatesincethecond.meanis
difficulttoevaluateèfindingthatminimizestheMSE.Thatis:
)(n
a

[ ] [ ]
NnnxyEnykyEk
N
k
,,1,)()()()(
1
L = =
å
=

a
Inmatrixform:
xyy
rRα
1 -
=
TheminimizedMSEisgiven:
xy
T
x
rα - =
22

s s
e
Ifxandy(n)arenonzeromeanrandomvariables:
[ ]
)()()(
1

nnynxx
y
N
n
x
x

m a m m
- = - = -
å
=
Ù Ù
Ù
èProve
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
26/27
9. Someresultsfrominformationtheory
ØInformation:Thereisasourcegeneratingadiscretesetofindependent
message(e.ggraylevel)r
k
,withprobabilitiesp
k
.Sincep
k
≤1è
kk

pI
2
log - =
ØEntropy:Definedasaverageinfogeneratedbythesource(bit/message)
å
=
- =
L
k
kk
ppH
1
2
log
üForagivenL,maxentropyofasourceisdeterminedforuniform
distribution,i.e
L
LL
H
L
k
p
k
2
1
2
log
1
log
1

max = - =
å
=
LkLp
k
,,1,/1 L = =
èExp.2.13
BinarysourceèL=2.Thenp
1
=p,p
2
=1p,
10 £ £ p
Entropyis: èH
max
=?
)1(log)1(log)(
22
pppppHH - - - - = =
14
HanoiUniversityofScienceandTechnology SchoolofElectronicsandTelecommunications
2Dsystemsandmathematicalpreliminaries
**0
1/2012
Introduction
27/27
ỉRatedistortionfunction(RDF)
ỹRDFofarandomvariablexgivesminaveragerateR
D
requiredto

represent(orcode)itforafixeddistortionD
ỹx:Gaussianwithy:reproducedvalueMSE(xy):Distortionmeasure
2

s

[
]
2
)( yxED - =
ThenRDFofxisdefinedas:
( )
ỳ
ỷ
ự
ờ
ở
ộ
ữ
ữ
ứ
ử
ỗ
ỗ
ố
ổ
=
ù
ợ
ù

ớ
ỡ
>
Ê
=
D
D
DD
R
D
2
2
2
22
2
log
2
1
,0max
0
/log
2
1

s
s
s s
ỹMeansquaredistortionforGaussianvariables
andtheirreproducedvalues isdefinedas:
{ }

)1(,),1(),0( -Nxxx L
{ }
)1(,),1(),0( -Nyyy L
[ ]
ồ
-
=
- =
1
0
2
)()(
1
N
k
kykxE
N
D