(ebook) crc press - pattern recognition in speech and language processing 2003

CRC PRESS
Boca Raton London New York Washington, D.C.
Edited by
WU CHOU
Avaya Labs Research
BIING HWANG JUANG
Georgia Institute of Technology
PATTERN
RECOGNITION in
SPEECH and
LANGUAGE
PROCESSING
E\&5&3UHVV//&
7KLVERRNFRQWDLQVLQIRUPDWLRQREWDLQHGIURPDXWKHQWLFDQGKLJKO\UHJDUGHGVRXUFHV5HSULQWHGPDWHULDO
LVTXRWHGZLWKSHUPLVVLRQDQGVRXUFHVDUHLQGLFDWHG$ZLGHYDULHW\RIUHIHUHQFHVDUHOLVWHG5HDVRQDEOH
HIIRUWVKDYHEHHQPDGHWRSXEOLVKUHOLDEOHGDWDDQGLQIRUPDWLRQEXWWKHDXWKRUVDQGWKHSXEOLVKHUFDQQRW
DVVXPHUHVSRQVLELOLW\IRUWKHYDOLGLW\RIDOOPDWHULDOVRUIRUWKHFRQVHTXHQFHVRIWKHLUXVH
1HLWKHUWKLVERRNQRUDQ\SDUWPD\EHUHSURGXFHGRUWUDQVPLWWHGLQDQ\IRUPRUE\DQ\PHDQVHOHFWURQLF
RUPHFKDQLFDOLQFOXGLQJSKRWRFRS\LQJPLFURÀOPLQJDQGUHFRUGLQJRUE\DQ\LQIRUPDWLRQVWRUDJHRU
UHWULHYDOV\VWHPZLWKRXWSULRUSHUPLVVLRQLQZULWLQJIURPWKHSXEOLVKHU
$OOULJKWVUHVHUYHG$XWKRUL]DWLRQWRSKRWRFRS\LWHPVIRULQWHUQDORUSHUVRQDOXVHRUWKHSHUVRQDORU
LQWHUQDOXVHRIVSHFLÀFFOLHQWVPD\EHJUDQWHGE\&5&3UHVV//&SURYLGHGWKDWSHUSDJH
SKRWRFRSLHGLVSDLGGLUHFWO\WR&RS\ULJKW&OHDUDQFH&HQWHU5RVHZRRG'ULYH'DQYHUV0$
86$7KHIHHFRGHIRUXVHUVRIWKH7UDQVDFWLRQDO5HSRUWLQJ6HUYLFHLV,6%1
7KHIHHLVVXEMHFWWRFKDQJHZLWKRXWQRWLFH)RURUJDQL]DWLRQVWKDWKDYHEHHQJUDQWHG
DSKRWRFRS\OLFHQVHE\WKH&&&DVHSDUDWHV\VWHPRISD\PHQWKDVEHHQDUUDQJHG
7KHFRQVHQWRI&5&3UHVV//&GRHVQRWH[WHQGWRFRS\LQJIRUJHQHUDOGLVWULEXWLRQIRUSURPRWLRQIRU
FUHDWLQJQHZZRUNVRUIRUUHVDOH6SHFLÀFSHUPLVVLRQPXVWEHREWDLQHGLQZULWLQJIURP&5&3UHVV//&
IRUVXFKFRS\LQJ
'LUHFWDOOLQTXLULHVWR&5&3UHVV//&1:&RUSRUDWH%OYG%RFD5DWRQ)ORULGD

7UDGHPDUN1RWLFH3URGXFWRUFRUSRUDWHQDPHVPD\EHWUDGHPDUNVRUUHJLVWHUHGWUDGHPDUNVDQGDUH
XVHGRQO\IRULGHQWLÀFDWLRQDQGH[SODQDWLRQZLWKRXWLQWHQWWRLQIULQJH
9LVLWWKH&5&3UHVV:HEVLWHDWZZZFUFSUHVVFRP
E\&5&3UHVV//&
1RFODLPWRRULJLQDO86*RYHUQPHQWZRUNV
,QWHUQDWLRQDO6WDQGDUG%RRN1XPEHU
/LEUDU\RI&RQJUHVV&DUG1XPEHU
3ULQWHGLQWKH8QLWHG6WDWHVRI$PHULFD
3ULQWHGRQDFLGIUHHSDSHU
/LEUDU\RI&RQJUHVV&DWDORJLQJLQ3XEOLFDWLRQ'DWD
3DWWHUQUHFRJQLWLRQLQVSHHFKDQGODQJXDJHSURFHVVLQJHGLWHGE\:X&KRXDQG
%LLQJ+ZDQJ-XDQJ
SFP
,QFOXGHVELEOLRJUDSKLFDOUHIHUHQFHVDQGLQGH[
,6%1DONSDSHU
$XWRPDWLFVSHHFKUHFRJQLWLRQ3DWWHUQUHFRJQLWLRQV\VWHPV,&KRX:X,,
-XDQJ%+%LLQJ+ZDQJ
7.63
d³GF

E\&5&3UHVV//&
Preface
#RRTQCEJGU VQ VJG RTQDNGOU QH FGUKIPKPI URGGEJ CPF NCPIWCIG RTQEGUUKPI CNIQTKVJOU
HQT JWOCP OCEJKPG EQOOWPKECVKQP WUGF VQ DG VCMGP HTQO VJG RGTURGEVKXGU QH NKP
IWKUVKEU CPF URGGEJ UEKGPEG WPVKN VJG NCVG U &WG VQ VJG CFXCPEGU KP EQORWV
KPI CPF UVCVKUVKECN OQFGNKPI FCVC FTKXGP RCVVGTP TGEQIPKVKQP OGVJQFU JCXG DGEQOG C
HCUV OQXKPI TGUGCTEJ CTGC FWTKPI VJG RCUV VYQ FGECFGU CPF EQPVTKDWVGF OWEJ VQ VJG
RTQITGUU KP VJKU ſGNF #U VJG GTC QH KPHQTOCVKQP CIG EQPVKPWGU VQ FGXGNQR YG YKVPGUU
CP GXGT KPETGCUKPI PGGF KP KPVGNNKIGPV JWOCPOCEJKPG EQOOWPKECVKQPU CU YGNN CU VJG
ETGCVKQP QH OCEJKPG WPFGTUVCPFCDNG OGVCFCVC HQT 9GD EQPVGPV CPF QVJGT KPHQTOCVKQP

UQWTEGU 6JKU JCPFDQQM KU VQ ſNN VJG PGGF QH C U[UVGOCVKE CPF WRVQFCVG RTGUGPVCVKQP
QH PGY RCVVGTP TGEQIPKVKQP CRRTQCEJGU KP URGGEJ CPF NCPIWCIG RTQEGUUKPI
6JG DQQM UVCTVU YKVJ HWPFCOGPVCNU CPF TGEGPV VJGQTGVKECN CFXCPEGU KP RCVVGTP TGEQI
PKVKQP YKVJ CP GORJCUKU QP ENCUUKſGT FGUKIP ETKVGTKC CPF QRVKOK\CVKQP RTQEGFWTGU
+V EQXGTU UGXGTCN TGEGPV TGUGCTEJ CFXCPEGU KP VJKU CTGC UWEJ CU VJG OKPKOWO GTTQT
TCVG /%' OGVJQF VJG OKPKOWO $C[GU TKUM CRRTQCEJ CFCRVKXG U[UVGO FGUKIP CPF
FGEKUKQP TWNGU PGWTCN PGVYQTMU FKUVTKDWVGF TGEQIPK\GTU CPF FGEKUKQP HWUKQP 6JGUG
OGVJQFU FGRCTV HTQO VJG EQPXGPVKQPCN RCTCFKIO YJKEJ NKPMU C ENCUUKſGT FGUKIP VQ
VJG ENCUUKECN RTQDNGO QH FKUVTKDWVKQP GUVKOCVKQP +PUVGCF OQTG OGCPKPIHWN ETKVGTKC
CTG KPVTQFWEGF YJKEJ UKIPKſECPVN[ KORTQXG VJG FKUETKOKPCVKQP RQYGT QH C ENCUUKſGT
RCTVKEWNCTN[ YJGP CRRNKGF VQ URGGEJ RTQDNGOU KP YJKEJ VJG PQVKQP QH FCVC FKUVTKDWVKQP
KU FKHſEWNV VQ TGCNK\G
6JG UGEQPF RCTV QH VJG DQQM KU VJGTGHQTG URGEKCNN[ HQEWUGF QP VJG CRRTQCEJGU CPF
OGVJQFU CRRNKGF VQ URGGEJ RTQEGUUKPI +V EQXGTU VQRKEU UWEJ CU $C[GU OKPKOWO TKUM
CRRTQCEJ VQ URGGEJ TGEQIPKVKQP NCTIG XQECDWNCT[ URGGEJ TGEQIPKVKQP DCUGF QP UVCVKU
VKECN OGVJQFU TGEQIPKVKQP QH URQPVCPGQWU URGGEJ KP FKCNQIWG KPVGTCEVKQP URGGEJ CPF
URGCMGT XGTKſECVKQP CPF CWFKQ KPHQTOCVKQP TGVTKGXCN CPF KPFGZKPI 6JGUG EJCRVGTU
RTQXKFG C EQORTGJGPUKXG EQXGTCIG QH TGEGPV CFXCPEGU KP CRRN[KPI RCVVGTP TGEQIPK
VKQP VQ TGCN U[UVGOU KP URGGEJ CPF CWFKQ RTQEGUUKPI
6JG VJKTF RCTV QH VJG DQQM KU FGXQVGF VQ VQRKEU QH RCVVGTP TGEQIPKVKQP KP NCPIWCIG
RTQEGUUKPI +V EQPVCKPU EJCRVGTU KP NCPIWCIG OQFGNKPI DCUGF QP NCVGPV UGOCPVKE KP
FGZKPI UCNKGPV KPHQTOCVKQP TGRTGUGPVCVKQP CPF RTQEGUUKPI KP PCVWTCN NCPIWCIG FKC
NQIWG U[UVGO UVCVKUVKECN OCEJKPG VTCPUNCVKQP OGVJQFU KP VQRKE FGVGEVKQP VTCEMKPI
CPF PCOG KFGPVKV[ KFGPVKſECVKQP 6JGUG VQRKEU CTG PGY VTGPFU KP NCPIWCIG RTQEGUU
KPI CPF UKIPKſECPV RTQITGUU JCU DGGP OCFG KP TGEGPV [GCTU +V JCU C FKTGEV KORCEV VQ
VJG RTCEVKEG CPF KORNGOGPVCVKQP QH KPHQTOCVKQP RTQEGUUKPI U[UVGOU HQT 9GD EQPVGPV
DTQCFECUV PGYU CPF QVJGT EQPVGPVTKEJ KPHQTOCVKQP TGUQWTEGU
6JKU DQQM KU C EQNNGEVKXG GHHQTV OQVKXCVGF D[ VJG GZEKVGOGPV QH VJG PGY CFXCPEGU KP
E\&5&3UHVV//&
VJKU ſGNF CPF VJG WTIGPV PGGF VQ DTKPI VJGUG CFXCPEGU VQ C IGPGTCN CWFKGPEG 6JG EQP

VTKDWVKPI CWVJQTU QH VJKU DQQM CTG NGCFKPI GZRGTVU KP VJG ſGNF QH URGGEJ CPF NCPIWCIG
RTQEGUUKPI #VVGORVU CTG OCFG VQ OCMG GCEJ EJCRVGT UGNHEQPVCKPGF CPF EQORTG
JGPUKDNG HQT TGCFGTU YKVJ IGPGTCN DCEMITQWPF KP RCVVGTP TGEQIPKVKQP CPF KPHQTOCVKQP
RTQEGUUKPI +V KU KPVGPFGF VQ DG C JCPFDQQM QT TGHGTGPEG VGZVDQQM HQT TGUGCTEJGTU
ITCFWCVG UVWFGPVU CPF CFXCPEGF WPFGTITCFWCVG UVWFGPVU YJQ YCPV VQ HQNNQY VJG PGY
CFXCPEGU KP RCVVGTP TGEQIPKVKQP 5WHſEKGPV TGHGTGPEGU CTG RTQXKFGF CV VJG GPF QH GCEJ
EJCRVGT VQ UGTXG CU CP GPVT[ RQKPV HQT CP KPVGTGUVGF TGCFGT VQ RWTUWG HWTVJGT
9G YQWNF NKMG VQ VJCPM CNN EQPVTKDWVQTU QH VJKU DQQM 9KVJQWV VJGKT EQOOKVOGPV CPF
SWCNKV[ QH YQTM VJKU DQQM YQWNF PQV DG RQUUKDNG 9G CRRTGEKCVG VJG UWRRQTV CPF
GPEQWTCIGOGPV HTQO QWT EQNNGCIWGU CV #XC[C .CDU 4GUGCTEJ FWTKPI VJG RTGRCTCVKQP
QH VJKU DQQM +V YCU C RNGCUCPV YQTMKPI GZRGTKGPEG YKVJ %4% 2TGUU  VJGKT VGEJPKECN
UWRRQTV YCU XGT[ JGNRHWN VQ WU
9W %JQW
$KKPI*YCPI ,WCPI
Basking Ridge, New Jersey
September, 2002
E\&5&3UHVV//&
Contributors
A. Abella
5RGGEJ 4GUGCTEJ
#66 .CDQTCVQTKGU
(NQTJCO 2CTM 0,
James Allan
&GRCTVOGPV QH %QORWVGT 5EKGPEG
7PKXGTUKV[ QH /CUUCEJWUGVVU #OJGTUV
#OJGTUV /#
T. Alonso
5RGGEJ 4GUGCTEJ
#66 .CDQTCVQTKGU
(NQTJCO 2CTM 0,

Jerome R. Bellegarda
5RQMGP .CPIWCIG )TQWR
#RRNG %QORWVGT +PE
%WRGTVKPQ %#
William Byrne
%GPVGT HQT .CPIWCIG CPF 5RGGEJ
2TQEGUUKPI
,QJPU *QRMKPU 7PKXGTUKV[
$CNVKOQTG /&
Wu Chou
#XC[C .CDU 4GUGCTEJ
$CUMKPI 4KFIG 0,
Sadaoki Furui
&GRCTVOGPV QH %QORWVGT 5EKGPEG
6QM[Q +PUVKVWVG QH 6GEJPQNQI[
6QM[Q ,CRCP
Jean-Luc Gauvain
.+/5+%045
7PKXGTUKVyG FG 2CTKU 5WF
1TUC[ %GFGZ (TCPEG
Vaibhava Goel
6, 9CVUQP 4GUGCTEJ %GPVGT
+$/
;QTMVQYP *GKIJVU 0;
Allen L. Gorin
5RGGEJ 4GUGCTEJ
#66 .CDQTCVQTKGU
(NQTJCO 2CTM 0,
Qiang Huo
&GRCTVOGPV QH %QORWVGT 5EKGPEG CPF

+PHQTOCVKQP 5[UVGOU
6JG 7PKXGTUKV[ QH *QPI -QPI
*QPI -QPI %JKPC
Biing-Hwang Juang
#XC[C .CDU 4GUGCTEJ
$CUMKPI 4KFIG 0,
Shigeru Katagiri
+PVGNNKIGPV %QOOWPKECVKQP .CDQTCVQT[CPF
5RGGEJ 1RGP .CDQTCVQT[
0KRRQP 6GNGITCRJ CPF 6GNGRJQPG
%QTRQTCVKQP
6QM[Q ,CRCP
Lori Lamel
.+/5+%045
7PKXGTUKVyG FG 2CTKU 5WF
1TUC[ %GFGZ (TCPEG
Qi (Peter) Li
$GNN .CDQTCVQTKGU
E\&5&3UHVV//&
.WEGPV 6GEJPQNQIKGU
/WTTC[ *KNN 0,
John Makhoul
$$0 6GEJPQNQIKGU
%CODTKFIG /#
Hermann Ney
.GJTUVWJN HWGT +PHQTOCVKM 8+
*WOCP .CPIWCIG 6GEJPQNQI[ CPF
2CVVGTP 4GEQIPKVKQP
%QORWVGT 5EKGPEG &GRCTVOGPV
7PKXGTUKV[ QH 6GEJPQNQI[

#CEJGP )GTOCP[
F. J. Och
.GJTUVWJN HWGT +PHQTOCVKM 8+
*WOCP .CPIWCIG 6GEJPQNQI[ CPF
2CVVGTP 4GEQIPKVKQP
%QORWVGT 5EKGPEG &GRCTVOGPV
7PKXGTUKV[ QH 6GEJPQNQI[
#CEJGP )GTOCP[
G. Riccardi
5RGGEJ 4GUGCTEJ
#66 .CDQTCVQTKGU
(NQTJCO 2CTM 0,
Richard M. Schwartz
$$0 6GEJPQNQIKGU
%CODTKFIG /#
J. H. Wright
5RGGEJ 4GUGCTEJ
#66 .CDQTCVQTKGU
(NQTJCO 2CTM 0,
E\&5&3UHVV//&
Contents
1 Minimum Classification Error (MCE) Approach in Pattern Recognition
WuChou
#XC[C.CDU4GUGCTEJ#XC[C+PE75#
 +PVTQFWEVKQP
1RVKOCN%NCUUKſGTHTQO$C[GU&GEKUKQP6JGQT[
 &KUETKOKPCPV (WPEVKQP #RRTQCEJ VQ %NCUUKſGT &GUKIP
 5RGGEJ 4GEQIPKVKQP CPF *KFFGP /CTMQX /QFGNKPI
 *KFFGP /CTMQX /QFGNKPI QH 5RGGEJ
 /%' %NCUUKſGT &GUKIP 7UKPI &KUETKOKPCPV (WPEVKQPU

 /%' %NCUUKſGT &GUKIP 5VTCVGI[
 1RVKOK\CVKQP /GVJQFU
 1VJGT 1RVKOK\CVKQP /GVJQFU
 *// CU C &KUETKOKPCPV (WPEVKQP
 4GNCVKQP DGVYGGP /%' CPF //+
 &KUEWUUKQPU CPF %QOOGPVU
 'ODGFFGF 5VTKPI /QFGN $CUGF /%' 6TCKPKPI
 5VTKPI /QFGN $CUGF /%' #RRTQCEJ
 %QODKPGF 5VTKPI /QFGN $CUGF /%' #RRTQCEJ
 &KUETKOKPCVKXG (GCVWTG 'ZVTCEVKQP
 8GTKſECVKQP CPF +FGPVKſECVKQP
 5RGCMGT 8GTKſECVKQP CPF +FGPVKſECVKQP
 7VVGTCPEG 8GTKſECVKQP
 5WOOCT[
2 Minimum Bayes-Risk Methods in Automatic Speech Recognition
VaibhavaGoel
andWilliamByrne +$/ ,QJPU*QRMKPU7PKXGTUKV[
 /KPKOWO $C[GU4KUM %NCUUKſECVKQP (TCOGYQTM
 .KMGNKJQQF 4CVKQ $CUGF *[RQVJGUKU 6GUVKPI
 /CZKOWO #2QUVGTKQTK 2TQDCDKNKV[ %NCUUKſECVKQP
 2TGXKQWU 5VWFKGU QH #RRNKECVKQP 5GPUKVKXG #54
 2TCEVKECN /$4 2TQEGFWTGU HQT #54
 5WOOCVKQP QXGT *KFFGP 5VCVG 5GSWGPEGU
 /$4 4GEQIPKVKQP YKVJ 0DGUV .KUVU
 /$4 4GEQIPKVKQP YKVJ .CVVKEGU
 5GIOGPVCN /$4 2TQEGFWTGU
 5GIOGPVCN 8QVKPI
 418'4
E\&5&3UHVV//&
 G418'4

 'ZRGTKOGPVCN 4GUWNVU
 2CTCOGVGT 6WPKPI YKVJKP VJG /$4 %NCUUKſECVKQP 4WNG
 7VVGTCPEG .GXGN /$4 9QTF CPF -G[YQTF 4GEQIPKVKQP
 418'4 CPF G418'4 HQT /WNVKNKPIWCN #54
 5WOOCT[
 #EMPQYNGFIGOGPVU
3 A Decision Theoretic Formulation for Robust Automatic Speech Recog-
nition
QiangHuo
6JG7PKXGTUKV[QH*QPI-QPI*QPI-QPI%JKPC
 +PVTQFWEVKQP
 1RVKOCN $C[GUŏ &GEKUKQP 4WNG HQT #54
 #FCRVKXG &GEKUKQP 4WNGU %QPUVTWEVGF HTQO 6TCKPKPI 5CORNGU
 2NWIKP $C[GUŏ &GEKUKQP 4WNGU YKVJ /CZKOWONKMGNKJQQF
&GPUKV[ 'UVKOCVG
 /CZKOWO&KUETKOKPCPV &GEKUKQP 4WNGU /KPKOK\KPI VJG 'O
RKTKECN %NCUUKſECVKQP 'TTQT
 &KUEWUUKQP
 8KQNCVKQPU QH /QFGNKPI #UUWORVKQPU KP #54
 6[RGU QH &KUVQTVKQPU
 6QYCTFU #FCRVKXG CPF 4QDWUV #54
 +ORTQXKPI #FCRVKXG &GEKUKQP 4WNGU XKC &GEKUKQP 2CTCOGVGT #FCRVC
VKQP
 &GEKUKQP 2CTCOGVGT #FCRVCVKQP HQT 5VCVKQPCT[ 1RGTCVKPI %QP
FKVKQPU
 &GEKUKQP 2CTCOGVGT #FCRVCVKQP HQT 5NQYN[ %JCPIKPI 1RGT
CVKPI %QPFKVKQPU
 &GEKUKQP 2CTCOGVGT #FCRVCVKQP HQT 5YKVEJKPI 1RGTCVKPI %QP
FKVKQPU
 &KUEWUUKQP

 4QDWUV &GEKUKQP 4WNGU
 &GEKUKQP 4WNG 4QDWUVPGUU
 /KPKOCZ %NCUUKſECVKQP 4WNG
 $C[GUKCP 2TGFKEVKXG %NCUUKſECVKQP 4WNG
 &KUEWUUKQP
 5WOOCT[
4 Speech Pattern Recognition using Neural Networks
ShigeruKatagiri
066%QOOWPKECVKQP5EKGPEG.CDQTCVQTKGU
 +PVTQFWEVKQP
 $C[GU &GEKUKQP 6JGQT[
 2TGRCTCVKQPU
 &GEKUKQP 4WNG
 /KPKOWO 'TTQTTCVG %NCUUKſECVKQP
E\&5&3UHVV//&
 2TQDCDKNKV[ (WPEVKQP 'UVKOCVKQP
 &KUETKOKPCVKXG 6TCKPKPI
 5RGGEJ 4GEQIPK\GTU $CUGF QP 0GWTCN 0GVYQTMU
 2TGRCTCVKQPU
 %NCUUKſECVKQP 'TTQT /KPKOK\CVKQP
 5SWCTGF 'TTQT /KPKOK\CVKQP
 %TQUU 'PVTQR[ /KPKOK\CVKQP
 (WUKQP QH /WNVKRNG %NCUUKſECVKQP &GEKUKQPU
 2TKPEKRNGU
 'ZCORNGU QH 'ODQFKOGPV
 %QPENWFKPI 4GOCTMU
#RRGPFKZ/CZKOK\KPI/WVWCN+PHQTOCVKQP
5 Large Vocabulary Speech Recognition Based on Statistical Methods
Jean-LucGauvainandLoriLamel
.+/5+(TCPEG

 +PVTQFWEVKQP
 1XGTXKGY
 .CPIWCIG /QFGNKPI
 6GZV 2TGRCTCVKQP
 8QECDWNCT[ 5GNGEVKQP
 0ITCO 'UVKOCVKQP
 ./ #FCRVCVKQP
 2TQPWPEKCVKQP /QFGNKPI
 #EQWUVKE /QFGNKPI
 #EQWUVKE (TQPVGPF
 /QFGNKPI #NNQRJQPGU
 *// 2CTCOGVGT 'UVKOCVKQP
 *// #FCRVCVKQP
 &GEQFKPI
 5RGGEJ0QPURGGEJ &GVGEVKQP
 &GEQFKPI 5VTCVGIKGU
 'HſEKGPE[
 %QPſFGPEG /GCUWTGU
 +PFKECVKXG 2GTHQTOCPEG .GXGNU
 &KEVCVKQP
 5RGGEJ 4GEQIPKVKQP HQT &KCNQI 5[UVGOU
6TCPUETKRVKQPHQT#WFKQ+PFGZCVKQP
 2QTVCDKNKV[ CPF .CPIWCIG &GRGPFGPEKGU
6TowardSpontaneousSpeechRecognitionandUnderstanding
SadaokiFurui
6QM[Q+PUVKVWVGQH6GEJPQNQI[
 +PVTQFWEVKQP
 (QWT %CVGIQTKGU QH 5RGGEJ 4GEQIPKVKQP 6CUMU
 5RQPVCPGQWU 5RGGEJ 4GEQIPKVKQP CPF 7PFGTUVCPFKPI  4GXKGY
 %CVGIQT[ + JWOCPVQJWOCP FKCNQIWG

E\&5&3UHVV//&
 %CVGIQT[ ++ JWOCPVQJWOCP OQPQNQIWG
 %CVGIQT[ +++ JWOCPVQOCEJKPG FKCNQIWG
 ,CRCPGUG 0CVKQPCN 2TQLGEV QP 5RQPVCPGQWU 5RGGEJ %QTRWU CPF 2TQ
EGUUKPI 6GEJPQNQI[
 2TQLGEV 1XGTXKGY
 %QTRWU
 #WVQOCVKE 6TCPUETKRVKQP QH 5RQPVCPGQWU 2TGUGPVCVKQP
 4GEQIPKVKQP 6CUM
 .CPIWCIG CPF #EQWUVKE /QFGNKPI
 4GEQIPKVKQP 4GUWNVU
 #PCN[UKU QP +PFKXKFWCN &KHHGTGPEGU
 &KUEWUUKQP
 #WVQOCVKE 5RGGEJ 5WOOCTK\CVKQP CPF 'XCNWCVKQP
 5WOOCTK\CVKQP QH 'CEJ 5GPVGPEG 7VVGTCPEG
 5WOOCTK\CVKQP QH /WNVKRNG 7VVGTCPEGU
 'XCNWCVKQP
 &KUEWUUKQP
 5RQPVCPGQWU 5RGGEJ 4GEQIPKVKQP CPF 7PFGTUVCPFKPI 4GUGCTEJ +U
UWGU
 .CPIWCIG /QFGNU CPF %QTRQTC
 /GUUCIGFTKXGP 5RGGEJ 4GEQIPKVKQP CPF 7PFGTUVCPFKPI
 5VCVKUVKECN #RRTQCEJGU CPF 5RGGEJ 5EKGPEG
 4GUGCTEJ QP VJG *WOCP $TCKP
 &[PCOKE 5RGEVTCN (GCVWTGU
 %QPENWUKQP
7 Speaker Authentication
QiLi
andBiing-HwangJuang $GNN.CDU #XC[C.CDU4GUGCTEJ
 +PVTQFWEVKQP

 5RGCMGT 4GEQIPKVKQP CPF 8GTKſECVKQP
 8GTDCN +PHQTOCVKQP 8GTKſECVKQP
 2CVVGTP 4GEQIPKVKQP KP 5RGCMGT #WVJGPVKECVKQP
 $C[GUKCP &GEKUKQP 6JGQT[
 5VQEJCUVKE /QFGNU HQT 5VCVKQPCT[ 2TQEGUU
 5VQEJCUVKE /QFGNU HQT 0QP5VCVKQPCT[ 2TQEGUU
 5RGGEJ 5GIOGPVCVKQP
 5VCVKUVKECN 8GTKſECVKQP
 5RGCMGT 8GTKſECVKQP 5[UVGO
 8GTDCN +PHQTOCVKQP 8GTKſECVKQP
 7VVGTCPEG 5GIOGPVCVKQP
 5WDYQTF *[RQVJGUKU 6GUVKPI
 %QPſFGPEG /GCUWTG %CNEWNCVKQP
 5GSWGPVKCN 7VVGTCPEG 8GTKſECVKQP
 8+8 'ZRGTKOGPVCN 4GUWNVU
 5RGCMGT #WVJGPVKECVKQP D[ %QODKPKPI 58 CPF 8+8
E\&5&3UHVV//&
 5WOOCT[
8 HMMs for Language Processing Problems
RichardM.SchwartzandJohnMakhoul
$$06GEJPQNQIKGU8GTK\QP
 +PVTQFWEVKQP
 7UG QH 2TQDCDKNKVKGU
 *KFFGP /CTMQX /QFGNU
 0COG 5RQVVKPI
 6QRKE %NCUUKſECVKQP
 6JG /QFGN
 'UVKOCVKPI *// 2CTCOGVGTU
 %NCUUKſECVKQP
 'ZRGTKOGPVU

 +PHQTOCVKQP 4GVTKGXCN
 # $C[GUKCP /QFGN HQT +4
 6TCKPKPI VJG +4 *//
 2GTHQTOCPEG
 'XGPV 6TCEMKPI
 7PUWRGTXKUGF 6QRKE &GVGEVKQP
 5WOOCT[
9 Statistical Language Models With Embedded Latent Semantic Knowl-
edge
JeromeR.Bellegarda
#RRNG%QORWVGT+PE
 +PVTQFWEVKQP
 5EQRG .QECNKV[
 5[PVCEVKECNN[Ō&TKXGP 5RCP 'ZVGPUKQP
 5GOCPVKECNN[Ō&TKXGP 5RCP 'ZVGPUKQP
 1TICPK\CVKQP
 .CVGPV 5GOCPVKE #PCN[UKU
 (GCVWTG 'ZVTCEVKQP
 5KPIWNCT 8CNWG &GEQORQUKVKQP
 )GPGTCN $GJCXKQT
 .5# (GCVWTG 5RCEG
 9QTF %NWUVGTKPI
 9QTF %NWUVGT 'ZCORNG
 &QEWOGPV %NWUVGTKPI
 &QEWOGPV %NWUVGT 'ZCORNG
 5GOCPVKE %NCUUKſECVKQP
 (TCOGYQTM 'ZVGPUKQP
 5GOCPVKE +PHGTGPEG
 %CXGCVU
 0ITCO.5# .CPIWCIG /QFGNKPI

 .5# %QORQPGPV
 +PVGITCVKQP YKVJ 0ITCOU
E\&5&3UHVV//&
 %QPVGZV 5EQRG 5GNGEVKQP
 5OQQVJKPI
 9QTF 5OQQVJKPI
 &QEWOGPV 5OQQVJKPI
 ,QKPV 5OQQVJKPI
 'ZRGTKOGPVU
 'ZRGTKOGPVCN %QPFKVKQPU
 'ZRGTKOGPVCN 4GUWNVU
 %QPVGZV 5EQRG 5GNGEVKQP
 +PJGTGPV 6TCFG1HHU
 %TQUU&QOCKP 6TCKPKPI
 &KUEWUUKQP
 %QPENWUKQP
10SemanticInformationProcessingofSpokenLanguage–HowMayI
Help You?
sm
A.L.Gorin,A.Abella,T.Alonso,G.Riccardi,andJ.H.Wright,
#66.CDQTCVQ
TKGU
 +PVTQFWEVKQP
 %CNN%NCUUKſECVKQP
.CPIWCIG/QFGNKPIHQT4GEQIPKVKQPCPF7PFGTUVCPFKPI
 &KCNQI
 %QPENWUKQPU
11 Machine Translation Using Statistical Modeling
HermanNey,andF.J.Och
#CEJGP7PKXGTUKV[QH6GEJPQNQI[)GTOCP[

 +PVTQFWEVKQP
 5VCVKUVKECN &GEKUKQP 6JGQT[ CPF .KPIWKUVKEU
 6JG 5VCVKUVKECN #RRTQCEJ
 $C[GU &GEKUKQP 4WNG HQT 9TKVVGP .CPIWCIG 6TCPUNCVKQP
 4GNCVGF #RRTQCEJGU
 #NKIPOGPV CPF .GZKEQP /QFGNU
 %QPEGRV QH #NKIPOGPV /QFGNNKPI
 *KFFGP /CTMQX /QFGNU
 /QFGNU +$/ Ō
 6TCKPKPI
 5GCTEJ
 #NIQTKVJOKE &KHHGTGPEGU DGVYGGP 5RGGEJ 4GEQIPKVKQP CPF
.CPIWCIG 6TCPUNCVKQP
 #NKIPOGPV 6GORNCVGU (TQO 5KPING 9QTFU VQ 9QTF )TQWRU
 %QPEGRV
 6TCKPKPI
 5GCTEJ
 'ZRGTKOGPVCN 4GUWNVU
 6JG 6CUM CPF VJG %QTRWU
E\&5&3UHVV//&
 1HƀKPG 4GUWNVU
 +PVGITCVKQP KPVQ VJG 8
'4$/1$+. 2TQVQV[RG 5[UVGO
 (KPCN 'XCNWCVKQP
 5RGGEJ 6TCPUNCVKQP 6JG +PVGITCVGF #RRTQCEJ
 2TKPEKRNG
 2TCEVKECN +ORNGOGPVCVKQP
 5WOOCT[
 4GHGTGPEGU
12 Modeling Topics for Detection and Tracking

JamesAllan
7PKXGTUKV[QH/CUUCEJWUGVVU#OJGTUV
 6QRKE &GVGEVKQP CPF 6TCEMKPI
 6QRKE CPF 'XGPVU
 6&6 6CUMU
 %QTRQTC
 'XCNWCVKQP
 $CUKE 6QRKE /QFGNU
 8GEVQT 5RCEG
 .CPIWCIG /QFGNU
 +ORNGOGPVKPI VJG /QFGNU
 0COGF 'PVKVKGU
 &QEWOGPV 'ZRCPUKQP
 %NWUVGTKPI
 6KOG &GEC[
 %QORCTKPI /QFGNU
 0GCTGUV 0GKIJDQTU
 &GEKUKQP 6TGGU
 /QFGNVQ/QFGN
 /KUEGNNCPGQWU +UUWGU
 &GHGTTCN
 /WNVKOQFCN +UUWGU
 /WNVKNKPIWCN +UUWGU
 7UKPI 6&6 +PVGTCEVKXGN[
 &GOQPUVTCVKQPU
 6KOGNKPGU
 /QFGNKPI 'XGPVU
 %QPENWUKQP
E\&5&3UHVV//&
1

Minimum Classification Error (MCE)
Approach in Pattern Recognition
Wu Chou
Avaya Labs Research, Avaya Inc., USA
CONTENTS
+PVTQFWEVKQP
1RVKOCN%NCUUKſGTHTQO$C[GU&GEKUKQP6JGQT[
&KUETKOKPCPV(WPEVKQP#RRTQCEJVQ%NCUUKſGT&GUKIP
5RGGEJ4GEQIPKVKQPCPF*KFFGP/CTMQX/QFGNKPI
/%'%NCUUKſGT&GUKIP7UKPI&KUETKOKPCPV(WPEVKQPU
'ODGFFGF5VTKPI/QFGN$CUGF/%'6TCKPKPI
8GTKſECVKQPCPF+FGPVKſECVKQP
5WOOCT[
#EMPQYNGFIGOGPV
4GHGTGPEGU
1.1 Introduction
2CVVGTP TGEQIPKVKQP KU C HCUV OQXKPI TGUGCTEJ CTGC 6JG CFXGPV QH RQYGTHWN EQORWV
KPI FGXKEGU CPF VJG UWEEGUU QH UVCVKUVKECN CRRTQCEJGU UWEJ CU JKFFGP /CTMQX OQFGN
HQT URGGEJ CPF NCPIWCIG RTQEGUUKPI VTKIIGTGF C TGPGYGF RWTUWKV HQT OQTG RQYGTHWN
UVCVKUVKECN OGVJQFU VQ HWTVJGT TGFWEG VJG RCVVGTP TGEQIPKVKQP GTTQT TCVG CPF KORTQXG
VJG TQDWUVPGUU QH VJG RCVVGTP ENCUUKſGT CETQUU XCTKQWU CFXGTUG EQPFKVKQPU #OQPI
VJKU PGY RWTUWKV VJG WUG QH FKUETKOKPCPV HWPEVKQP OGVJQFU KP RCVVGTP TGEQIPKVKQP
JCU GOGTIGF CU C RTQOKUKPI CRRTQCEJ CPF KV KU CRRNKGF UWEEGUUHWNN[ VQ URGGEJ CPF
NCPIWCIG RTQEGUUKPI 6JKU EJCRVGT KU KPVGPFGF VQ RTQXKFG C TGXKUKV VQ VJG UVCVKUVKECN
HQTOWNCVKQP QH VJG OKPKOWO ENCUUKſECVKQP GTTQT /%' DCUGF FKUETKOKPCVKXG OGVJ
QFU KP URGGEJ CPF NCPIWCIG RTQEGUUKPI VCMG C ETKVKECN XKGY QH VJG CRRTQCEJ RTQXKFG
C EQORTGJGPUKXG QXGTXKGY QH VJG ſGNF CPF JQRGHWNN[ KPURKTG QVJGT KPPQXCVKQPU VJCV
YQWNF RQVGPVKCNN[ NGCF VQ PGY FKUETKOKPCVKXG OGVJQFU KP RCVVGTP TGEQIPKVKQP
#NVJQWIJ VJG UVCVKUVKECN HQTOWNCVKQP QH /%' DCUGF FKUETKOKPCVKXG OGVJQFU JCU KVU
TQQV KP VJG ENCUUKECN $C[GU FGEKUKQP VJGQT[ KV FGRCTVU HTQO VJG EQPXGPVKQPCN RCTCFKIO

6JKU EJCRVGT KU FGXGNQRGF DCUGF QP Œ&KUETKOKPCPVHWPEVKQPDCUGF OKPKOWO TGEQIPKVKQP GTTQT TCVG RCVVGTPTGEQIPKVKQP
CRRTQCEJ VQ URGGEJ TGEQIPKVKQPŒ D[ 9W %JQW CRRGCTGF KP Proceedings of The IEEE 8QN  0Q 
E
 +'''
E\&5&3UHVV//&
YJKEJ NKPMU C TGEQIPKVKQP VCUM VQ VJG RTQDNGO QH FKUVTKDWVKQP GUVKOCVKQP +PUVGCF KV
VCMGU C FKUETKOKPCPV HWPEVKQP DCUGF UVCVKUVKECN RCVVGTP ENCUUKſECVKQP CRRTQCEJ CPF HQT
C IKXGP HCOKN[ QH FKUETKOKPCPV HWPEVKQP QRVKOCN ENCUUKſGTTGEQIPK\GT FGUKIP KPXQNXGU
ſPFKPI C UGV QH RCTCOGVGTU YJKEJ OKPKOK\G VJG GORKTKECN RCVVGTP TGEQIPKVKQP GTTQT
TCVG 6JG WUG QH FKUETKOKPCPV HWPEVKQP KP RCVVGTP TGEQIPKVKQP YCU UVCTVGF OCP[ [GCTU
CIQ 1PG ENCUUKECN GZCORNG QH WUKPI FKUETKOKPCPV HWPEVKQP HQT ENCUUKſGT FGUKIP KP
UVCVKUVKECN NKVGTCVWTG KU VJG VYQ ENCUU ENCUUKſECVKQP RTQDNGO WUKPI NKPGCT FKUETKOKPCPV
HWPEVKQPU = ? +P RCTVKEWNCT C YKPFQY DCUGF OGVJQF YCU FGUETKDGF KP =? HQT
VJG VYQ ENCUU ENCUUKſECVKQP RTQDNGO WUKPI NKPGCT FKUETKOKPCPV HWPEVKQPU VJCV OKPKOK\G
VJG RTQDCDKNKV[ QH ENCUUKſECVKQP GTTQT TCVG 6JG HQEWU QH VJKU EJCRVGT KU QP VJG TGEGPV
FGXGNQROGPV QH VJG IGPGTCN /%' DCUGF FKUETKOKPCVKXG OGVJQFU 6JG FKUETKOKPCPV
HWPEVKQPU VJCV YG GPEQWPVGT CTG WUWCNN[ PQPNKPGCT CPF QHVGP TGNCVGF VQ VJG UVTWEVWTG
QH VJG UVCVKUVKECN HTCOGYQTM WUGF KP URGGEJ CPF NCPIWCIG RTQEGUUKPI UWEJ CU JKFFGP
/CTMQX OQFGNU
6JG TGCUQP QH VCMKPI C FKUETKOKPCPV HWPEVKQP DCUGF CRRTQCEJ VQ ENCUUKſGT FGUKIP CU
YKNN DG HWTVJGT GNCDQTCVGF KU FWG OCKPN[ VQ VJG HCEV VJCV YG NCEM EQORNGVG MPQYNGFIG
QH VJG HQTO QH VJG FCVC FKUVTKDWVKQP CPF VTCKPKPI FCVC CTG KPCFGSWCVG RCTVKEWNCTN[ KP
FGCNKPI YKVJ URGGEJ CPF NCPIWCIG RTQDNGOU 6JG RGTHQTOCPEG QH C TGEQIPK\GTKU
PQTOCNN[ FGſPGF D[ KVU GZRGEVGF TGEQIPKVKQP GTTQT TCVG CPF CP QRVKOCN TGEQIPK\GT
UJQWNF DG VJG QPG VJCV CEJKGXGU VJG NGCUV GZRGEVGF TCVG QH TGEQIPKVKQP GTTQT 6JG
FKHHGTGPEG DGVYGGP VJG FKUVTKDWVKQP GUVKOCVKQP DCUGF CRRTQCEJ CPF VJG FKUETKOKPCPV
HWPEVKQP DCUGF /%' CRRTQCEJ NKGU KP VJG YC[ VJG TGEQIPKVKQP GTTQT KU GZRTGUUGF
CPF KP VJG EQORWVCVKQPCN UVGRU VJCV YQWNF NGCF VQ VJG OKPKOK\CVKQP QH UWEJ GTTQT
HWPEVKQPU # MG[ VQ VJG FGXGNQROGPV QH VJG /%' OGVJQF KU C PGY GTTQT HWPEVKQP
YJKEJ KPEQTRQTCVGU VJG TGEQIPKVKQP QRGTCVKQP CPF RGTHQTOCPEG KP C HWPEVKQPCN HQTO

HTQO YJKEJ VJG RGTHQTOCPEG QH VJG ENCUUKſGT ECP DG FKTGEVN[ GXCNWCVGF CPF QRVKOK\GF
%NCUUKſGT FGUKIP YKVJQWV CUUWOKPI VJG MPQYNGFIG QH ENCUU RQUVGTKQT RTQDCDKNKVKGU
YJKEJ CTG VJG DCUKU QH VJG FKUVTKDWVKQP GUVKOCVKQP DCUGF ENCUUKGT FGUKIP JCU DGGP
UVWFKGF KP OCP[ CTGCU +P RCTVKEWNCT 6U[RMKP =? CPF #OCTK = ? RKQPGGTGF VJKU
CRRTQCEJ HQT UGNHNGCTPKPI CPF UGNHQTICPK\KPI PGVU 6JG[ HQTOWNCVGF VJG RTQDNGO QH
UGNHNGCTPKPI KPVQ C ENCUUKſECVKQP RTQDNGO YJKEJ EQPUKUVU QH QRVKOCN RCTVKVKQPKPI QH
VJG QDUGTXCVKQP URCEG KPVQ TGIKQPU
 HQT YJKEJ VJG GZRGEVGF TKUM  KU OKPKOK\GF
+P CFFKVKQP C OCVJGOCVKECN OKPKOK\CVKQP RTQEGFWTG IGPGTCNK\GF RTQDCDKNKUVKE FG
UEGPV )2& CNIQTKVJO QT UVQEJCUVKE CRRTQZKOCVKQP YCU RTQRQUGF CU C OGCPU HQT
ENCUUKſGT FGUKIP WPFGT VJKU HTCOGYQTM 5KPEG VJGP XCTKQWU NQUU HWPEVKQPU JCXG DGGP
WUGF KP FGUKIPKPI ENCUUKſGTU KPENWFKPI VJQUG RQRWNCT OGCPUSWCTG GTTQT DCUGF NQUU
HWPEVKQPU *QYGXGT OCP[ VTCEVCDNG NQUU HWPEVKQPU FQ PQV JCXG C FKTGEV TGNCVKQP VQ
VJG TGEQIPKVKQP GTTQT TCVG OKPKOK\CVKQP CPF VJGTGHQTG CNDGKV DCUGF QP FKUETKOKPCPV
HWPEVKQPU VJG[ CTG PQV FKTGEVN[ TGNCVGF VQ TGEQIPKVKQP GTTQT TCVG YJKEJ UJQWNF DG VJG
OQUV UGPUKDNG EJQKEG HQT ENCUUKſGT FGUKIP
1XGT VJG RCUV FGECFG VJG /%' DCUGF CRRTQCEJ JCU DGGP FGXGNQRGF VQ QXGTEQOG VJG
HWPFCOGPVCN NKOKVCVKQPU QH VJG VTCFKVKQPCN CRRTQCEJ CPF VQ FKTGEVN[ NKPM VJG ENCUUKſGT
FGUKIP RTQDNGO VQ ENCUUKſECVKQP GTTQT TCVG OKPKOK\CVKQP +P QTFGT VQ CNNGXKCVG VJG
FGRGPFGPE[ QP VJG ENCUU RQUVGTKQT FKUVTKDWVKQPU C FKUETKOKPCPV HWPEVKQP DCUGF /%'
CRRTQCEJ YCU RTQRQUGF D[ ,WCPI GV CN =? CU CP CNVGTPCVKXG VQ QRVKOCN ENCUUKſGT FG
E\&5&3UHVV//&
UKIP #NVJQWIJ VJKU CRRTQCEJ CRRNKGU VQ VJG RCVVGTP TGEQIPKVKQP RTQDNGO KP IGPGTCN
KV ſPFU XCTKQWU CRRNKECVKQPU KP URGGEJ CPF NCPIWCIG RTQEGUUKPI +V YCU ſTUV CRRNKGF
VQ F[PCOKE VKOG YCTRKPI DCUGF RCVVGTP TGEQIPKVKQP U[UVGOU = ? #RRNKECVKQP VQ
JKFFGP /CTMQX OQFGN DCUGF EQPVKPWQWU URGGEJ TGEQIPKVKQP U[UVGOU YCU HQTOWNCVGF
CU C UGIOGPVCN CPF UVTKPI OQFGN DCUGF /%' CRRTQCEJ = ? CPF UWEEGUUHWN CR
RNKECVKQPU QH VJKU CRRTQCEJ YGTG TGRQTVGF KP =     ? 6JKU CRRTQCEJ
YCU HWTVJGT GZVGPFGF VQ HQTO C EQODKPGF UVTKPI OQFGN KP YJKEJ VTCKPKPI QH QVJGT
OQFGN EQORQPGPVU KP URGGEJ CPF NCPIWCIG RTQEGUUKPI ECP DG CEJKGXGF WPFGT C WPK

ſGF /%' HTCOGYQTM = ? +V YCU CRRNKGF VQ FKUETKOKPCVKXG OQFGN EQODKPCVKQP
= ? CPF VQ CRRNKECVKQPU KP URGCMGT KFGPVKſECVKQP CPF XGTKſECVKQP =  ?
6JG DCUKE KFGC QH VJG /%' CRRTQCEJ YCU HWTVJGT FGXGNQRGF HQT CRRNKECVKQPU KP WVVGT
CPEG XGTKſECVKQP RTQDNGOU =  ? # IGPGTCN HTCOGYQTM QH EQODKPKPI FG
VGEVKQP CPF XGTKſECVKQP KP URGGEJ TGEQIPKVKQP CPF WPFGTUVCPFKPI YCU CNUQ RTQRQUGF
KP YJKEJ VJG FKUETKOKPCPV HWPEVKQP DCUGF RCVVGTP TGEQIPKVKQP CRRTQCEJ YCU CRRNKGF
KP DQVJ FGVGEVKQP CPF XGTKſECVKQP RTQEGUUGU = ?
9G DGIKP KP VJG PGZV UGEVKQP YKVJ C DTKGH TGXKGY QH VJG $C[GU FGEKUKQP VJGQT[ CPF
KVU CRRNKECVKQP VQ VJG HQTOWNCVKQP QH UVCVKUVKECN RCVVGTP TGEQIPKVKQP RTQDNGO 9G KP
VTQFWEG VJG FKUETKOKPCPV HWPEVKQP DCUGF UVCVKUVKECN RCVVGTP TGEQIPKVKQP CRRTQCEJ KP
5GEVKQP  +P 5GEVKQP  YG RTQXKFG C DTKGH KPVTQFWEVKQP VQ URGGEJ TGEQIPKVKQP CPF
JKFFGP /CTMQX OQFGNKPI 6JG FKUETKOKPCPV HWPEVKQP DCUGF /%' RCVVGTP TGEQIPKVKQP
CRRTQCEJ CPF KVU CRRNKECVKQP VQ *// DCUGF URGGEJ TGEQIPKVKQP U[UVGOU CTG KPVTQ
FWEGF KP 5GEVKQP  %QORCTKUQPU CTG OCFG VQ QVJGT ETKVGTKC KP URGGEJ TGEQIPKVKQP
CPF KP RCTVKEWNCT YG UVWF[ VJG TGNCVKQP DGVYGGP /%' CPF //+ OCZKOWO OWVWCN
KPHQTOCVKQP ETKVGTKC KP ENCUUKſGT FGUKIP KP VJG UGEQPF JCNH QH 5GEVKQP  +P 5GEVKQP 
YG UVWF[ VJG GODGFFGF UVTKPI OQFGN DCUGF /%' CRRTQCEJ CPF KVU GZVGPUKQP VQ VJG
JKIJGT NGXGN EQODKPGF UVTKPI OQFGN 9G FKUEWUU KUUWGU CPF CRRNKECVKQPU KP FKUETKO
KPCVKXG OQFGN EQODKPCVKQP FKUETKOKPCVKXG NCPIWCIG OQFGN GUVKOCVKQP CPF FKUETKO
KPCVKXG HGCVWTG GZVTCEVKQP WPFGT VJG IGPGTCN VJGQTGVKECN HTCOGYQTM QH VJG EQODKPGF
UVTKPI OQFGN 5GEVKQP  KU FGXQVGF VQ CRRNKECVKQPU QH FKUETKOKPCPV HWPEVKQP DCUGF
RCVVGTP TGEQIPKVKQP CRRTQCEJ KP XGTKſECVKQP CPF KFGPVKſECVKQP 6JG FKUETKOKPCPV
HWPEVKQP CRRTQCEJ KU UVWFKGF HQT XCTKQWU CRRNKECVKQPU KP URGGEJ CPF NCPIWCIG RTQ
EGUUKPI UWEJ CU URGCMGT KFGPVKſECVKQP CPF XGTKſECVKQP WVVGTCPEG XGTKſECVKQP TGEQI
PKVKQP DCUGF QP IGPGTCNK\GF EQPſFGPEG OGCUWTGU FGVGEVKQP CPF XGTKſECVKQP DCUGF
CRRTQCEJ KP URGGEJ TGEQIPKVKQP CPF WPFGTUVCPFKPI 6JG EJCRVGT KU UWOOCTK\GF YKVJ
FKUEWUUKQPU KP 5GEVKQP 
1.2 Optimal Classifier from Bayes Decision Theory
(QT CP ENCUU ENCUUKſECVKQP RTQDNGO C ENCUUKſGT KU VQ ENCUUKH[ GCEJ TCPFQO UCORNG
Z KPVQ QPG QH VJG / ENCUUGU 9G FGPQVG VJGUG ENCUUGU D[

  6JG
ENCUUKſGT
FGſPGU C OCRRKPI HTQO VJG UCORNG URCEG VQ VJG FKUETGVG
E\&5&3UHVV//&
ECVGIQTKECN UGV  .GV DG VJG LQKPV RTQDCDKNKV[ FKUVTKDWVKQP QH CPF
 C SWCPVKV[ YJKEJ KU CUUWOGF VQ DG MPQYP VQ VJG FGUKIPGT QH VJG ENCUUKſGT +P QVJGT
YQTFU VJG FGUKIPGT JCU HWNN MPQYNGFIG QH VJG TCPFQO PCVWTG QH VJG UQWTEG (TQO VJG
UGV QH LQKPV RTQDCDKNKV[ FKUVTKDWVKQPU VJG OCTIKPCN CPF VJG EQPFKVKQPCN RTQDCDKNKV[
FKUVTKDWVKQPU ECP DG GCUKN[ ECNEWNCVGF
+P QTFGT VQ EJCTCEVGTK\G VJG RGTHQTOCPEG QH VJG ENCUUKſGT GXGT[ ENCUU RCKT
ECP DG
CUUQEKCVGF YKVJ C EQUV QT NQUU HWPEVKQP
YJKEJ UKIPKſGU VJG EQUV QH ENCUUKH[KPI QT
TGEQIPK\KPI C ENCUU
QDUGTXCVKQP KPVQ C ENCUU GXGPV 6JG NQUU HWPEVKQP KU IGPGTCNN[
PQPPGICVKXG YKVJ
TGRTGUGPVKPI EQTTGEV ENCUUKſECVKQP 6JG NQUU HWPEVKQP KU C
HWPEVKQP HTQO
YJGTG KU VJG UGV QH TGCN PWODGTU +P ENCUUKſECVKQP YG
OCMG C FGEKUKQP
HQT QDUGTXKPI C TCPFQO UCORNG  5KPEG KU VJG ENCUU
RQUVGTKQT RTQDCDKNKV[ VJCV VJG TCPFQO KPRWV
KU HTQO  VJG CXGTCIG NQUU CUUQEKCVGF
YKVJ OCMKPI C FGEKUKQP
ECP DG FGſPGF CU =?

6JKU NGCFU VQ C TGCUQPCDNG RGTHQTOCPEG OGCUWTG HQT VJG ENCUUKſGT KG VJG GZRGEVGF
NQUU FGſPGF CU

YJGTG

TGRTGUGPVU VJG ENCUUKſGTŏU FGEKUKQP CUUWOKPI QPG QH VJG őXCNWGUŒ 
 DCUGF QP C TCPFQO QDUGTXCVKQP FTCYP HTQO C RTQDCDKNKV[ FKUVTKDWVKQP
 6JG FGEKUKQP HWPEVKQP  FGRGPFU QP VJG ENCUUKſGT FGUKIP 1DXKQWUN[ KH
VJG ENCUUKſGT KU UQ FGUKIPGF VJCV HQT GXGT[

VJG GZRGEVGF NQUU KP GSWCVKQP  YKNN DG OKPKOK\GF
(QT OCP[ CRRNKECVKQPU KPENWFKPI URGGEJ TGEQIPKVKQP VJG NQUU HWPEVKQP
KU WUWCNN[
EJQUGP VQ DG VJG \GTQQPG NQUU HWPEVKQP FGſPGF D[

YJKEJ CUUKIPU PQ NQUU VQ EQTTGEV ENCUUKſECVKQP CPF C WPKV NQUU VQ CP[ GTTQT TGICTFNGUU
QH VJG ENCUU 9KVJ VJKU V[RG QH NQUU HWPEVKQP VJG GZRGEVGF NQUU
KU VJWU VJG GTTQT
RTQDCDKNKV[ QH ENCUUKſECVKQP QT TGEQIPKVKQP 6JG EQPFKVKQPCN NQUU DGEQOGU

6JG QRVKOCN ENCUUKſGT VJCV CEJKGXGU OKPKOWO
KU VJWU VJG QPG VJCV KORNGOGPVU VJG
HQNNQYKPI
KH 
E\&5&3UHVV//&
(QT OKPKOWO GTTQT TCVG ENCUUKſECVKQP VJG ENCUUKſGT GORNQ[U VJG FGEKUKQP TWNG QH 
YJKEJ KU ECNNGF VJG őOCZKOWO C RQUVGTKQTŒ /#2 FGEKUKQP 6JG OKPKOWO GTTQT TCVG
CEJKGXGF D[ /#2 FGEKUKQP KU ECNNGF ő$C[GU TKUMŒ 9JGP CNN RQUVGTKQT RTQDCDKNKVKGU
CTG MPQYP VJG ENCUUKſGT DCUGF QP /#2 TWNG KU CP QRVKOCN ENCUUKſGT DCUGF QP VJG
$C[GU FGEKUKQP VJGQT[ *QYGXGT KH VJGUG RTQDCDKNKVKGU CTG PQV MPQYP QT VJG FGEKUKQP
TWNG KU PQV DCUGF QP VJG ENCUU RQUVGTKQT RTQDCDKNKV[ VJGP YG ECPPQV WUG VJKU TGUWNV
FKTGEVN[
+P RTCEVKEG VJGUG RTQDCDKNKVKGU JCXG VQ DG GUVKOCVGF HTQO C VTCKPKPI FCVC UGV YKVJ
MPQYP ENCUU NCDGNU 6JG ENCUUKECN $C[GU FGEKUKQP VJGQT[ VJWU GHHGEVKXGN[ VTCPUHQTOU
VJG ENCUUKſGT FGUKIP RTQDNGO KPVQ C FKUVTKDWVKQP GUVKOCVKQP RTQDNGO 6JKU KU VJG DCUKU

QH VJG $C[GUKCP UVCVKUVKECN CRRTQCEJ VQ RCVVGTP TGEQIPKVKQP YJKEJ ECP DG UVCVGF CU
IKXGP QT EQNNGEV C UGV QH VTCKPKPI FCVC QDUGTXCVKQPU
YKVJ MPQYP
ENCUU NCDGNU GUVKOCVG VJG C RQUVGTKQT RTQDCDKNKVKGU
 HQT CP[
VQ KORNGOGPV VJG OCZKOWO C RQUVGTKQT FGEKUKQP HQT OKPKOWO $C[GU TKUM 6JG C
RQUVGTKQT RTQDCDKNKV[
ECP DG TGYTKVVGP CU

5KPEG KU PQV C HWPEVKQP QH VJG ENCUU KPFGZ CPF VJWU JCU PQ GHHGEV KP VJG /#2
FGEKUKQP VJG PGGFGF RTQDCDKNKUVKE MPQYNGFIG ECP DG TGRTGUGPVGF D[ VJG ENCUU RTKQT
CPF VJG EQPFKVKQPCN RTQDCDKNKV[ 
6JGTG CTG UGXGTCN KUUWGU CUUQEKCVGF YKVJ VJKU ENCUUKECN CRRTQCEJ (KTUV VJG FKUVTKDW
VKQPU WUWCNN[ JCXG VQ DG RCTCOGVGTK\GF KP QTFGT HQT VJGO VQ DG RTCEVKECNN[ WUGHWN HQT
VJG KORNGOGPVCVKQP QH VJG /#2 TWNG QH  6JG ENCUUKſGT FGUKIPGT VJGTGHQTG JCU VQ
FGVGTOKPG VJG TKIJV RCTCOGVTKE HQTO QH VJG FKUVTKDWVKQPU (QT OQUV QH VJG TGCN YQTNF
RTQDNGOU VJKU KU C FKHſEWNV VCUM 1WT EJQKEG QH VJG FKUVTKDWVKQP HQTO KU QHVGP NKO
KVGF D[ VJG OCVJGOCVKECN VTCEVCDKNKV[ QH VJG RCTVKEWNCT FKUVTKDWVKQP HWPEVKQPU CPF KU
XGT[ NKMGN[ VQ DG KPEQPUKUVGPV YKVJ VJG CEVWCN FKUVTKDWVKQP 6JKU OGCPU VJCV VJG VTWG
/#2 FGEKUKQP ECP TCTGN[ DG KORNGOGPVGF CPF VJG OKPKOWO $C[GU TKUM IGPGTCNN[
TGOCKPU CP WPCEJKGXCDNG NQYGT DQWPF 5GEQPF IKXGP C RCTCOGVGTK\GF FKUVTKDWVKQP
HQTO VJG WPMPQYP RCTCOGVGTU FGſPKPI VJG FKUVTKDWVKQP JCXG VQ DG GUVKOCVGF HTQO
C ſPKVG COQWPV QH NCDGNGF VTCKPKPI FCVC TGSWKTKPI VJCV VJG GUVKOCVKQP OGVJQF JCU VQ
DG CDNG VQ RTQFWEG EQPUKUVGPV RCTCOGVGT XCNWGU YJGP VJG UK\G QH VJG VTCKPKPI UCORNGU
XCTKGU 6JKTF KV TGSWKTGU C VTCKPKPI FCVC UGV QH UWHſEKGPV UK\G KP QTFGT VQ JCXG TGNK
CDNG RCTCOGVGT GUVKOCVGU $WV KP RTCEVKEG CPF HQT URGGEJ CPF NCPIWCIG RTQEGUUKPI
KP RCTVKEWNCT VTCKPKPI FCVC CTG CNYC[U URCTUG EQORCTGF VQ CNN RQUUKDNG TGCNK\CVKQPU
CPF XCTKCVKQPU KP JWOCP URGGEJ CPF NCPIWCIG 6JGUG VJTGG DCUKE KUUWGU RQKPV QWV
C HWPFCOGPVCN HCEV VJCV KU FGURKVG VJG EQPEGRVWCN QRVKOCNKV[ QH VJG $C[GU FGEKUKQP
VJGQT[ CPF KVU CRRNKECVKQPU VQ RCVVGTP TGEQIPKVKQP KV ECPPQV CNYC[U DG CEEQORNKUJGF

KP RTCEVKEG DGECWUG OQUV RTCEVKECN ő/#2Œ FGEKUKQPU KP URGGEJ CPF NCPIWCIG RTQ
EGUUKPI CTG PQV VTWG /#2 FGEKUKQPU 6JKU WPFGTUVCPFKPI KU ETKVKECN HQT VJG FKUEWUUKQP
VJCV HQNNQYU
E\&5&3UHVV//&
1.3 Discriminant Function Approach to Classifier Design
&KUETKOKPCPV HWPEVKQPU QP VJG QVJGT JCPF CTG VJQUG HWPEVKQPU YJKEJ EJCTCEVGTK\G VJG
FGEKUKQP TWNG QH VJG ENCUUKſGT 6JG[ OC[ QT OC[ PQV DG RTQDCDKNKV[ QT NKMGNKJQQF
DCUGF HWPEVKQPU CPF VJG[ ECP EQOG HTQO FKHHGTGPV RCTCOGVTKE HCOKNKGU KPENWFKPI
VJQUG HCOKNKGU YJKEJ JCXG PQ TGNCVKQP VQ VJG RCTCOGVTKE HQTO QH VJG ENCUU RQUVGTKQT
FKUVTKDWVKQP
CU TGSWKTGF KP VJG ENCUUKECN $C[GU FGEKUKQP VJGQT[ 1PG YGNN
UVWFKGF HCOKN[ QH FKUETKOKPCPV HWPEVKQP KU VJG NKPGCT FKUETKOKPCPV HWPEVKQP YJKEJ JCU
EQORWVCVKQPCN CFXCPVCIGU CPF FWG VQ KVU CPCN[VKE HQTO JCU TGEGKXGF EQPUKFGTCDNG
CVVGPVKQP CPF VJGQTGVKECN FGXGNQROGPV HQT KVU FGUKIP 6Q KNNWUVTCVG VJG EQPEGRV YG
EQPUKFGT VJG ECUG QH C VYQ ENCUU
ENCUUKſECVKQP RTQDNGO 6JG ENCUUKſGT
WUGU C FKUETKOKPCPV HWPEVKQP
UWEJ VJCV
KH VJGP Z KU ENCUUKſGF VQ
KH VJGP Z KU ENCUUKſGF VQ

.KPGCT FKUETKOKPCPV HWPEVKQPU CTG VJQUG HWPEVKQPU QH VJG HQTO

YJGTG
CPF C TGCN PWODGT 1T OQTG IGPGTCNN[

YJGTG

YJGTG
KU VJG VTCPURQUKVKQP PQVCVKQP CPF VJG CTG MPQYP NKPGCTN[ KPFGRGPFGPV

HWPEVKQPU QH

(QT / ENCUU ENCUUKſECVKQP RTQDNGO WUKPI FKUETKOKPCPV HWPEVKQPU C UGV QH FKUETKOKPCPV
HWPEVKQPU
CTG WUGF CPF VJG ENCUUKſGT KU FGſPGF UWEJ
VJCV
KHH 
9JGP VJG NQUU HWPEVKQP
KU URGEKſGF VJG RTQDNGO QH QRVKOCN ENCUUKſGT
FGUKIP WUKPI FKUETKOKPCPV HWPEVKQPU DGEQOGU C OKPKOK\CVKQP RTQDNGO QH ſPFKPI C
DGUV UGV QH FKUETKOKPCPV HWPEVKQPU
HTQO C ENCUU QH FKUETKOK
PCPV HWPEVKQPU YJKEJ OKPKOK\GU VJG GZRGEVGF NQUU
CU FGſPGF KP 'S  +P QVJGT
YQTFU VJG ENCUUKſGT FGUKIP RTQDNGO KU VQ ſPF

E\&5&3UHVV//&
YJGTG KU VJG IKXGP HCOKN[ QH FKUETKOKPCPV HWPEVKQPU +H VJG NQUU HWPEVKQP
KU IKXGP CU KP 'S  CPF VJG RCTVKEWNCT UGV QH FKUETKOKPCPV HWPEVKQPU WUGF KP VJG
ENCUUKſGT CTG VJG őVTWGŒ ENCUU RQUVGTKQT RTQDCDKNKV[
 VJGP 'S  KORNG
OGPVU VJG UCOG /#2 FGEKUKQP TWNG CU FGſPGF KP 'S  *QYGXGT KV KU KORQTVCPV
VQ RQKPV QWV VJCV VJG FKUETKOKPCPV HWPEVKQP CRRTQCEJ VQ VJG QRVKOCN ENCUUKſGT FGUKIP
CU URGEKſGF KP 'S  QHVGP JCU CP KPſPKVG PWODGT QH UQNWVKQPU GXGP HQT VJG UCOG
ENCUUKſGT +V KU GCU[ VQ UGG VJCV KH
KU CP QRVKOCN UQNWVKQP VQ 'S
 VJGP HQT CP[
 KU CPQVJGT
QRVKOCN UQNWVKQP CPF FGſPGU VJG UCOG ENCUUKſGT #ICKP VJKU KU SWKVG FKHHGTGPV HTQO
VJG FKUVTKDWVKQP GUVKOCVKQP DCUGF CRRTQCEJ KP RCVVGTP ENCUUKſECVKQP +H VJG FKUETKOK

PCPV HWPEVKQPU CTG NKOKVGF VQ VJG ENCUU RQUVGTKQT RTQDCDKNKVKGU CP[ FGXKCVKQP HTQO VJG
őVTWGŒ ENCUU RQUVGTKQT RTQDCDKNKV[
YKNN TGUWNV KP C FKHHGTGPV ENCUUKſGT CPF
YKNN DG KPHGTKQT VQ VJG QRVKOCN /#2 ENCUUKſGT VJCV CEJKGXGU VJG OKPKOWO $C[GU TKUM
6JG WUG QH FKUETKOKPCPV HWPEVKQPU KP UVCVKUVKECN RCVVGTP TGEQIPKVKQP KU VQ UQNXG VJG
ENCUUKſGT FGUKIP RTQDNGO YJGP VJG GZCEV HQTO CPF XCNWG QH VJG ENCUU RQUVGTKQT RTQDC
DKNKVKGU
CTG PQV MPQYP GXGP YKVJ VJG JGNR QH VTCKPKPI FCVC QT VJG ENCUUKſGT
JCU VQ DG DCUGF QP C RCTVKEWNCT ENCUU QH FKUETKOKPCPV HWPEVKQPU 6JGUG FKUETKOKPCPV
HWPEVKQPU KP VJG ENCUUKſGT OC[ EQOG HTQO GKVJGT VJG OQFGN WUGF VQ EJCTCEVGTK\G VJG
IGPGTCVKQP RTQEGUU QH VJG TGEQIPKVKQP QDLGEVU QT VJG RTCEVKECN EQPUKFGTCVKQP QH OCVJ
GOCVKECN VTCEVCDKNKV[ CPF CNIQTKVJOKE EQORNGZKV[ %NCUUKH[KPI JWOCP URGGEJ OGGVU
DQVJ UEGPCTKQU +P RCTVKEWNCT VJG OGVJQF QH JKFFGP /CTMQX OQFGNKPI KU C RTGXC
NGPV CRRTQCEJ KP RTQXKFKPI UVCVKUVKECN EJCTCEVGTK\CVKQP QH JWOCP URGGEJ CPF VJG HWNN
EQORNGZKV[ QH ENCUUKH[KPI URQPVCPGQWU JWOCP URGGEJ KU UVKNN VQQ ITGCV VQ JCPFNG +P
VJG PGZV UGEVKQP YG IKXG C DTKGH FKUEWUUKQP QH VJG URGGEJ TGEQIPKVKQP RTQDNGO CPF
*//DCUGF CEQWUVKE OQFGNKPI DGHQTG KPVTQFWEKPI C FKUETKOKPCPV HWPEVKQP DCUGF CR
RTQCEJ VQ URGGEJ TGEQIPKVKQP
1.4 Speech Recognition and Hidden Markov Modeling
5RGGEJ TGEQIPKVKQP KU C RTQDNGO QH TGEQIPK\KPI C YQTF UGSWGPEG HTQO JWOCP URGGEJ
+V ECP DG XKGYGF CU C EQOOWPKECVKQP RTQDNGO 6JG JWOCP DTCKP UGTXGU CU VJG VGZV
IGPGTCVQT YJKEJ IGPGTCVGU VJG YQTF UVTKPI
 6JG YQTF UVTKPI IQGU VQ VJG CEQWUVKE
EJCPPGN YJKEJ EQPUKUVU QH C URGCMGTŏU CTVKEWNCVQT[ CRRCTCVWU CPF QVJGT CEQWUVKE RTQ
EGUUGU VJCV EQPXGTV VJG VGZV UVTKPI KPVQ CP CWFKDNG CEQWUVKE YCXGHQTO 6JG CEQWUVKE
EJCPPGN KP XGTDCN EQOOWPKECVKQP CEVU CU C FCVC VTCPUFWEGT CPF EQORQUGT 6JG URGGEJ
TGEQIPK\GT KU C FGEQFGT YJKEJ RGTHQTOU CP KPXGTUG QRGTCVKQP VQ FGEQFG VJG OGUUCIG
HTQO VJG URGGEJ YCXGHQTO 6JGTGHQTG C FGEQFGT RGTHQTOU C OCZKOWO C RQUVGTKQT
FGEKUKQP VJCV FGVGTOKPGU VJG YQTF UGSWGPEG
UWEJ VJCV


E\&5&3UHVV//&
YJGTG KU VJG UEQTG HTQO CEQWUVKE OQFGNKPI CPF KU VJG UEQTG HTQO
VJG NCPIWCIG OQFGN # V[RKECN URGGEJ TGEQIPKVKQP U[UVGO EQPUKUVU QH VJG HQNNQYKPI
DCUKE EQORQPGPVU
#EQWUVKE HGCVWTG GZVTCEVKQP #EQWUVKE HGCVWTG GZVTCEVKQP KU VQ GZVTCEV VJG HGC
VWTGU HQT URGGEJ TGEQIPKVKQP HTQO VJG URGGEJ YCXGHQTO +V V[RKECNN[ KPENWFGU
C UJQTVVKOG EGRUVTCN CPCN[UKU YJKEJ IGPGTCVGU C HGCVWTG XGEVQT QH NQY HTG
SWGPE[  EGRUVTCN EQGHſEKGPVU HQT GXGT[
 8CTKQWU UKIPCN RTQEGUU
KPI RTQEGFWTGU CTG RGTHQTOGF VQ UGRCTCVG VJG UCNKGPV CEQWUVKE KPHQTOCVKQP HQT
URGGEJ TGEQIPKVKQP RWTRQUGU (TQO PQY QP YG YKNN WUG VJG PQVCVKQP
VQ TGRTGUGPV VJG CEQWUVKE QDUGTXCVKQP HGCVWTG XGEVQT UGSWGPEG
#EQWUVKE OQFGNKPI #EQWUVKE OQFGNKPI RTQXKFGU UVCVKUVKECN OQFGNKPI HQT VJG
CEQWUVKE QDUGTXCVKQP
 *KFFGP /CTMQX OQFGNKPI KU VJG RTGXCNGPV EJQKEG HQT
VJKU RWTRQUG CNVJQWIJ VJG PGWTCN PGVYQTM DCUGF CRRTQCEJ KU CNUQ WUGF KP OCP[
U[UVGOU 6JG OQFGN WPKVU ECP DG DCUGF QP UGOCPVKECNN[ OGCPKPIHWN WPKVU UWEJ
CU YQTFU QT RJQPGVKECNN[ OGCPKPIHWN UWDYQTF WPKVU UWEJ CU RJQPGOGU
.CPIWCIG OQFGNKPI .CPIWCIG OQFGNKPI RTQXKFGU NKPIWKUVKE CPF ITCOOCT
EQPUVTCKPVU VQ VJG VGZV UGSWGPEG
 +V KU QHVGP DCUGF QP UVCVKUVKECN 0ITCOU
NCPIWCIG OQFGNU #P 0ITCO NCPIWCIG OQFGN KU QH VJG HQTO
 YJKEJ KU VJG RTQDCDKNKV[ QH QDUGTXKPI YQTF IKXGP VJG YQTF
JKUVQT[

&GEQFKPI GPIKPG 6JG FGEQFKPI GPIKPG UGCTEJGU HQT VJG DGUV YQTF UGSWGPEG
IKXGP VJG HGCVWTG CPF VJG OQFGN (QT URGGEJ TGEQIPKVKQP DCUGF QP *// OQF
GNKPI VJKU KU CEJKGXGF VJTQWIJ 8KVGTDK FGEQFKPI (QT C FKUETGVG QDUGTXCVKQP
RTQDCDKNKV[ DCUGF U[UVGO YQTF UVTKPI

KU IKXGP D[

YJGTG
KU VJG DGUV UVCVG UGSWGPEG IKXGP  CPF VJG OQFGN (QT
EQPVKPWQWU FGPUKV[ *//U

YJKEJ KU DCUGF QP VJG NQINKMGNKJQQF UEQTG CNQPI VJG DGUV UVCVG UGSWGPEG

1.4.1 Hidden Markov Modeling of Speech
5RGGEJ KU IGPGTCVGF HTQO JWOCP CTVKEWNCVQT CPF KV KU WPKSWG KP OCP[ YC[U 9JGP
YG URGCM QWT CTVKEWNCVQT[ CRRCTCVWU VJG NKRU LCY VQPIWG CPF XGNWO OQFWNCVGU VJG
CKT RTGUUWTG CPF ƀQY VQ RTQFWEG CP CWFKDNG UGSWGPEG QH UQWPFU &WG VQ VJG RJ[UKECN
EQPUVTCKPVU VJG CTVKEWNCVQT EQPſIWTCVKQP ECPPQV WPFGTIQ XGT[ FTCUVKE EJCPIGU CPF
FWTKPI VJG UJQTV KPVGTXCN YJGTG VJG CTVKEWNCVQT[ EQPſIWTCVKQP UVC[U TGNCVKXGN[ EQP
UVCPV C TGIKQP QH őSWCUKUVCVKQPCTKV[Œ KP VJG RTQFWEGF URGGEJ ECP QHVGP DG QDUGTXGF
E\&5&3UHVV//&
*KFFGP /CTMQX OQFGNKPI KU C RQYGTHWN UVCVKUVKECN HTCOGYQTM HQT VKOG XCT[KPI SWCUK
UVCVKQPCT[ RTQEGUU CPF C RQRWNCT EJQKEG HQT UVCVKUVKECN OQFGNKPI QH URGGEJ UKIPCN
)KXGP C URGGEJ WVVGTCPEG NGV
DG C HGCVWTG XGEVQT UGSWGPEG
GZVTCEVGF HTQO VJG URGGEJ YCXGHQTO YJGTG
FGPQVGU C UJQTVVKOG XGEVQT OGCUWTG
OGPV CPF KV KU EQPXGPVKQPCNN[ C EGRUVTCN XGEVQT
(WTVJGT EQPUKFGT C ſTUVQTFGT
UVCVG /CTMQX EJCKP IQXGTPGF D[ C UVCVG VTCPUKVKQP
RTQDCDKNKV[ OCVTKZ
 YJGTG KU VJG RTQDCDKNKV[ QH OCMKPI C VTCPUKVKQP
HTQO UVCVG
VQ UVCVG  #UUWOG VJCV CV VJG UVCVG QH VJG U[UVGO KU URGEKſGF
D[ CP KPKVKCN UVCVG RTQDCDKNKV[

 6JGP HQT CP[ UVCVG UGSWGPEG
 VJG RTQDCDKNKV[ QH DGKPI IGPGTCVGF D[ VJG /CTMQX EJCKP KU

5WRRQUG VJG U[UVGO YJGP CV UVCVG
 RWVU QWV CP QDUGTXCVKQP CEEQTFKPI VQ C
FKUVTKDWVKQP
  6JG JKFFGP /CTMQX OQFGN WUGF
CU C FKUVTKDWVKQP HQT VJG URGGEJ WVVGTCPEG
KU VJGP FGſPGF CU

YJGTG
KU VJG RCTCOGVGT UGV HQT VJG OQFGN
#U ECP DG UGGP KP 
FGſPGU VJG FKUVTKDWVKQP HQT UJQTVVKOG QDUGTXCVKQPU
CPF
EJCTCEVGTK\GU VJG DGJCXKQT CPF KPVGTTGNCVKQPUJKR DGVYGGP FKHHGTGPV UVCVGU QH VJG
URGGEJ IGPGTCVKQP RTQEGUU +P QVJGT YQTFU VJG UVTWEVWTG QH C JKFFGP /CTMQX OQFGN
RTQXKFGU C TGCUQPCDNG OGCPU HQT EJCTCEVGTK\KPI VJG FKUVTKDWVKQP QH C URGGEJ UKIPCN
0QTOCNN[
 VJG VQVCN PWODGT QH UVCVGU KU OWEJ UOCNNGT VJCP  VJG VKOG FWTCVKQP
QH VJG URGGEJ WVVGTCPEG 6JG UVCVG UGSWGPEG
FKURNC[U C EGTVCKP FGITGG QH UVCDKNKV[
COQPI CFLCEGPV
U FWG VQ VJG CDQXG OGPVKQPGF őSWCUKUVCVKQPCTKV[Œ 6JG WUG QH
*//U CU URGGEJ FKUVTKDWVKQPU KU UJQYP VQ DG RTCEVKECNN[ GHHGEVKXG
+V UJQWNF DG PQVGF VJCV VJG EJQKEG QH UVCVG QDUGTXCVKQP FKUVTKDWVKQPU
KU PQV
URGEKſGF &KHHGTGPV EJQKEGU QH URGGEJ FKOGPUKQPU HQT VJG QDUGTXCVKQP URCEG OC[
TGSWKTG FKHHGTGPV HQTOU QH VJG UVCVG QDUGTXCVKQP FKUVTKDWVKQP (QT EGRUVTCN XGEVQTU
C OKZVWTG )CWUUKCP FGPUKV[ KU EQOOQPN[ GORNQ[GF /QTGQXGT TGICTFNGUU QH VJG

RTCEVKECN GHHGEVKXGPGUU QH *// KP URGGEJ TGEQIPKVKQP KV UJQWNF PQV DG VCMGP CU VJG
VTWG FKUVTKDWVKQP HQTO QH URGGEJ CPF VJGTGHQTG CP[ TGEQIPKVKQP U[UVGO QT FGEKUKQP
TWNG VJCV QRGTCVGU DCUGF QP *// KU PQV IQKPI VQ CEJKGXG VJG OKPKOWO GTTQT TCVGCU
KORNKGF KP VJG VTWG $C[GU /#2 FGEKUKQP
+P QTFGT VQ CRRN[ *//U VQ URGGEJ TGEQIPKVKQP VJTGG DCUKE RTQDNGOU JCXG VQ DG TG
UQNXGF PCOGN[ VJG GXCNWCVKQP RTQDNGO VJG FGEQFKPI RTQDNGO CPF VJG GUVKOCVKQP
RTQDNGO = ? 6JG GXCNWCVKQP RTQDNGO KU VQ GUVKOCVG VJG RTQDCDKNKV[
QH QDUGTXKPI VJG URGGEJ HGCVWTG XGEVQT UGSWGPEG IKXGP VJG JKFFGP /CTMQX OQFGN
6JG FGEQFKPI RTQDNGO KU VQ ſPF C DGUV UVCVG UGSWGPEG
YJKEJ KU QRVKOCN KP C EGTVCKP
UGPUG IKXGP VJG URGGEJ HGCVWTG UGSWGPEG
 5KPEG UVCVGU KP *// CTG TGNCVGF VQ YQTFU
CPF YQTF ENCUUGU VJG YQTF UGSWGPEG KP URGGEJ WVVGTCPEG ECP DG KFGPVKſGF D[ VTCEKPI
VJTQWIJ VJG YQTF NCDGNU KP UVCVG UGSWGPEG
 6JG GUVKOCVKQP RTQDNGO KU VQ GUVKOCVG
E\&5&3UHVV//&
*// RCTCOGVGTU HTQO C IKXGP UGV QH VTCKPKPI UCORNGU CEEQTFKPI VQ UQOG OGCP
KPIHWN ETKVGTKQP 6JG EQPXGPVKQPCN CRRTQCEJ KU DCUGF QP VJG OCZKOWO NKMGNKJQQF
/. RTKPEKRNG CPF VJG OQFGN RCTCOGVGT UGV
KU GUVKOCVGF UQ VJCV VJG NKMGNKJQQF
QP VJG VTCKPKPI FCVC KU OCZKOK\GF 8CTKQWU JKIJN[ GHſEKGPV /. DCUGF CNIQTKVJOU CTG
FGXGNQRGF KP URGGEJ TGEQIPKVKQP HQT *//U UWEJ CU $CWO9GNEJ CNIQTKVJO =? CPF
UGIOGPVCN MOGCPU CNIQTKVJO =? /QTG FKUEWUUKQPU QH RCTCOGVGT GUVKOCVKQP RTQD
NGO HQT *//U ECP DG HQWPF KP =? +V UJQWNF DG PQVGF VJCV VJG EQPXGPVKQPCN /.
OGVJQF KP URGGEJ TGEQIPKVKQP FQGU PQV PGEGUUCTKN[ NGCF VQ C OKPKOWO GTTQT TCVG RGT
HQTOCPEG HQT VJG TGEQIPK\GT 6JKU KU FWG OCKPN[ VQ  VJG NKMGN[ OKUOCVEJ DGVYGGP
VJG EJQUGP FKUVTKDWVKQP HQTO CPF VJG CEVWCN URGGEJ FCVC FKUVTKDWVKQP CPF  VJG ſPKVG
VTCKPKPI MPQYP FCVC UGV YJKEJ KU QHVGP KPCFGSWCVG
1.5 MCE Classifier Design Using Discriminant Functions
#U KV KU PQVGF YKVJQWV VJG MPQYNGFIG QH VJG HQTO QH VJG ENCUU RQUVGTKQT RTQDCDKNK

VKGU TGSWKTGF KP VJG ENCUUKECN $C[GU FGEKUKQP VJGQT[ ENCUUKſGT FGUKIP D[ FKUVTKDWVKQP
GUVKOCVKQP QHVGP FQGU PQV NGCF VQ CP QRVKOCN RGTHQTOCPEG 6JKU OQVKXCVGU VJG GHHQTV
QH UGCTEJKPI HQT QVJGT CNVGTPCVKXG ETKVGTKC KP ENCUUKſGT FGUKIP +P RCTVKEWNCT ETKVGTKC
QH //+ OCZKOWO OWVWCN KPHQTOCVKQP CPF /&+ OKPKOWO FKUETKOKPCVKXG KPHQT
OCVKQP CTG WUGF KP OCP[ CRRNKECVKQPU = ? #NVJQWIJ VJGUG OGVJQFU FGOQPUVTCVG
UKIPKſECPV RGTHQTOCPEG CFXCPVCIGU QXGT VJG VTCFKVKQPCN /. CRRTQCEJ VJG[ CTG PQV
DCUGF QP C FKTGEV OKPKOK\CVKQP QH C NQUU HWPEVKQP YJKEJ NKPMU VQ VJG ENCUUKſECVKQP
GTTQT TCVG
&Q6W GV CN =? UVWFKGF /%' UQNWVKQP HQT VJG VYQ ENCUU PQPRCTCOGVTKE ENCUUK
ſECVKQP RTQDNGO WUKPI NKPGCT FKUETKOKPCPV HWPEVKQPU 6JG[ GORNQ[GF C YKPFQYGF
UEJGOG VQ QXGTEQOG VJG RTQDNGO QH UKPIWNCT ITCFKGPV HWPEVKQPU CUUQEKCVGF YKVJ VJG
GTTQT EQWPV KPFKECVQT HWPEVKQP QH VJG ENCUUKſGT # IGPGTCN CRRTQCEJ HQT OWNVKENCUU
CPF PQPNKPGCT FKUETKOKPCPV HWPEVKQPU CTG RTQRQUGF D[ ,WCPI GV CN =? 6JKU IGP
GTCN CRRTQCEJ KU ECNNGF őOKPKOWO ENCUUKſECVKQP GTTQT /%' OGVJQFŒ KP YJKEJ VJG
ENCUUKſGT FGUKIP CPF RCTCOGVGT GUVKOCVKQP CTG VQ EQTTGEVN[ FKUETKOKPCVG VJG QDUGTXC
VKQPU HQT DGUV TGEQIPKVKQPENCUUKſECVKQP TGUWNVU TCVJGT VJCP VQ ſV VJG FKUVTKDWVKQPU VQ
VJG FCVC
1.5.1 MCE Classifier Design Strategy
.GV WU EQPUKFGT C UGV QH ENCUU FKUETKOKPCPV HWPEVKQPU  FG
ſPGF D[ VJG RCTCOGVGT UGV
 6JG ENCUUKſGT KU VJG QPG VJCV HQT CP QDLGEV 
KHH 
6JG IGPGTCN /%' ENCUUKſGT FGUKIP UVTCVGI[ KU DCUGF QP C URGEKCN V[RG QH NQUU HWPEVKQP
2CTCOGVGTU QH VJG ENCUUKſGT CTG GUVKOCVGF KP UWEJ C YC[ VJCV OKPKOK\KPI VJG GZRGEVGF
E\&5&3UHVV//&
NQUU TGNCVGU VQ C OKPKOK\CVKQP QH VJG TGEQIPKVKQP GTTQT TCVG QH VJG ENCUUKſGT 6JKU KU
CEJKGXGF VJTQWIJ C VJTGG UVGR RTQEGUU
 6JG OKUENCUUKſECVKQP OGCUWTG KP VJG /%' DCUGF CRRTQCEJ KU FGſPGF CU

YJGTG

KU C RQUKVKXG PWODGT =? 6JKU OKUENCUUKſECVKQP OGCUWTG KU C EQPVKPWQWU
HWPEVKQP QH VJG ENCUUKſGT RCTCOGVGTU
CPF CVVGORVU VQ GOWNCVG VJG FGEKUKQP TWNG
IKXGP VJG FKUETKOKPCPV HWPEVKQP
 (QT CP ENCUU WVVGTCPEG 
KORNKGU OKUENCUUKſECVKQP CPF OGCPU C EQTTGEV FGEKUKQP 9JGP CR
RTQCEJGU
 VJG VGTO KP VJG DTCEMGV KU VJG PQTO QP VJG FKUETGVG KPVGIGT UGV
YJKEJ EQPXGTIGU VQ VJG PQTO CPF DGEQOGU
 $[ XCT[KPI VJG XCNWG QH CPF  QPG ECP VCMG CNN VJG EQORGVKPI
ENCUUGU KPVQ EQPUKFGTCVKQP CEEQTFKPI VQ VJG KPFKXKFWCN UKIPKſECPEG YJGP UGCTEJKPI
HQT VJG ENCUUKſGT RCTCOGVGT

 6JG NQUU HWPEVKQP KU WUGF HQT TGEQIPKVKQP GTTQT TCVG OKPKOK\CVKQP 6JG OKUENCUUKſ
ECVKQP OGCUWTG QH  KU GODGFFGF KP C UOQQVJ \GTQQPG HWPEVKQP HQT YJKEJ CP[
OGODGT QH VJG UKIOQKF HWPEVKQP HCOKN[ KU CP QDXKQWU ECPFKFCVG # IGPGTCN HQTO QH
VJG loss function ECP VJGP DG FGſPGF CU

YJGTG
KU C UKIOQKF HWPEVKQP QPG GZCORNG QH YJKEJ KU

YKVJ
PQTOCNN[ UGV VQ CPF UGV VQ ITGCVGT QT GSWCN VQ QPG %NGCTN[ YJGP
KU OWEJ UOCNNGT VJCP \GTQ YJKEJ KORNKGU EQTTGEV ENCUUKſECVKQP XKTVWCNN[ PQ NQUU KU
KPEWTTGF 9JGP
KU RQUKVKXG KV NGCFU VQ C RGPCNV[ YJKEJ DGEQOGU GUUGPVKCNN[ C
ENCUUKſECVKQPTGEQIPKVKQP GTTQT EQWPV
 6JG ENCUUKſGT RCTCOGVGT GUVKOCVKQP KU DCUGF QP VJG OKPKOK\CVKQP QH VJG GZRGEVGF
NQUU (QT CP[ WPMPQYP QDLGEV
 VJG ENCUUKſGT RGTHQTOCPEG KU OGCUWTGF D[


YJGTG
KU VJG KPFKECVQT HWPEVKQP 6JG GZRGEVGF NQUU YJKEJ KU TGNCVGF VQ TGEQIPK
VKQP GTTQT TCVG KU IKXGP D[

6JKU VJTGGUVGR FGſPKVKQP GOWNCVGU VJG ENCUUKſECVKQP QRGTCVKQP CU YGNN CU VJG TGEQI
PKVKQP GTTQT TCVG DCUGF RGTHQTOCPEG GXCNWCVKQP KP C UOQQVJ HWPEVKQPCN HQTO UWKVCDNG
E\&5&3UHVV//&