KỸ THUẬT GOM NHÓM VĂN BẢN BẰNG THUẬT TOÁN K-MEAN

Đại Học Quốc Gia Tp. Hồ Chí Minh
Trường Đại Học Công Nghệ Thông Tin

Báo cáo cuối kỳ môn Khai Phá Dữ Liệu
KỸ THUẬT GOM NHÓM VĂN BẢN
BẰNG THUẬT TOÁN K-MEAN
GVHD : PGS.TS. Đỗ Phúc
Học Viên : Đỗ Duy Phúc
MSHV : CH1101122
Lớp : Cao Học K6
Tp. HCM, Tháng 11 Năm 2012
Mục Lục

Lời Nói Đầu
 !"#$
%&'$()*+,!$&-&.&'
,/*&.!01234)245,
&'$)67-*$'"#$8
$0*9%,01*(*4:$67
*$67-";$)*(-&,$&<
=$,>0?(@A'$0+&&'"""
,0%&*BC%,$$'@"
D%CE)4F$'),!G&'H#,IJ&J
KG,JK=GJL*0-=,$#$&'HKML
6E)"-/,*9
N.*B&-$&.&'=$3/HGJOGJ&
&L,3/H,OGJ&&L"D7&P,$3=8'
@$IJ=E+IJ=GJ1$Q30=$,$,7
*',+"R17(13N,+'A8>
!G&'H!&'*B3/L%&'(JS
C+T',"K,*+3*4*C,*((%C

$)UV&'1"W,$#$&'Q=8
:1,++$*B&-&.
Q=83*4*C,($*B$J,>
GUA$)-&<*,$!")U7
,A$)(*B$'*(G&P,$P*7G"
M+G@=,$A ,#F$KX'DQW8*BY*C
$=,$ZRJ;GJ[ZRJ;GG\,[ZRJ]^,[_
R,*V$`J-'+G@a1,Q=8
H&,JGJL-'$&P1,$#OJG,,
Q=8"
b
Phần 1: Cơ Sở Lý Thuyết
1. Khai Phá Dữ Liệu
Khai phá dữ liệu (data mining) =-67,A$%khám phá tri
thức từ cơ sở dữ liệu( Knowledge discovery in database, KDD). c6+
d<2,e*BCBU4d<$f, $
7+A$%%C$$$(S)U+1-&
'"1&P$!$E)U$d<$7'
6.,$ @E'@!G&'"R,*G&P$
!$U$ @E43"
MPEfA$%$)*(/7*B)U+
1&'Gg(*9/3/,0((
*B,PEG&P!"WE.$=-672=,
V$7.fA8&'!G&'4G&'%
G&h*$-*,*$$JY*+).1$
3/%*B=(&h<A"_
2. Gom Nhóm Văn Bản
Gom nhóm văn bản(document/ text clustering)+d<,fgom
nhóm dữ liệu(data clustering). B$$'U$d<
3H\,,JJijkL<EH

J,JGGiXFL$ HJJOMXL"RU*6,
=C,Q=8Gl*B*41 ,"
m$%,--$'$<EU*-'
+$%9Af$6-Hf*4L,+1B'Q
=8"R,d<7'6.,*6*BJ+!$$ 
$G$"n,E=>S-A$%6-"R,
+=,$6-Q=8G@B6-H$73fL*B
=C-$'Q=8*B6@,$-"B.
,+3*4,=3o8G@B+7,73
Ef$6-*C*B=C-"]<6='*B 
%p"F6-E*BJ+7&Pf!$$ $
G$"
q
r%p"
M+!$,@(*BJ+$9)+1
',$,*$'Q=8,s+*4
G<!<+G<$='G,-'$"m$
%,8QGlG8$6$=',$6
,HVLE"R,$6,8Q+
'Gl3',4";6`**4%3E+
!$,'A8tD-1fCf$
E)%680,6`)f$1,$$
*$$/"
3. Phân Loại Thuật Toán Gom Nhóm Văn Bản
a) Phân loại theo đặc tính của thuật toán
;$1,$,(6,.u$1,$$='
H&GJL$1,$G8GHJJJL"
M+$($1,$$=',$G,G$*+!*VJ,
U,v'Q=8&<EG<!*VGl@
+E5*(.,+$,@"cf$1

,$0$6&'$='"
w+-$$1,$G8G8*>N+G<6@$
G3,&'"D'f/@% 6f$6
x
G,,$ 6sB-G<6@&'+$@*"
]85f$1,$,.$6,V"
b) Phân loại theo mô hình biểu diễn dữ liệu
R$)*3E,+=,$,$*>*7,f
+'*B&s*(.,76,."+$$/%
+%=(&h&',Q=8"K*=E.$6,.E
(6,.1,$&<J,$/< %=(&h&
',A$%"
F-$$C1,'. =(&hv'
+J!*.&',+*(4"RU*( +
'Q=8+&.=(&hsB,$C1,&
'4@"
M+$%C,$=(&hJ!*B.,=N$
UyPU'Q=8+14H&zJL"M+
14,C*V>*>dG<C@f$*:,+*V>"
K-%J!$CA8+&.*V> G@"D',
E*V>C*3*4(=,$**0
HI,=JL!$*0%$6*:*V>-
G<*){$.@$*:7@"
M+CA8$U'< =(&h=NJ!1UO'
*B.,(3-"D'+-.,+E=8@
*/C.3H,IOLf+1U*.,+
!$6,.@,s1&'"
M+*(C4f%J,{4
)<fU"K,1*6+G@-C$=(&h
$,4*>dG<!*V$=(&h-2=,

VE3'f$UyPUG33's-
,J,+v,*"
4. Các Phương Pháp Đánh Giá
;$1,$,(*B*$$J,4$$"#
37G<*V14$,$@3"]<< !
$,0&<Ed<E)*B<'"D7&P+
E  )  |j   l  E  7  !  $     s  H
}
\,,L,+G@E)Ud<jkGl !$
~OJGJ";$!$*$$Gl*B=1.*6"
;'<$4'Qf$1,$,độ chính
xácHprecisionL độ triệu tậpHrecallL"R,d<jkđộ triệu tập *B
*>d:'$'Q=8EAH*C6*B
=9'Lđã được truy xuất raG,-tất cả các tài liệu có liên quan'
V.,,=+1'*BY"R,*độ chính xác:'
'đã được truy xuất và có liên quanG,-tất cả tài liệu đã được
truy xuất. W%3&h*(*B*+'1+$,8,d
=N$3,=+'*,=3o3,,E~O
JGJ+-*,BCB8*+'1*+7$*B-
'*(,.=`.C"RA-k*+'1F*+7$%
)~OJGJ9A$*B*>dGu
c+7$*+'10*B,s G@HF
p
L
=C(V.*$ G@/$7&P*+7$,
!G,-*+'1,B.H/*B${B
F
•"x
F


L
c(+),'*$$+1,$,/
8*>NG<V.f+1$6-8,H*B&<*,$
-Q)*$$L$6*B%3H*f$
1,$,L*B*@$'*B3U$6-
8,"R%)*4'Gu
R,*u
nu9G@'
argmaxu%G@jG,,F*.*B$>@*
iu)<f6-8,
ju)<f6*f1,$
€
R,~OJGJ:*B3B9A$fA$%,
,9'G33:*BG<3f=8
6$"r-*,=9Gđộ thuần khiếtHpurityLf
entropy"F*,3Qf$E+6-9+!
,,+H-!%@!L,J,%CG<6
@$3f$'Uv6-EA,vH`!
@!L"D/)Gu
CG@B6=N-G@Bf$(,.'+G<
!)(C1%$-*,E(*B$&P-+
"REC+G<3A$G@B6-
8,$6*B%=1,$%$7$*
1G<:*B3Bf$6"
c{=N):N,0Bs
H\,,OMjL+-*,C!G,-*+C
,J,"R,<CMj*B@84$*+&*!>H,^J&
MjiMjL"RJ,*>dA-n
h
G@',6-hn

l
G@
S,6ln
h,l
G@S,6-h 6l,%
)Gu
Mj-.,,8••p‚,*$>fG@p*.&'G<G,-
'*@$6$6-".*+<fMj
+-*,*+7$,A$%,(>*BG<=3
@3G@B$6%3G@B$6-
8,";+G@B*$($=C4,Q=8*48*>
+G@6,-k, %C,0B$*,
4@(*BG&P"
ƒ
M+$%$43Bf+1,$,G<9*>f
$6.,A4."M+-C19=1*(*,
*+9*>%'Q=%"R,1sHMjL
CA8.,*B |MjHJJ,^J&\,,Lu,
+1frG@,+.,P(%|Mj
*B*>du
R,$1SH,\G,L3'0
E,$=C4,$!$$E8E
/3'*f*(+/7*6"M+1S
+P=(&h<A3+G<9B4v6.,
='@"
W


•
Bảng 1: Một ma trận nhầm lẫn cho các phân lớp:

A (4 phần tử), B (4 phần tử), C (8 phần tử)
W8p(&h8Gu1,$,(6='
6|6W6='$*@B,6-;
+$,8,"
R38!$E8E*4*B&s,,.6$='"
kE*@-$1,$,G&P,0+!$
&P*(*$$*f+1,$,0G$6
$='„U+CA8*0HA$%*B A$
%cứng hóa - hardening$6L=N$*+$>…
,<*+Ef$'";d$',)*+
EB+…$>f+,*%*B,+6
*C+'*4B…f6%Gl+
8"
†
5. Một Số Thuật Toán Tiêu Biểu
a) K- means và các mở rộng
* Thuật toán k-means
R,d<$&'1,$,OJG+!
$ f 6 7 "MP E f  1 ,$   n *@ B P
EH*B=(&h$*(,*4Lk,
*v*@BPE+4*(3"#OJG
1  ,$  E =(  , ,. 1  ,$ ,   $  =' H&GJ
,GL"
FE=8&P3fOJGG&P+a1:
,"D%G<79=C+f1OJG0*BE
*CE=87$!%*61,$X,&H
Ef0$L$E*B-, $7*C0
E!";C1,$Gu
;,+1k *(m
1

(1)
,…,m
k
(1)
1,$<=N$,
=-Gu
OW-$u$v*@BPE,*(
3"
R,*vx
p
*B67$,+S
i
(t)
8(
6,*B$"
OW-:uv7,$$*(-=N 6*B
7U$*@BPE*B6,,*"
R1,$*B,N*{+P&U.+$G<6=92
*9"
* Các dạng mở rộng của k-means
R,d<,Q=8*(sB-(&'*s1,$
OJG4@*B=C*9$&.sB!u
p•
OR1,$ k-means cầuHspherical k-meansLu*61,$!<-
1,$OJG4@"#G&P1,$&'Q=8
*B($J!*4*B@8"REG&P
,G%,8$‡&*(%7!*V,$*9B
6"
M&s9=C+&,7&h*OJG4
*(.CuP+-,G<.,SE$

+$ˆ*4*(&S*C1,$+P4$*(@
*>!ˆ$6Gl()4,..hˆ
*+).fO(nkl),,*nG@B'Q=8,1&
'k G@,@lG@2"R1,$0
sB$&P,$1&'Q=8-7*9"
OR1,$k-means cầu trực tuyếnHonline spherical k-meansLu*61
,$+fOJG"G&Pa1  .    *(  
Q@A$%,,S*.*B*+7$!<,
@!"
D-S .<C$Q=8*B4*CE
P"r,,$B-+1,$.,,E+1&'
+1,$OJG"#$Q=8*B*,A$%6
$Gl.*(3&'*,*(*B6=9,
f%"DCGl<*4:%&<,+:' 
3*>*((8).!,$&'*,CJ,"R1
,$0*B$&P*(6=8G*B84U+%
%C,3H7&Pn,,JL"
OR1,$k-means nhân kernel kmeansur% +1&'
OJG(,7$=%$*($='+
$C7"‰*NGJJJG%+$.,$*(
*C+44!!$&'Q=8G<$
='+$C7"
pp
Hình 2: (a) cho thấy một hình dạng phân bố dữ liệu không lồi mà thuật toán k-means gặp
vấn đề khi gom nhóm. Trong (b) là thuật toán kernel kmeans đã thành công phân biệt rõ
hai nhóm như là kết quả của việc chiếu các vectơ văn bản lên một không gian có nhiều
chiều hơn.
b) Các thuật toán sản sinh (generative algorithm)
R,E=(1,$c-means mờ (fuzzy c-means), +1
,$3.8-$,."R,$1&'Q=8.

$,.3'30E"K,1=N$.,$
8*>4G<6@$3f&'$!$@E'A8
,(*B$&P*($$$sG<'&'f&'
h*V03173Ef+'Q=8,
46$"
;$!$$='&<E*+!*V'J,UJ,*>
d-*+).O(n
2
)"R0$*+!*V(*B
7-,+1"RJ,+$$$%G8G
*611G&P+fP,B
$=--B%$G3$Q=8"
M+1,$%$3*BG&P,1,$G8
Gu
OM%nGG"
p
OR1,$,$,*BHJJ,^,L"
OM%,MGJGO~GJ"
OR1,$OJG&<E%$G3H,&JO=GJ&JGL"
c) Các thuật toán gom nhóm phổ (spectral clustering algorithms)
M+1+&.=(&h<E,44$*:
&,*%J!(*B&h&>+*V>"n,9=,
%$*0,*V>*(G$63B
@"
W,$=60*%$*0@E*V>CA8.,
+G@$E5HJ,\,L$1,$,9
--@";/=,V:'**0HI,L@
8H,^J&LO"RE$*V>!$*B
6='+$@*Š%38a1E*4-'G3!
<"RE+G<s*$($*4*(%

8-&'Q=8%$*0OGl<'@!"
M+G@1,$E=(+u
OR1,$` .H&&J&JJ,L"
OcV,0H\^^GJL"
pb
Phần 2: Demo Thực Hiện Gom Nhóm Văn Bản
Bằng Thuật Toán K-Means
1. Cụ Thể Thuật Toán K-means
c,u
O R1$*@B‹Œ•

ŽŒp_•
O ]@Pu#
cu
O ;$P;

HŒp""#L$0
R1,$OJGV=@=-u
p" ; SE*( 6=*fP
" n$H,$.LU*(,P 6*(*Y
3"CY$.,%&U"D%Y$.
,d$P*{9*>1,$(8'
8*+6='!*B
b" R7. 6,UP
q" m.=-
Hình 3. Sơ đồ khối chương trình
pq
Hình 4: Các vector trọng tâm ban đầu của từ cụm được biểu diễn bằng chấm đỏ.
Trọng tâm sau khi cập nhật được biểu diễn bằng dấu x đỏ
px

Hình 5: Cập nhật trọng tâm sau lần lặp thứ 2 và thứ 3
2. Áp Dụng Thuật Toán K-mean Vào Phân Lớp Văn Bản
a) Giới thiệu
c($&P1,$#OJ,6-Q=88<'J,
  Q  =8  v  Q  =8  *B  =(  &h  &-  &.  J,
'J,Q=8G&P%J,
"]1$J,($&P1,$#OJ,
,PQ=8"
b) Mô hình vector không gian
M%J,%J,.+%*.G@,*.
&'$'Q=8H=3o*@B,LJ!*>&
•.7&P4:G@"*BG&P, V
1:P=8C.EA"]&P*Ef'@
V]M|kR"
R'$3*B=(&hJ!"
Mv7-!)-+1E='"C.8,
'$>f,J,"M+G@$$7,$$$
>2*B  BH.L*{*B$("M+,
!$@3*B=C*C'G@ tf-idf (term frequency – inverse
document frequency)"c>df1P+,)&P"R
0*4'U*!U,$,PU&"C*B< 
$*4,8G@4fJ,G@U,U<HG@B$UE
='8,'L"DJ,,.*+(*BG&P*(G,G$$
'-$3"
R)A$% J, 6=*3A C%*B
$ 6,U6='*E%3@E*4
<'*B"R,!%J:<'k&, 6=
*"]@=*?+3*4/8$*>*BU*
p}
G@,$)&P<C%G@/*BU1&

'"
c) Xác định trọng số (tọa độ) của hạn (term) trong văn bản
R,%J,9*( BP(.,$J!
'Q=87f$=CG@*>!,P"DJ, G@,
'&J,
R,* *+I
&
*B7=)tf-idf,+.,Q=8&
Gu
\O&\HLŒ\H&L"&\HL
D-u
O \iJ\JAJuG33'f+.,+Q=8
P(&"
O &\OJGJ&,J\JAJu>*8,G33'f+
.,,=+1Q=8*,K*B7=N)u
D- G@Q=8.3')*B+E
p*($0B,•"
d) Xác định các hạn(term) đặc trưng cho văn bản
‹$*>$.*,Q=87'$$UPU*
,Q=8"R$U,Q=8+=8,$*=',
CD'"
c@-'$*>$.,Q=8%$U@d$&36Gl
*B,.=`*E$U@d0$-U.,UuZ%[
Z[Z[ZGl[_;$U*B UG,OI,&,C|"
c@-Q=8C|'$U!*@&h%C$U,C
|U*!/($&P'$$,8<*
p€
='"REC$U,CD'$UY'$$U*!
1Gl,',7$"
Ví dụ:ZW$Gd[ZGd[Z.Gd[ZGd[_%'$U*!(

,',7$"
R,)&P`JG&P'$ UJ,=-u
O W-pu U=NU*("M+U*(V$UY9=C
,CD'*B  V$U0&sC$
&U"RU*(*BGCJ,)<8&fG@6C"D'
GC8&G@6CGl/'E *B$PU
46C-";$PU&*BJ7*
,Q=8"
D7&Pu
Z'$([
MSU*(u Z'[Z'[Z[
O W-u $PU9=C3'4,Q=8"
]*{,.=`*B$U $&U2.f
Q=8Gl*B$$U*!";$U*!Gl*B
A1,$%19=C*(%$PUG@
3',,Q=8"
R1,$%PU9=C=-Gu
c,u
O R1$U*!H*{,.=`U&36LUQ=8*Y
O ‘Gu33'@(f+UyPU,Q=8
cu
O R1$UyPU9=C,Q=8
R1,$u
p" R%19=C~
p
)$U*!9=CHG33'-!
‘GLU1U*!*,"
" n$~
Op
=N~

p
~

~1v
b" D-vUyPU‹+~
Op
<'
" D-vU*!’+~
p
<'
" ]Œ‹@v-’
pƒ
" C]FH]L“Œ‘G%u
p" ~

Œ~

•]•
" ~Œ~•]•
" ]Œ’@v-‹
" C]FH]L“Œ‘G%
p" ~

Œ~

•]•
" ~Œ~•]•
q" C~

$v

" R%1.=-b-~
Op
*B$=N~
 
1~
 
$=N1
v
=" B.%&U1,$841~
…‘G*BG&P,!%Hd3'
E,+Q=8L"r]FH]L*C3P],Q=8
*Y"
3. Chương Trình Demo
a) Giới thiệu
;!%&J,*BC-,&',G,J=N1%k="
R,!%*B*$=-,Q=8*{*41
$-"
b) Hướng dẫn sử dụng chương trình demo
;!%=,Vu
O ~Jmain.exe!%7
O RPdata)$1'&UC'$U
vB,!%
O RPinput)$1Q=8*,,P";$\J
*B-*>&.JH*"L";!%7
.GlAY38$1,Ps$P,
&'&,"
O RPoutput )$CA8=,V$P,"Mv
P,*.&',+PQ=8*G"R,
vP,)$1!)-+1*,
)$UyPU*s-G@3G3'fQ

=8!% *B"
p†
]&P!%u
O ;,1$Q=8,P,P";$Q=8
*B-*>&.J"
O ;<1"JJ
O #CA8Gl*B,P,
•
Phần 4: Kết Luận
m$%=E/*{%(*B$$'
4d<')$ 3*{*G6
,d<$&'P(d<,&'Q=8"
;/*{=C*B4'.$-C1',3*4
$A841,$%=(&h$a1@E$3
*BG&P"
R*{34CA8E)'+8$,
<G<<'*BA$%,Q=8+$,8,"Mv
-C18$*4C.C,0B
$"D,,+0B,(G$
7$<E,:"
R,=C!%&J,*{'< *B**f$
=-!=8,A$%,Q=8"RE%0E)
.,E!%:&U)'"#CA8*f
!%*+7$S2!*@3C9*>"*6
4*4A *((E)G6!,d<"
cC*6JC/=C";$!R$>$=.*{=`0
* =C"
p
Tham Khảo
•p‚cvF/Giáo Trình Khai Thác Dữ Liệu, ‹WcrmnR"r;M••}

•‚,G”"|&JIG&‡&I&|"~,Recent Developments in Document
Clustering”,=Jp}••€
•b‚  ;G  ]" FJ,J Machine Learning :: Text feature extraction (tf-idf)
]JJ=Jpƒ•pp
uyyJ,J"G,J\,J"JyI,&JGGytŒpxƒ†
•q‚RJJJFVector space document model
uyyG}ƒ•"G"G\"J&yJJGyJ,OGJO&,JO,&J
•x‚uyyJ"IJ&",yIyK‘
