LUẬN văn TOÁN ỨNG DỤNG một số TÍNH CHẤT cơ bản QUÁ TRÌNH MARKOV và ỨNG DỤNG

▲✉➟♥ ✈➠♥ tèt ♥❣❤✐➺♣
❈❤✉②➯♥ ♥❣➔♥❤✿ ❚♦→♥ Ù♥❣ ❉ö♥❣

▼❐❚ ❙➮ ❚➑◆❍ ❈❍❻❚ ❈❒ ❇❷◆ ❈Õ❆
◗❯⑩ ❚❘➐◆❍ ▼❆❘❑❖❱ ❱⑨ Ù◆● ❉Ö◆●
●✐→♦ ✈✐➯♥ ❤÷î♥❣ ❞➝♥✿ ❚❤➛② ▲➙♠ ❍♦➔♥❣ ❈❤÷ì♥❣
❙✐♥❤ ✈✐➯♥ t❤ü❝ ❤✐➺♥✿ ✣é ❚❤➔♥❤ ❚➔✐
▲î♣✿ ❚♦→♥ Ù♥❣ ❉ö♥❣ ❑✸✷
◆❣➔② ✷✹ t❤→♥❣ ✼ ♥➠♠ ✷✵✶✵


✷


▲❮■ ❈❷▼ ❒◆
❊♠ ①✐♥ ❝❤➙♥ t❤➔♥❤ ❝→♠ ì♥ ❇❛♥ ❣✐→♠ ❤✐➺✉ tr÷í♥❣ ✣↕✐ ❤å❝ ❈➛♥ ❚❤ì✱ ❇❛♥ ❝❤õ ♥❤✐➺♠ ❑❤♦❛
❑❤♦❛ ❍å❝ ❚ü ◆❤✐➯♥✱ ❝→❝ t❤➛② ❝æ ❜ë ♠æ♥ t♦→♥ ❑❤♦❛ ❑❤♦❛ ❍å❝ ❚ü ◆❤✐➯♥ ✤➣ ❣✐ó♣ ✤ï✱ ❤÷î♥❣
❞➝♥ tr♦♥❣ s✉èt t❤í✐ ❣✐❛♥ ❡♠ ❤å❝ t➟♣ t↕✐ tr÷í♥❣✳
✣➦❝ ❜✐➺t✱ ❡♠ ①✐♥ ❝❤➙♥ t❤➔♥❤ ❝→♠ ì♥ ❚❤➛② ▲➙♠ ❍♦➔♥❣ ❈❤÷ì♥❣✱ ❜ë ♠æ♥ t♦→♥ ❑❤♦❛ ❑❤♦❛
❍å❝ ❚ü ◆❤✐➯♥ ✲ ✣↕✐ ❤å❝ ❈➛♥ ❚❤ì ✲ ◆❣÷í✐ trü❝ t✐➳♣ ❤÷î♥❣ ❞➝♥ ❧✉➟♥ ✈➠♥✳ ❚❤➛② ✤➣ t➟♥ t➻♥❤
❣✐ó♣ ✤ï✱ ✤ë♥❣ ✈✐➯♥✱ t↕♦ ♠å✐ ✤✐➲✉ ❦✐➺♥ t❤✉➟♥ ❧ñ✐ ✤➸ ❡♠ ❤♦➔♥ t❤➔♥❤ ❜➔✐ ❧✉➟♥ ✈➠♥✳
❊♠ ①✐♥ ❝❤➙♥ t❤➔♥❤ ❝→♠ ì♥ ❝è ✈➜♥ ❤å❝ t➟♣ ❈æ ❉÷ì♥❣ ❚❤à ❚✉②➲♥ ✤➣ ❞↕② ❞é✱ r➧♥ ❧✉②➺♥✱
❤÷î♥❣ ❞➝♥ s✉èt ❜è♥ ♥➠♠ ❤å❝ t➟♣ ✈➔ ✤➦❝ ❜✐➺t tr♦♥❣ t❤í✐ ❣✐❛♥ ❡♠ ❧➔♠ ❜➔✐ ❧✉➟♥ ✈➠♥✳
❊♠ ❝ô♥❣ ①✐♥ ❣û✐ ❧í✐ ❝→♠ ì♥ ✤➳♥ ❝→❝ ❜↕♥ ❧î♣ ❚♦→♥ Ù♥❣ ❉ö♥❣ ❑✸✷ ✲ ❑❤♦❛ ❑❤♦❛ ❍å❝ ❚ü
◆❤✐➯♥ ✲ ✣↕✐ ❤å❝ ❈➛♥ ❚❤ì ✤➣ tr❛♦ ✤ê✐✱ ❣â♣ þ ❝❤♦ ❜➔✐ ❧✉➟♥ ✈➠♥✳
❈✉è✐ ❝ò♥❣ ❡♠ ①✐♥ ❝↔♠ ì♥ ❣✐❛ ✤➻♥❤✱ t↕♦ ♠å✐ ✤✐➲✉ ❦✐➺♥ ✈➟t ❝❤➜t✱ t✐♥❤ t❤➛♥ tr♦♥❣ t❤í✐ ❣✐❛♥
❡♠ ❤♦➔♥ t❤➔♥❤ ❜➔✐ ❧✉➟♥ ✈➠♥ ❝õ❛ ♠➻♥❤✳
❈➛♥ ❚❤ì✱ ♥❣➔② ✸✵ t❤→♥❣ ✹ ♥➠♠ ✷✵✶✵

✣é ❚❤➔♥❤ ❚➔✐

✷


P é
ỵ ồ t ử ự
tr ổ t ồ ừ rt t tỹ t t
tr ồ ổ õ ổ t sỹ t õ t tớ ừ ởt tố sỹ
t ở ừ tố
ởt tr ỳ ợ q tr q trồ ỵ tt ụ ữ ự ử
õ tr r tr t ồ t ỵ ờ t ữớ
r ữ r t ổ t ở ừ
tỷ t ọ tr ởt s ổ ữủ t tr sỷ ử
tr ỹ ỡ ồ ồ s ồ t
ự ởt q tr õ ởt q tr r õ
r st t t ỡ ừ ú t ởt ọ ợ ổ ữủ t
r õ q tr õ ở tử t ố ởt ữủ
õ ổ ổ t t ờ t ợt ỳ õ ụ
ữ r t qt

ố tữủ ự
rữợ t ữ r tr ỹ tr ỡ s õ ữ r
tr r ử t
r tữỡ ự q tr rớ r t tớ
tr r tữỡ ự q tr tử t tớ
t tr t t ỡ tt tổ q
ỵ ự ử ự ỵ r s ỵ
r

Pữỡ ự

ồ t q s t t ủ ợ tổ
t tứ trt
ữợ sỹ ữợ ừ ữỡ ỹ ồ tự ỡ
q trồ ự ử tr tr ỗ tớ ữ r ử
ồ ự ợ tự ữ r


✷


■❱✳ ❈➜✉ tró❝ ❝õ❛ ❧✉➟♥ ✈➠♥
▲✉➟♥ ✈➠♥ ❣ç♠ ✸ ❝❤÷ì♥❣✿
•

❈❤÷ì♥❣ ✶✿ ◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✈➔ ①➼❝❤ ▼❛r❦♦✈
❚r➻♥❤ ❜➔② ❝→❝ ✤à♥❤ ♥❣❤➽❛ ❝ì ❜↔♥ ✈➲ q✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥✱ ①➼❝❤ ▼❛r❦♦✈ ✈➔ ♠ët sè t➼♥❤
❝❤➜t q✉❛♥ trå♥❣ ❝õ❛ ❝❤ó♥❣✳ ❈❤÷ì♥❣ ♥➔② ❝á♥ ♠ët sè ❦❤→✐ ♥✐➺♠✱ ✤à♥❤ ❧þ ❧➔♠ ♥➲♥ t↔♥❣
❝❤♦ ❝→❝ ❝❤÷ì♥❣ s❛✉✿ ♣❤÷ì♥❣ tr➻♥❤ ❈❤❛♣♠❛♥ ✲ ❑♦❧♠♦❣♦r♦✈✱ ♣❤➙♥ ❜è ❜❛♥ ✤➛✉✱ ♣❤➙♥ ❜è
❞ø♥❣✱ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥✱ ♣❤➙♥ ❧♦↕✐ tr↕♥❣ t❤→✐ ①➼❝❤ ▼❛r❦♦✈ ✳ ✳ ✳

•

❈❤÷ì♥❣ ✷✿ ◗✉→ tr➻♥❤ ▼❛r❦♦✈
❚r➻♥❤ ❜➔② ❦❤→✐ ♥✐➺♠ q✉→ tr➻♥❤ ▼❛r❦♦✈✳ ❈→❝ ✤à♥❤ ♥❣❤➽❛✱ ✤à♥❤ ❧þ ✈➔ t➼♥❤ ❝❤➜t ❝õ❛ q✉→
tr➻♥❤ tr♦♥❣ ❜❛ tr÷í♥❣ ❤ñ♣✿ ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣ t❤→✐ ✤➳♠ ✤÷ñ❝✱ ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣ t❤→✐ ✈æ
❤↕♥ ✤➳♠ ✤÷ñ❝ ✈➔ tr÷í♥❣ ❤ñ♣ tê♥❣ q✉→t✳

•

❈❤÷ì♥❣ ✸✿ ▼ët sè ù♥❣ ❞ö♥❣ ❝❤♦ q✉→ tr➻♥❤ ▼❛r❦♦✈

❚r➻♥❤ ❜➔② ❤❛✐ ù♥❣ ❞ö♥❣ ❝õ❛ q✉→ tr➻♥❤ ▼❛r❦♦✈✳ ❈ö t❤➸ ❝❤ù♥❣ ♠✐♥❤ ♠ët ♣❤➛♥ ✤à♥❤ ❧þ
❍♦♣❢ ❊r❣♦❞✐❝ ✈➔ ✤à♥❤ ❧þ ❣✐î✐ ❤↕♥ tr✉♥❣ t➙♠ ✭●♦r❞✐♥✮✳

✣é ❚❤➔♥❤ ❚➔✐


✷


▼ö❝ ❧ö❝
❈❤÷ì♥❣ ✶✳ ◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✈➔ ❳➼❝❤ ▼❛r❦♦✈
✶✳✶✳
✶✳✷✳
✶✳✸✳
✶✳✹✳
✶✳✺✳
✶✳✻✳

◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳
❳➼❝❤ ▼❛r❦♦✈ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳
P❤➙♥ ❜è ❜❛♥ ✤➛✉ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳
P❤÷ì♥❣ tr➻♥❤ ❈❤❛♣♠❛♥ ✲ ❑♦❧♠♦❣♦r♦✈ ✳
P❤➙♥ ❜è ❞ø♥❣ ✈➔ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥ ✳ ✳
P❤➙♥ ❧♦↕✐ tr↕♥❣ t❤→✐ ①➼❝❤ ▼❛r❦♦✈ ✳ ✳ ✳

✳
✳
✳
✳
✳

✳

✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳

❈❤÷ì♥❣ ✷✳ ◗✉→ tr➻♥❤ ▼❛r❦♦✈
✷✳✶✳
✷✳✷✳
✷✳✸✳
✷✳✹✳

✳
✳
✳
✳
✳
✳

❑❤→✐ ♥✐➺♠ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳
❚r÷í♥❣ ❤ñ♣ ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣ t❤→✐ ❤ú✉ ❤↕♥ ✳ ✳ ✳ ✳ ✳
❚r÷í♥❣ ❤ñ♣ ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣ t❤→✐ ✈æ ❤↕♥ ✤➳♠ ✤÷ñ❝
❚r÷í♥❣ ❤ñ♣ tê♥❣ q✉→t ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳

❈❤÷ì♥❣ ✸✳ ▼ët sè ù♥❣ ❞ö♥❣ ❝❤♦ q✉→ tr➻♥❤ ▼❛r❦♦✈

✳
✳
✳

✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳

✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳

✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳

✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✳
✳
✳
✳
✳
✳
✳
✳
✳
✳

✺

✳ ✺
✳ ✺
✳ ✻
✳ ✼
✳ ✶✶
✳ ✶✼
✳
✳
✳
✳

✸✶

✸✶
✸✷
✸✼
✹✻

✺✶

✸✳✶✳ ✣à♥❤ ❧þ ❣✐î✐ ❤↕♥ tr✉♥❣ t➙♠ ❝❤♦ ①➼❝❤ ▼❛r❦♦✈ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✺✶
✸✳✷✳ ✣à♥❤ ❧þ ❍♦♣❢ ❊r❣♦❞✐❝ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✳ ✺✺

❚➔✐ ❧✐➺✉ t❤❛♠ ❦❤↔♦

✸

✸


✹

▼ö❝ ❧ö❝


❈❤÷ì♥❣ ✶
◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✈➔ ❳➼❝❤ ▼❛r❦♦✈

✶✳✶✳ ◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥
◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ {Xt, t ∈ T } ❧➔ ♠ët t➟♣ ❤ñ♣ ❝→❝ ✤↕✐ ❧÷ñ♥❣ ♥❣➝✉ ♥❤✐➯♥✳ ❈❤➾ sè t ✤÷ñ❝
①❡♠ ♥❤÷ t❤í✐ ❣✐❛♥ ✈➔ Xt ❧➔ tr↕♥❣ t❤→✐ ❝õ❛ q✉→ tr➻♥❤ t↕✐ t❤í✐ ✤✐➸♠ t✳
❱➼ ❞ö ✶✳✶✳ Xt ❧➔ tê♥❣ sè ❦❤→❝❤ ❤➔♥❣ ✈➔♦ s✐➯✉ t❤à t↕✐ t❤í✐ ✤✐➸♠ t ❤♦➦❝ tê♥❣ sè s↔♥ ♣❤➞♠ ❜→♥
✤÷ñ❝ ✤➳♥ t❤í✐ ✤✐➸♠ t✳
❚➟♣ ❤ñ♣ T ✤÷ñ❝ ❣å✐ ❧➔ t➟♣ ❤ñ♣ ❝❤➾ sè ❝õ❛ q✉→ tr➻♥❤✳
✰ T ✤➳♠ ✤÷ñ❝ t❤➻ q✉→ tr➻♥❤ ✤÷ñ❝ ❣å✐ ❧➔ q✉→ tr➻♥❤ rí✐ r↕❝ t❤❡♦ t❤í✐ ❣✐❛♥✳
✰ T ❧➔ ♠ët ❦❤♦↔♥❣ tr➯♥ trö❝ sè t❤ü❝ t❤➻ q✉→ tr➻♥❤ ✤÷ñ❝ ❣å✐ ❧➔ ❧✐➯♥ tö❝ t❤❡♦ t❤í✐ ❣✐❛♥✳
❚➟♣ ❤ñ♣ t➜t ❝↔ ❝→❝ ❣✐→ trà ❝â t❤➸ ❝õ❛ ❝→❝ ✤↕✐ ❧÷ñ♥❣ ♥❣➝✉ ♥❤✐➯♥ ❣å✐ ❧➔ ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣
t❤→✐ ❝õ❛ q✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥✳ ❑➼ ❤✐➺✉ E ✳

✶✳✷✳ ❳➼❝❤ ▼❛r❦♦✈
◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ {Xn, n = 0, 1, 2, . . .} ✭❝â ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣ t❤→✐ E ✤➳♠ ✤÷ñ❝✮ ❧➔ ♠ët
①➼❝❤ ♠❛r❦♦✈ ♥➳✉✿
P {Xn+1 = j|Xn = i, Xn−1 = in−1 , . . . , X0 = i0 } = P {Xn+1 = j|Xn = i} = Pij
✭✶✳✶✮
✈î✐ ♠å✐ tr↕♥❣ t❤→✐ i0, . . . , in−1, i, j ✈➔ ✈î✐ ♠å✐ n ≥ 0✳
Ð ✤➙② t❛ ❝❤ó þ✿
•

•
•

◆➳✉ Xn = i t❤➻ q✉→ tr➻♥❤ ð tr↕♥❣ t❤→✐ t❤ù i t↕✐ t❤í✐ ✤✐➸♠ n✳
●✐→ trà ❝è ✤à♥❤ Pij ❧➔ ①→❝ s✉➜t ✤➸ q✉→ tr➻♥❤ ð tr↕♥❣ t❤→✐ t❤ù i ❜✐➳♥ ✤ê✐ s❛♥❣ tr↕♥❣ t❤→✐
t❤ù j ❤❛② ❝á♥ ❣å✐ ❧➔ ①→❝ s✉➜t ❝❤✉②➸♥✳
◆➳✉ ❜✐➳t tr↕♥❣ t❤→✐ ❤✐➺♥ t↕✐ Xn t❤➻ q✉→ ❦❤ù X0, . . . , Xn−1 ✈➔ t÷ì♥❣ ❧❛✐ Xn+1 ✤ë❝ ❧➟♣ ✈î✐
♥❤❛✉ ✭t➼♥❤ ▼❛r❦♦✈✮✳
✺




ữỡ tr r


st ổ q tr ờ tr t õ
Pij 0,

i, j 0





Pij = 1,

i = 0, 1, . . .

j=0

ỵ P tr ừ st Pij q ữợ ờ
P =

P00
P10
P20
...

P01
P11
P21
...

P02
P12
P22
...

...
...
...
...

ử sỷ trớ ữ ổ ử tở ổ

trớ ữ ổ ổ ử tở tớ tt ừ q sỷ t r
ổ trớ ữ t st trớ ữ ổ trớ ổ ữ

t st trớ ữ
t q tr tr t 0 trớ ữ tr t 1 trớ ổ ữ t q
tr õ s r ỗ tr t {0, 1} ợ tr st ờ
P =

P00 P01
P10 P11

=

1
1

P ố
P ố ừ X0 ữủ ồ ố uiP (X0 = i)
ỵ P ố ỗ tớ ừ (X0, X1, . . . , Xn) ữủ t tứ ố
st
P {X0 = i0 , X1 = i1 , . . . , Xn = in } = ui0 Pi0 i1 . . . Pin1 in

ự ổ tự st t õ

P (X0 = i0 , X1 = i1 , . . . , Xn = in ) = P (X0 = i0 )P (X1 = i1 |X0 = i0 ) ì . . .
ì P (Xk = ik |P (X0 = i0 , . . . , Xk1 = ik1 ) ì . . .
ì P (Xn = in |P (X0 = i0 , . . . , Xn1 = in1 )

ỷ ử t r t ữủ
P (Xk = ik |P (X0 = i0 , . . . , Xk1 = ik1 )) = P (Xk = ik |Xk1 = ik1 )
= Pik1 ik

r

P (X0 = i0 , X1 = i1 , . . . , Xn = in ) = ui0 Pi0 i1 . . . Pin1 in


✼

✶✳✹✳ P❤÷ì♥❣ tr➻♥❤ ❈❤❛♣♠❛♥ ✲ ❑♦❧♠♦❣♦r♦✈

✶✳✹✳ P❤÷ì♥❣ tr➻♥❤ ❈❤❛♣♠❛♥ ✲ ❑♦❧♠♦❣♦r♦✈
❚❛ ❝â✿
Pij = P (Xn+1 = j|Xn = i)

Pijn = P (Xn+k = j|Xk = i) ,

n ≥ 0 i, j ≥ 0

P❤÷ì♥❣ tr➻♥❤ ❈❤❛♣♠❛♥ ✲ ❑♦❧♠♦❣♦r♦✈ ❝❤♦ t❛ ♠ët ❝→❝❤ t➼♥❤ ❝→❝ ①→❝ s✉➜t s❛✉ n ❜÷î❝ ❜✐➳♥
✤ê✐✿
∞

Pijn+m

m
Pikn Pkj
,

=

∀m, n ≥ 0,

∀i, j

✭✶✳✷✮

k=0

❈❤ù♥❣ ♠✐♥❤✳
Pijn+m = P (Xn+m = j|X0 = i)
∞

P (Xn+m = j|Xn = k, X0 = i)P (Xn = k|X0 = i)

=
k=0
∞

m n
Pkj
Pik

=
k=0

✣➦t P (n) ❦þ ❤✐➺✉ ♠❛ tr➟♥ ❝õ❛ ❝→❝ ①→❝ s✉➜t q✉❛ n ❜÷î❝ ❜✐➳♥ ✤ê✐ Pijn✳ ❑❤✐ ✤â✱ ♣❤÷ì♥❣ tr➻♥❤
✭✶✳✷✮ ✤÷ñ❝ ✈✐➳t ❧↕✐ ♥❤÷ s❛✉✿
P (n+m) = P (n) · P (m)

❚❛ ❝â✿

P (2) = P (1+1) = P · P = P 2

P (n) = P (n−1+1) = P (n−1) · P = . . . = P n

❉♦ ✤â✱ ♠❛ tr➟♥ q✉❛ n ❜÷î❝ ❜✐➳♥ ✤ê✐ ❝â t❤➸ t➻♠ ✤÷ñ❝ ❜➡♥❣ ❝→❝❤ ♥❤➙♥ ♠❛ tr➟♥ P ✈î✐ ❝❤➼♥❤
♥â n ❧➛♥✳
●å✐ uni = P (Xn = i)✳ ❑➼ ❤✐➺✉ ✈❡❝t♦r U n = (un1 , . . . , und) ❧➔ ✈❡❝t♦r ❤➔♥❣ ❞ ✲ ❝❤✐➲✉ ♠æ t↔
♣❤➙♥ ❜è ❝õ❛ Xn✱ U = U0 = (u1, u2, . . . , ud) ❧➔ ✈❡❝t♦r ❤➔♥❣ ❞ ✲ ❝❤✐➲✉ ♠æ t↔ ♣❤➙♥ ❜è ❜❛♥ ✤➛✉✳


✽

❈❤÷ì♥❣ ✶✳ ◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✈➔ ❳➼❝❤ ▼❛r❦♦✈

✣à♥❤ ❧þ ✶✳✷✳ ❚❛ ❝â✿
U m+n = U m P n

◆â✐ r✐➯♥❣✿
Un = UP n

❈❤ù♥❣ ♠✐♥❤✳ ❚❤❡♦ ❝æ♥❣ t❤ù❝ ①→❝ s✉➜t ✤➛② ✤õ t❛ ❝â✿
um+n
= P (Xm+n = j)
j
d

=

P (Xm = i)P (Xm+n = j|Xm = i)
i=0
d

n
um
i Pij

=
i=0

❱➼ ❞ö ✶✳✸✳ ❳➨t ❧↕✐ ✈➼ ❞ö ✶✳✷ ♠➔ ð ✤â t❤í✐ t✐➳t ✤÷ñ❝ ①❡♠ ♥❤÷ ❧➔ ✶ ①➼❝❤ ▼❛r❦♦✈ ❣ç♠ ✷ tr↕♥❣

t❤→✐✳ ❈❤♦ α = 0, 7 ✈➔ β = 0, 4✱ t➼♥❤ ①→❝ s✉➜t trí✐ s➩ ♠÷❛ ✈➔♦ ✹ ♥❣➔② tî✐ ❜✐➳t r➡♥❣ ❤æ♠ ♥❛②
trí✐ ♠÷❛❄
▲í✐ ❣✐↔✐✿ ▼❛ tr➟♥ ①→❝ s✉➜t q✉❛ ✶ ❜÷î❝ ❜✐➳♥ ✤ê✐ ❧➔✿
P =

P00 P01
P10 P11

=

0, 7 0, 3
0, 4 0, 6

❱➻ ✈➟②✱
P (2) = P 2 =
=

P (4) = (P 2 )2 =
=

0, 7

0, 4
0, 61
0, 52

0, 3
·
0, 6
0, 39
0, 48

0, 7 0, 3
0, 4 0, 6

0, 61 0, 39
0, 61 0, 39
·
0, 52 0, 48
0, 52 0, 48
0, 5749 0, 4251
0, 5668 0, 4332

❚ø ✤â✱ ①→❝ s✉➜t ❝➛♥ t➼♥❤ P004 = 0, 5749✳


✾

✶✳✹✳ P❤÷ì♥❣ tr➻♥❤ ❈❤❛♣♠❛♥ ✲ ❑♦❧♠♦❣♦r♦✈

❱➼ ❞ö ✶✳✹✳ ❈❤♦ ✭ ξ ✮✱ ♥ ❂ ✵✱ ✶✱ ✷✱ ✳ ✳ ✳ ❧➔ ❞➣② ❝→❝ ✤↕✐ ❧÷ñ♥❣ ♥❣➝✉ ♥❤✐➯♥ ✤ë❝ ❧➟♣✱ ❝ò♥❣ ♣❤➙♥
n

n

❜è✳ ●✐↔ sû P✭ξn❂✐✮ ❂ ai✱ i ∈ Z ✳ ✣➦t Xn ❂ ξi✳ ❑❤✐ ✤â ✭Xn✮ ❧➔ ♠ët ①➼❝❤ ▼❛r❦♦✈ ✈î✐ ❦❤æ♥❣
i=1
❣✐❛♥ tr↕♥❣ t❤→✐ ❩✳

P (Xn+1 = in+1 |X0 = i0 , X1 = i1 , . . . , Xn = in ) = P (Xn + ξn+1 = in+1 |ξ0 = i0 , . . . , ξn = in − in−1 )
= P (ξn+1 = in+1 − in )
= P (Xn+1 = in+1 |Xn = in )

❱➟② ✭ Xn ✮ ❧➔ ♠ët ①➼❝❤ ♠❛r❦♦✈ ✈î✐ ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣ t❤→✐ ❩✳ ❳→❝ s✉➜t ❝❤✉②➸♥ ❧➔ Pij = ai−j ✳
❱➼ ❞ö ✶✳✺✳ ✭▼æ ❤➻♥❤ ❊❤r❡♥❢❡st✮ ❈â ❤❛✐ ❜➻♥❤ ❆✱ ❇ ✈➔ ❝â ❞ q✉↔ ❝➛✉ ✤→♥❤ sè ✶✱ ✷✱ ✳ ✳ ✳ ✱ ❞✳ ❚↕✐
t❤í✐ ✤✐➸♠ ❜❛♥ ✤➛✉ ❝â a q✉↔ ❝➛✉ tr♦♥❣ ❆ ✈➔ ❞ ✲ ❛ q✉↔ ❝➛✉ tr♦♥❣ ❇✳ ❚↕✐ ♠é✐ t❤í✐ ✤✐➸♠ ♥ t❛
❝❤å♥ ♥❣➝✉ ♥❤✐➯♥ ♠ët sè tr♦♥❣ t➟♣ ✶✱ ✷✱ ✳ ✳ ✳ ✱ ❞✳ ❑❤✐ ✤â q✉↔ ❝➛✉ ♠❛♥❣ ❝❤➾ sè ✤÷ñ❝ ❝❤å♥ s➩ ✤÷ñ❝
❝❤✉②➸♥ tø ❜➻♥❤ ✤❛♥❣ ❝❤ù❛ ♥â s❛♥❣ ❜➻♥❤ ❦✐❛✳
❑➼ ❤✐➺✉ Xn ❧➔ sè q✉↔ ❝➛✉ tr♦♥❣ ❜➻♥❤ ❆ t↕✐ t❤í✐ ✤✐➸♠ ♥✳ ❍✐➸♥ ♥❤✐➯♥ Xn ❧➔ ①➼❝❤ ▼❛r❦♦✈✳ ❚❛
t➼♥❤ ①→❝ s✉➜t ❝❤✉②➸♥ P (Xn+1 = j | Xn = i)✳
i
d

❚↕✐ t❤í✐ ✤✐➸♠ Xn = i tr♦♥❣ ❆ ❝❤ù❛ ✐ q✉↔ ❝➛✉ ♥➯♥ ①→❝ s✉➜t ✤➸ ❝❤å♥ ✤÷ñ❝ q✉↔ ❝➛✉ tø ❆ ❧➔
✈➔ q✉↔ ❝➛✉ ♥➔② s➩ ✤÷ñ❝ ❝❤✉②➸♥ s❛♥❣ ❇✳
❱➟② P (Xn+1 = i − 1 | Xn = i) = di ✳

❚÷ì♥❣ tü tr♦♥❣ ❇ ❝❤ù❛ d − i q✉↔ ❝➛✉ ♥➯♥ ①→❝ s✉➜t ✤➸ ❝❤å♥ ✤÷ñ❝ q✉↔ ❝➛✉ tø ❇ ❧➔ d −d i ✈➔
q✉↔ ❝➛✉ ♥➔② s➩ ✤÷ñ❝ ❝❤✉②➸♥ s❛♥❣ ❆✳
❱➟② P (Xn+1 = i + 1 | Xn = i) = d −d i
❈❤♦ ♥➯♥✿
Pij =









 0

i
d
d−i
d

♥➳✉ j = i − 1
♥➳✉ j = i + 1

♥➳✉ j = i + 1, j = i − 1
❱➼ ❞ö ✶✳✻✳ ❑❤✐ ♥❣❤✐➯♥ ❝ù✉ ✈➜♥ ✤➲ ♥❣❤✐➺♥ ❤ót✱ ❦➼ ❤✐➺✉ tr↕♥❣ t❤→✐ 0 ❧➔ ❦❤æ♥❣ ♥❣❤✐➺♥ ✈➔ tr↕♥❣
t❤→✐ 1 ❧➔ ♥❣❤✐➺♥✳ ✣ì♥ ✈à t❤í✐ ❣✐❛♥ ❧➔ ♠ët q✉þ✳
❚❤è♥❣ ❦➯ ♥❤✐➲✉ ♥➠♠ ❝❤♦ t❤➜② ✧①→❝ s✉➜t ✤➸ ♠ët ♥❣÷í✐ ❦❤æ♥❣ ♥❣❤✐➺♥ s❛✉ ♠ët q✉þ ✈➝♥ ❦❤æ♥❣
♥❣❤✐➺♥✧ ❧➔ ✵✱✾✾ ✈➔ ✧①→❝ s✉➜t ✤➸ ♠ët ♥❣÷í✐ ♥❣❤✐➺♥ s❛✉ ♠ët q✉þ ✈➝♥ t✐➳♣ tö❝ ♥❣❤✐➺♥✧ ❧➔ ✵✱✽✽✳


✶✵

❈❤÷ì♥❣ ✶✳ ◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✈➔ ❳➼❝❤ ▼❛r❦♦✈

◆❤÷ ✈➟② tr↕♥❣ t❤→✐ ❝õ❛ ♠ët ♥❣÷í✐ ✭♥❣❤✐➺♥ ❤❛② ❦❤æ♥❣ ♥❣❤✐➺♥✮ ✤÷ñ❝ ♠æ t↔ ❜ð✐ ♠ët ①➼❝❤
▼❛r❦♦✈ ✈î✐ ❤❛✐ tr↕♥❣ t❤→✐ E = {0, 1} ✈î✐ ♠❛ tr➟♥ ①→❝ s✉➜t ❝❤✉②➸♥ ♥❤÷ s❛✉✿
P =

0, 99 0, 01
0, 12 0, 88

●✐↔ sû tr♦♥❣ q✉þ ✶ ❝â ✶✼✪ sè ♥❣÷í✐ ♥❣❤✐➺♥✳ ◆❤÷ ✈➟② ♣❤➙♥ ❜è ❜❛♥ ✤➛✉ ❧➔ U (0) = (0, 83; 0, 17)
❙❛♥❣ q✉þ ✷✱ t❤❡♦ ✤à♥❤ ❧þ ✶✳✷ ♣❤➙♥ ❜è sè ♥❣÷í✐ ♥❣❤✐➺♥ ✈➔ ❦❤æ♥❣ ♥❣❤✐➺♥ s➩ ❧➔✿
U (1) = U (0)P = (0, 83; 0, 17)

0, 99 0, 01
0, 12 0, 88

= (0, 845; 0, 155)

❙❛♥❣ q✉þ ✸ ♣❤➙♥ ❜è sè ♥❣÷í✐ ❦❤æ♥❣ ♥❣❤✐➺♥ ✈➔ ♥❣❤✐➺♥ s➩ ❧➔✿
U(2) = U(1)P = (0, 845; 0, 155)

0, 99 0, 01
0, 12 0, 88

= (0, 855; 0, 145)

❚ù❝ ❧➔ t↕✐ t❤í✐ ✤✐➸♠ ♥➔② ❝â ✶✹✱✺✪ sè ♥❣÷í✐ ♥❣❤✐➺♥✳
❱➼ ❞ö ✶✳✼✳ ●✐↔ sû t❛ ❝â d ❝û❛ ❤➔♥❣ ❦➼ ❤✐➺✉ 1, 2, . . . , d ❝ò♥❣ ❜→♥ ♠ët s↔♥ ♣❤➞♠ ♥➔♦ ✤â✳ ❑❤→❝❤
❤➔♥❣ ❝â t❤➸ ❝❤å♥ ♠✉❛ s↔♥ ♣❤➞♠ ð ♠ët tr♦♥❣ d ❝û❛ ❤➔♥❣ ♥➔② tò② t❤❡♦ sð t❤➼❝❤ ❝õ❛ ❤å ✈➔ tr♦♥❣
tø♥❣ t❤→♥❣ ❤å ❦❤æ♥❣ t❤❛② ✤ê✐ ❝❤é ♠✉❛ ❤➔♥❣✳
●å✐ (Xn) ❧➔ ❝û❛ ❤➔♥❣ ♠➔ ❦❤→❝❤ ❤➔♥❣ ❝❤å♥ ♠✉❛ s↔♥ ♣❤➞♠ ð t❤→♥❣ t❤ù n✳
✣➙② ❧➔ ♠ët ①➼❝❤ ▼❛r❦♦✈ ❝â d tr↕♥❣ t❤→✐✱ ①→❝ s✉➜t ❝❤✉②➸♥ Pij ❧➔ ①→❝ s✉➜t ✤➸ ❦❤→❝❤ ❤➔♥❣

❤✐➺♥ t↕✐ ✤❛♥❣ ♠✉❛ ❤➔♥❣ t↕✐ ❝û❛ ❤➔♥❣ i q✉❛ t❤→♥❣ s❛✉ ❝❤✉②➸♥ s❛♥❣ ♠✉❛ ð ❝û❛ ❤➔♥❣ j ✳ ❳➨t
d = 3 ✈➔ ♠❛ tr➟♥ ①→❝ s✉➜t ❝❤✉②➸♥ ❧➔✿



0, 800 0, 100 0, 100
P =  0, 070 0, 900 0, 030 
0, 083 0, 067 0, 850

●✐↔ sû tr♦♥❣ t❤→♥❣ ✶ ❝û❛ ❤➔♥❣ ✶ ❝❤✐➳♠ ✷✵✪ ❦❤→❝❤ ❤➔♥❣✱ ❝û❛ ❤➔♥❣ ✷ ❝❤✐➳♠ ✺✵✪ ❦❤→❝❤
❤➔♥❣ ✈➔ ❝û❛ ❤➔♥❣ ✸ ❝❤✐➳♠ ✸✵✪ ❦❤→❝❤ ❤➔♥❣✳
❱➟② ♣❤➙♥ ❜è ❜❛♥ ✤➛✉ ❧➔ U (0) = (0, 2; 0, 5; 0, 3)
◗✉❛ t❤→♥❣ ✷ ♣❤➙♥ ❜è ❦❤→❝❤ ❤➔♥❣ tr♦♥❣ ❜❛ ❝û❛ ❤➔♥❣ s➩ ❧➔ U (1) = U (0)P = (0, 22; 0, 49; 0, 29)


✶✶

✶✳✺✳ P❤➙♥ ❜è ❞ø♥❣ ✈➔ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥

◗✉❛ t❤→♥❣ ✸ ♣❤➙♥ ❜è ❦❤→❝❤ ❤➔♥❣ tr♦♥❣ ❜❛ ❝û❛ ❤➔♥❣ s➩ ❧➔ U (2) = U (1)P = (0, 234; 0, 483; 0, 283)
❚÷ì♥❣ tü t❛ ❝â t❤➸ t➼♥❤ ð t❤→♥❣ ✶✷ ♣❤➙♥ ❜è ❦❤→❝❤ ❤➔♥❣ tr♦♥❣ ❜❛ ❝û❛ ❤➔♥❣ s➩ ❧➔
U (11) = (0, 270; 0, 459; 0, 271)

❚ù❝ ❧➔ tr♦♥❣ t❤→♥❣ ✶✷ ❝û❛ ❤➔♥❣ ✶ ❝❤✐➳♠ ✷✼✪ ❦❤→❝❤ ❤➔♥❣✱ ❝û❛ ❤➔♥❣ ✷ ❝❤✐➳♠ ✹✺✱✾✪ ❦❤→❝❤
❤➔♥❣ ✈➔ ❝û❛ ❤➔♥❣ ✸ ❝❤✐➳♠ ✷✼✱✶✪ ❦❤→❝❤ ❤➔♥❣✳

✶✳✺✳ P❤➙♥ ❜è ❞ø♥❣ ✈➔ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥
✣à♥❤ ♥❣❤➽❛ ✶✳✶✳ P❤➙♥ ❜è ❜❛♥ ✤➛✉ U = (u )✱ ✐ ∈ ❊ ✤÷ñ❝ ❣å✐ ❧➔ ♣❤➙♥ ❜è ❞ø♥❣ ♥➳✉ t❛ ❝â
i

Un = U

✈î✐ ♠å✐ ♥ tù❝ ❧➔ uni = ui✱ ∀i ∈ E, ∀n✳ ❑❤✐ ✤â ❞➣② Xn ❝â ❝ò♥❣ ♣❤➙♥ ❜è✳

❚ø ✤à♥❤ ❧➼ ✭ ✶✳✶✮ t❛ s✉② r❛ U = (ui) ❧➔ ♣❤➙♥ ❜è ❞ø♥❣ ♥➳✉ ✈➔ ❝❤➾ ♥➳✉✿
✭✐✮

ui

✭✐✐✮

uj

✵ ✈➔
❂

ui = 1
i∈E

ui Pij ,

∀j ∈ E

i∈E

❱➼ ❞ö ✶✳✽✳ ❈❤♦ ✭X ✮ ❧➔ ♠ët ①➼❝❤ ▼❛r❦♦✈ ❝â ❜❛ tr↕♥❣ t❤→✐ E = {1, 2, 3} ✈î✐ ♠❛ tr➟♥ ①→❝ s✉➜t
❝❤✉②➸♥

n




P =



1
3
1
4
1
6

1
3
1
2
1
3

1
3
1
4
1
2









❚➻♠ t➜t ❝↔ ❝→❝ ♣❤➙♥ ❜è ❞ø♥❣✳
✣➦t ❯ ❂ ①✱ ②✱ ③ ❦❤✐ ✤â ❯ ❧➔ ♣❤➙♥ ❜è ❞ø♥❣ ❦❤✐ ✈➔ ❝❤➾ ❦❤✐ ①✱ ②✱ ③ ❧➔ ♥❣❤✐➺♠ ❦❤æ♥❣ ➙♠ ❝õ❛
❤➺ s❛✉
+ y4
+ y2
+ y4



x + y





●✐↔✐ ♣❤÷ì♥❣ tr➻♥❤ t❛ ✤÷ñ❝ ① ❂

x
3
x
3
x
3

+ z6

+ z3
+ z2
+ z

=
=
=
=

6
10
9
,y = ,z =
25
25
25

x
y
z
1




ữỡ tr r

P ố ứ ổ ớ ụ tỗ t ợ t tỗ t ố
ứ P ố ứ tỗ t t õ t ổ
ỵ sỷ Xn r ợ ổ tr t E = {1,n 2, . . .}n ợ

tr st P = Pij tr st s ữợ P = Pij sỷ
r ợ ồ i, j E tỗ t ợ


lim Pijn = j

n

ợ ổ ử tở õ


j

1

iE



j =

j Pij
iE

j = 0 ợ ồ j E


j = 1
iE

t U = (1, 2, . . .) ố ứ ố ứ t
j = 0 ợ ồ j E t ố ứ ổ tỗ t
=1

iE

ự ờ t t õ
lim Pijn

j =
iE

iE

n

Pijn = 1

lim inf

n

iE

ỷ ử ờ t ữỡ tr r t õ
lim Pkin Pij

i Pij =
iE

iE

n

Pkin Pij

lim inf

n

=

t sj = j

j Pij
iE

0 j E

iE
n+1
lim inf Pkj
n



õ
j

sj =

jE

= j

jE

j Pij
jE iE

j

=
jE

i Pij
iE jE

j

=
jE

i
iE

j

=
jE

Pij
jE

i = 0
iE




✶✸

✶✳✺✳ P❤➙♥ ❜è ❞ø♥❣ ✈➔ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥

❱➟② sj ❂ ✵✱

∀j ∈ E

❤❛② πj =

i∈E

πi Pij ✱

∀j ∈ E

✭✐✐✮ ❚❛ ❝â✿
πj =

πi Pij
i∈E

=

(

πk Pkj )Pij

i∈E k∈E

=

πk (
k∈E

=

Pkj Pij )

i∈E
2
πk Pkj

✭✶✳✻✮

k∈E

❇➡♥❣ q✉② ♥↕♣ ❞➵ t❤➜② ✈î✐ ♠å✐ n
n
πk Pkj

πj =
k∈E

❱➻ ❝❤✉é✐ ❤ë✐ tö ✤➲✉ ✤è✐ ✈î✐ ♥ ♥➯♥✿
n→∞

n
πk lim Pkj
= πj

n
πk Pkj
=

πj = lim

n→∞

k∈E

k∈E

πk
k∈E

❙✉② r❛✿
πj (1 −

∀j ∈ E

πk ) = 0,
k∈E

❱➟② ♥➳✉✿
πk < 1

t❤➻

πj = 0,

∀j ∈ E

k∈E

✭✐✐✐✮ ◆➳✉ πk = 1 t❤➻ tø ❦❤➥♥❣ ✤à♥❤ ✭✐✮ t❛ s✉② r❛ π = (π1, π2, . . .) ❧➔ ♣❤➙♥ ❜è ❞ø♥❣✳ ❚❛
k∈E
❝❤ù♥❣ ♠✐♥❤ ✤➙② ❧➔ ♣❤➙♥ ❜è ❞ø♥❣ ❞✉② ♥❤➜t✳
●✐↔ sû U = (ui) ❧➔ ♣❤➙♥ ❜è ❞ø♥❣✳ ▲➟♣ ❧✉➟♥ ♥❤÷ tr➯♥ t❛ ❝â✿
n
uk Pkj

uj =
k∈E

❱➻ ❝❤✉é✐ ❤ë✐ tö ✤➲✉ ✈î✐ n ♥➯♥✿
n
uk lim Pkj
=

uj =

k∈E

n→∞

u k πj = π j
k∈E

❉♦ ✤â ♥➳✉ πj < 1 t❤➻ ♣❤➙♥ ❜è ❞ø♥❣ ❦❤æ♥❣ tç♥ t↕✐✳ ◆➳✉ πj = 1 t❤➻ π = (π1, π2, . . .)
j∈E
j∈E
❧➔ ♣❤➙♥ ❜è ❞ø♥❣ ❞✉② ♥❤➜t✳




ữỡ tr r

sỷ X r ợ ổ tr t E = {1, 2, . . .} ợ
n

tr st P = (Pij ) tr st s n ữợ P n = (Pijn)
õ r õ ố ợ ợ ồ i, j E tỗ t ợ
lim Pijn = j

n

ợ ổ ử tở i E

jE

j = 1

õ tr ợ

= (1 , 2 , . . .)

t ởt ố st tr
ị ừ ố ợ ữ s
ồ uni = P (Xn = i) tr U n = (un1 , un2 , . . .) tr ổ t
ố ừ Xn õ
P (X0 = i)Pijn

P (Xn = j) =
iE

õ
P (X0 = i) lim Pijn

lim P (Xn = j) =

n

n

iE

=

P (X0 = i)j
iE

= j



ố U n ừ Xn ở tử tợ ố ợ n ợ t õ P (Xn = j) j
ố ợ tỗ t t ố ứ ụ tỗ t
t ố trũ õ ỳ r tỗ t ố ứ
ữ ổ tỗ t ố ợ
ử r Xn õ tr t ợ tr st
P =

0 1
1 0

õ
P 2n =

0 1
1 0


✶✺

✶✳✺✳ P❤➙♥ ❜è ❞ø♥❣ ✈➔ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥
❱➔
0 1
1 0

P 2n+1 =

❉♦ ✤â ❦❤æ♥❣ tç♥ t↕✐ n→∞
lim P n ✳
❚✉② ♥❤✐➯♥ t❛ t❤➜② π = ( 12 , 12 ) ❧➔ ♣❤➙♥ ❜è ❞ø♥❣ ❞✉② ♥❤➜t✳

✣à♥❤ ❧þ ✶✳✹✳ ✭✣✐➲✉ ❦✐➺♥ tç♥ t↕✐ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥ ✈➔ ♣❤➙♥ ❜è ❞ø♥❣✮

❈❤♦ ✭Xn✮ ❧➔ ①➼❝❤ ▼❛r❦♦✈ ✈î✐ ❦❤æ♥❣ ❣✐❛♥ tr↕♥❣ t❤→✐ ❤ú✉ ❤↕♥ E = {1, 2, . . . , d} ✈î✐ ♠❛ tr➟♥ ①→❝
s✉➜t ❝❤✉②➸♥ s❛✉ ♥ ❜÷î❝ ❧➔ P n = (Pijn)✳ ❑❤✐ ✤â ❝â tç♥ t↕✐ ♣❤➙♥ ❜è ❣✐î✐ ❤↕♥ π = (π1, π2, . . . , πd)
✈î✐ πj > 0✱ ∀i ∈ E ❦❤✐ ✈➔ ❝❤➾ ❦❤✐ ①➼❝❤ ❧➔ ❝❤➼♥❤ q✉② t❤❡♦ ♥❣❤➽❛✿
❚ç♥ t↕✐ n0 s❛♦ ❝❤♦
Pijn0 > 0,

∀i, j ∈ E

❈❤ù♥❣ ♠✐♥❤✳ ●✐↔ t❤✐➳t ①➼❝❤ ❧➔ ❝❤➼♥❤ q✉②✳ ❚❛ ❝è ✤à♥❤ ❥ ✈➔ ✤➦t✿
mnj = mini∈E Pijn
Mjn = maxi∈E Pijn

❚❛ ❝â✿

(n+1)

Pij

n
Pik Pkj

=
k

❙✉② r❛✿

Pik mnj = mnj
k

(n+1)

mj

mnj

❱➟② ❞➣② (mnj) ✈î✐ ♥ ❂ ✶✱ ✷✱ ✳ ✳ ✳ ❧➔ ❞➣② t➠♥❣ ✈➔ ❜à ❝❤➦♥ tr➯♥ ❜ð✐ ✶✱ ❞♦ ✤â tç♥ t↕✐ ❣✐î✐ ❤↕♥
lim mnj = aj

n→∞

▲➟♣ ❧✉➟♥ t÷ì♥❣ tü t❛ ❝â ❞➣② (Mjn) ✈î✐ ♥ ❂ ✶✱ ✷✱ ✳ ✳ ✳ ❧➔ ❞➣② ❣✐↔♠ ❜à ❝❤➦♥ ❜ð✐ ✵✱ ❞♦ ✤â tç♥
t↕✐ ❣✐î✐ ❤↕♥✿
lim Mjn = Aj

n→∞

❚❛ ❝â✿ mnj

Pijn

Mjn

❑➼ ❤✐➺✉ r = mini,j Pijn

0

❚❛ ❝â✿ Pikn

0

r.1

Pikn

❞♦ ✤â ✤à♥❤ ❧➼ s➩ ✤÷ñ❝ ❝❤ù♥❣ ♠✐♥❤ ♥➳✉ t❛ ❝❤➾ r❛ aj = Aj ✳
>0

♥➯♥ Pikn

n
rPjk
,

0

Pijn0 +n =

∀i

n
Pikn0 Pkj
k

♥➯♥✿


✶✻

❈❤÷ì♥❣ ✶✳ ◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✈➔ ❳➼❝❤ ▼❛r❦♦✈
n
n
+r
)Pkj
(Pikn0 − rPjk

=
k
mnj

Pikn0

−

n
rPjk

+

n n
Pkj
Pjk

k

2
rPjj n

k

✭✶✳✽✮

= mnj (1 − r) + rPjj2 n

❱➻ ❜➜t ✤➥♥❣ t❤ù❝ ♥➔② ✤ó♥❣ ✈î✐ ♠å✐ ✐ ♥➯♥ t❛ ❝â✿
❚÷ì♥❣ tü t❛ ❝â✿
❙✉② r❛✿

mnj 0 +n

mnj (1 − r) + rPjj2 n

Mjn0 +n

Mjn (1 − r) + rPjj2 n

Mjn0 +n − mnj 0 +n

(1 − r)(Mjn − mnj )

(∗)

(1 − r)k (Mj1 − m1j )

(∗∗)

❇➡♥❣ q✉② ♥↕♣ t❛ t❤➜②✿
Mjkn0 +1 − mjkn0 +1

❚❤➟t ✈➟② ð ✭✯✮ ❦❤✐ ❝❤♦ n ❂ ✶ t❤➻ ✭✯✯✮ ✤ó♥❣ ✈î✐ k ❂ ✶✳
●✐↔ sû ✤ó♥❣ ✈î✐ ❦✱ t❛ ❝â✿
[(k+1)n0 +1]

Mj

[(k+1)n0 +1]

− mj

❈❤♦ k → ∞ ð ✭✯✯✮ t❛ ✤÷ñ❝ Aj − aj

0✳

0 +1+n0
= Mjkn0 +1+n0 − mkn
j
0 +1
(1 − r)[Mjkn0 +1 − mkn
]
j
1
(k+1)
1
(1 − r)
(Mj − mj )

❱➻ Aj − aj

0

♥➯♥ t❛ ❦➳t ❧✉➟♥ Aj = aj

(i,j)
✣↔♦ ❧↕✐ ❣✐↔ sû ∀i, j ∈ E tç♥ t↕✐ n→∞
lim Pijn = πj > 0✳ ❑❤✐ ✤â tç♥ t↕✐ n0 s❛♦ ❝❤♦

Pijn > 0,

✣➦t n0 = maxi,j n(i,j)
t❛ ❝â Pijn > 0,
0

(i,j)

∀n > n0

∀i, j ∈ E,

∀n > n0

✭✶✳✾✮

P tr t r

ử ộ ữớ tr ởt ũ õ õ t tr t ợ

ừ ồ õ t tở ởt tr t ợ õ tr ợ st
tý tở ồ tr t ợ
sỷ tố ữớ t ữủ
ởt ữớ t ợ st ồ ồ
ồ
ởt ữớ t ợ st ồ ồ ồ

ởt ữớ t ợ st ồ ồ
ồ
ữ sỹ t ờ tr t ừ ởt tr ở tứ t q t
õ t ổ t ởt r tr t ợ st
ữ s



0, 448 0, 484 0, 068
P = 0, 054 0, 699 0, 247
0, 011 0, 503 0, 486

r q tỗ t ố ợ = (1, 2, 3) P ố
ố ứ t ữủ t ữỡ tr s
(1 , 2 , 3 )P = (1 , 2 , 3 )

ữỡ tr t ữủ 1 2 3 ữ q t
ũ ữ õ tr s õ ữớ ữớ

P tr t r
õ r tr t i ữủ tr t j tỗ t n 0 s
Pijn > 0

i j

ữợ
Pii0 = 1

i = j
tr t i j ữủ ồ tổ ợ i j j i
ỵ i j
Pij0 = 0


✶✽

❈❤÷ì♥❣ ✶✳ ◗✉→ tr➻♥❤ ♥❣➝✉ ♥❤✐➯♥ ✈➔ ❳➼❝❤ ▼❛r❦♦✈

❚➼♥❤ ❝❤➜t ✶✳✶✳ ✭✐✮ ❚r↕♥❣ t❤→✐ i ✤➲✉ ❧✐➯♥ t❤æ♥❣ ✈î✐ tr↕♥❣ t❤→✐ i✱ ✈î✐ ♠å✐ i ≥ 0✳

✭✐✐✮ ◆➳✉ tr↕♥❣ t❤→✐ i ❧✐➯♥ t❤æ♥❣ ✈î✐ tr↕♥❣ t❤→✐ j ✱ t❤➻ tr↕♥❣ t❤→✐ j ❝ô♥❣ ❧✐➯♥ t❤æ♥❣ ✈î✐ tr↕♥❣
t❤→✐ i✳
✭✐✐✐✮ ◆➳✉ tr↕♥❣ t❤→✐ i ❧✐➯♥ t❤æ♥❣ ✈î✐ tr↕♥❣ t❤→✐ j ✱ tr↕♥❣ t❤→✐ j ❧✐➯♥ t❤æ♥❣ ✈î✐ tr↕♥❣ t❤→✐ k✱ t❤➻
tr↕♥❣ t❤→✐ i ❝ô♥❣ ❧✐➯♥ t❤æ♥❣ ✈î✐ tr↕♥❣ t❤→✐ k✳
✣à♥❤ ♥❣❤➽❛ ✶✳✺✳ ❳➼❝❤ ▼❛r❦♦✈ ✤÷ñ❝ ❣å✐ ❧➔ tè✐ ❣✐↔♥ ♥➳✉ ❤❛✐ tr↕♥❣ t❤→✐ ❜➜t ❦➻ ❧➔ ❧✐➯♥ t❤æ♥❣✳
❱➼ ❞ö ✶✳✶✶✳ ❈❤♦ ①➼❝❤ ▼❛r❦♦✈ ✈î✐ ❜è♥ tr↕♥❣ t❤→✐ E = {1, 2, 3, 4} ✈➔ ♠❛ tr➟♥ ①→❝ s✉➜t ❝❤✉②➸♥
❧➔

1 1 

0


 0

P = 1


 2
1
2

0

0
1
2
1
2

2
1
2
0

2
1
2
0

0

0










❳➼❝❤ ♥➔② ❧➔ tè✐ ❣✐↔♥✳
❚❤➟t ✈➟②✿ 1 ↔ 3, 1 ↔ 4, 2 ↔ 3, 2 ↔ 4
❚❛ ❝â✿
1 → 3, 3 → 2

s✉② r❛ 1 → 2

2 → 3, 3 → 1

s✉② r❛ 2 → 1

❱➟② 1 ↔ 2
❚÷ì♥❣ tü 3 ↔ 4
❑➳t ❧✉➟♥ ❤❛✐ tr↕♥❣ t❤→✐ ❜➜t ❦➻ ❧➔ ❧✐➯♥ t❤æ♥❣ ❞♦ ✤â ✤➙② ❧➔ ①➼❝❤ tè✐ ❣✐↔♥✳
✣à♥❤ ♥❣❤➽❛ ✶✳✻✳ ❈❤✉ ❦➻ ❝õ❛ ntr↕♥❣ t❤→✐ ✐ ❦➼n ❤✐➺✉ ❧➔ ❞✭✐✮ ❧➔ ÷î❝ ❝❤✉♥❣ ❧î♥ ♥❤➜t ❝õ❛ t➜t ❝↔ ❝→❝
sè ♥❣✉②➯♥ ❞÷ì♥❣ n 1 ♠➔ Pii 0✳ ◆➳✉ Pii = 0 ✈î✐ ♠å✐ n 1 t❤➻ t❛ q✉② ÷î❝ ✤➦t ❞✭✐✮ ❂ ✵✳
✣à♥❤ ❧þ ✶✳✺✳ ◆➳✉ i ↔ j t❤➻ ❞✭✐✮ ❂ ❞✭❥✮✳ ❱➟② ❝→❝ tr↕♥❣ t❤→✐ ❝ò♥❣ ♠ët ❧î♣ ❝â ❝ò♥❣ ♠ët ❝❤✉ ❦➻

❞ ✈➔ t❛ ❣å✐ sè ❞ ❝❤✉♥❣ ✤â ❧➔ ❝❤✉ ❦➻ ❝õ❛ ❧î♣✳
❈❤ù♥❣ ♠✐♥❤✳ ❉♦ i ↔ j ♥➯♥ tç♥ t↕✐ k✱ l s❛♦ ❝❤♦ Pijk > 0, Pijl > 0✳ ❚❤❡♦ ♣❤÷ì♥❣ tr➻♥❤ ❈❤❛♣♠❛♥
✲ ❑♦❧♠♦❣♦r♦✈✱ t❛ ❝â✿
(k+l)

Pii

k l
Pih
Phi

=
h∈E

Pijk Pjil > 0




P tr t r

d(i)|(k + l)
sỷ n

1

s Pjjn > 0

ỷ ử ữỡ tr r ữ tr t õ
(k+l+n)

Pijk Pjjn Pjil > 0

Pii

d(i)|(k + l + n) d(i)|d(j)
ữỡ tỹ d(j)|d(i)
r d(i) = d(j)
fiin st t t tứ i t q i tớ
n
fiin = P (Xn = i, Xn1 = i, . . . , X1 = i | X0 = i)





fii =

fiin
n=1

st t t tứ i q tr i s ởt số ỳ ữợ fii = 1 t õ i
tr t ỗ q ữủ fii < 1 t õ i tr t ổ ỗ q
ỵ r t i ỗ q


Piin =
n=1

ự ỷ ử ờ

ờ õ
n

fiik Piink

Piin =

tr õ

fii0 = 0

k=0

ự ờ
ợ ộ 0
õ

k

n

ồ Ek ố Xn = i t q i ữợ tự k