Introduction to matrix analytic methods in stochastic modeling

www.pdfgrip.com

Introduction
to Matrix Analytic
Methods in

Stochastic Modeling


www.pdfgrip.com

ASA-SIAM Series on

Statistics and Applied Probability

T

he ASA-SIAM Series on Statistics and Applied Probability is published jointly by
the American Statistical Association and the Society for Industrial and Applied Mathematics. Th
series consists of a broad spectrum of books on topics in statistics and applied probability. The
purpose of the series is to provide inexpensive, quality publications of interest to the intersecting
membership of the two societies.

Editorial Board
Donald P. Gaver

Andrew Solow

Alan F. Karr

Werner Stuetzle

John Lehoczky

Grace Wahba

Naval Postgraduate School, Editor-in-Chief
National Institute of Statistical Sciences
Carnegie Mellon University

Robert L Mason

Southwest Research Institute

Woods Hole Oceanographlc Institution

University of Washington
University of Wisconsin

Eric Ziegel

Amoco Corporation

Robert Rodriguez
SAS Institute
Latouche, G. and Ramaswami, V., Introduction to Matrix Analytic Methods in Stochastic Modeling
Peck, R., Haugh, L. D., and Goodman, A., Statistical Case Studies: A Collaboration Between Academe
and Industry, Student Edition
Peck, R., Haugh, L D., and Goodman, A., Statistical Case Studies: A Collaboration Between Academe
and Industry

Barlow, R. E., Engineering Reliability
Czitrom, V. and Spagon, P. D., Statistical Case Studies for Industrial Process Improvement


www.pdfgrip.com

Introduction to
Matrix Analytic
Methods in
Stochastic Modeling
G. Latouche
University Libre de Bruxelles
Brussels, Belgium

V. Ramaswami
AT&T Labs
Holmdel, New Jersey

Society for Industrial and Applied Mathematics
Philadelphia, Pennsylvania

American Statistical Association
Alexandria, Virginia


www.pdfgrip.com

© 1999 by the American Statistical Association and the Society for Industrial and Applied
Mathematics.
10987654321

All rights reserved. Printed in the United States of America. No part of this book may be
reproduced, stored, or transmitted in any manner without the written permission of the
publisher. For information, write to the Society for Industrial and Applied Mathematics,
3600 University City Science Center, Philadelphia, PA 19104-2688.

Library of Congress Cataloging-in-Publication Data
Latouche, G (Guy)
Introduction to matrix analytic methods in stochastic modeling / G. Latouche, V.
Ramaswami.
p. cm. — (ASA-SIAM series on statistics and applied
probability)
Includes bibliographical references and index.
ISBN 0-8987M25-7 (pbk.)
1. Markov processes. 2. Queuing theory. 3. Matrices.
I. Ramaswami, V. II. Title. III. Series
QA274.7.L38 1999
519.2'3.-dc21
98-48647

EuaJlL is a registered


www.pdfgrip.com

Contents
Preface

ix

I

Quasi-Birth-and-Death Processes

1

1 Examples
1.1 The M/M/1 Queue
1.2 The M/M/1 Queue in a Random Environment
1.3 Phase-Type Queues
1.4 A Queue with Two Priority Classes
1.5 Tandem Queues with Blocking
1.6 Multiprogramming Queues
1.7 Open Jackson Networks
1.8 A Retrial Queue

3
3
5
8
18
19
21
25
28

II

The Method of Phases

2 PH

2.1
2.2
2.3
2.4
2.5
2.6
2.7
2.8

Distributions
The Exponential Distribution
Simple Generalizations
PH Random Variables
Distribution and Moments
Discrete PH Distributions
Closure Properties
Further Properties
Algorithms
v

31
33
34
36
39
41
47
50
55
56

www.pdfgrip.com

vi

Contents

3 Markovian Point Processes
3.1 The PH Renewal Process
3.2 The Number of Renewals
3.3 The Stationary Process
3.4 Discrete Time PH Renewal Processes
3.5 A General Markovian Point Process
3.6 Analysis of the General Process

61
61
65
70
71
72
75

III

81

The Matrix-Geometric Distribution

4 Birth-and-Death Processes
4.1 Terminating Renewal Processes
4.2 Discrete Time Birth-and-Death Processes
4.3 The Basic Nonlinear Equations
4.4 Transient Distributions
4.5 Continuous Time Birth-and-Death Processes
4.6 Passage and Sojourn Times

83
84
87
94
99
99
102

5 Processes Under a Taboo
5.1 Expected Time to Exit
5.2 Linking Subsets of States
5.3 Local Time
5.4 Gaussian Elimination
5.5 Continuous Time Processes

107
108
109
118
120
122

6 Homogeneous QBDs
6.1 Definitions
6.2 The Matrix-Geometric Property
6.3 Boundary Distribution
6.4 Continuous Time QBDs

129
129
130
139
141

7 Stability Condition
7.1 First Passage Times
7.2 Simple Drift Condition
7.3 Drift Conditions in General

147
147
151
159


www.pdfgrip.com

Contents

IV

Algorithms

vii

163

8 Algorithms for the Rate Matrix
8.1 A Basic Algorithm
8.2 Analysis of the Linear Algorithm
8.3 An Improved Algorithm
8.4 Quadratically Convergent Algorithms
8.5 Special Structure
8.6 Boundary Probabilities .
8.7 The Continuous Time Case

165
167
175
179
187
197
199
199

9 Spectral Analysis
9.1 The Caudal Characteristic
9.2 Examples
9.3 The Eigenvalues of the Matrix G
9.4 The Jordan Normal Form
9.5 Construction of the Towers

203
204
206
212
213
216

10 Finite QBDs
10.1 Linear Level Reduction
10.2 The Method of Folding
10.3 Matrix-Geometric Combination
10.4 Subdiagonal Matrices of Rank 1

221
222
228
230
235

11 First Passage Times
11.1 Generating Functions
11.2 Linear Level Reduction
11.3 Reduction by Bisection
11.4 Odd-Even Reduction

239
240
241
245
249

V

Beyond Simple QBDs

257

12 Nonhomogeneous QBDs
12.1 The Stationary Distribution
12.2 Algorithmic Approaches

259
260
263


www.pdfgrip.com

viii

Contents

13 Processes, Skip-Free in One Direction
13.1 Markov Chains of M/G/1-Type
13.2 Markov Chains of GI/M/1-Type

267
268
275

14 Tree Processes
14.1 The M/PH/1 LIFO Queue
14.2 Tree-Like Transition Diagrams
14.3 Matrix-Product Form Distribution
14.4 Skip-Free Processes

281
281
284
289
294

15 Product Form Networks
15.1 Independence of Level and Phase
15.2 Network of Exponential Servers
15.3 The General Case

295
295
299
302

16 Nondenumerable States
16.1 General Irreducible Markov Chains
16.2 The Operator-Geometric Property
16.3 Computational Issues

305
305
307

310

Bibliography

313

Index

325


www.pdfgrip.com

Preface
Matrix analytic methods constitute a success story, illustrating the enrichment of a science, applied probability, by a technology, that of digital computers. Marcel Neuts has played a seminal role in these exciting
developments, promoting numerical investigation as an essential part
of the solution of probability models. He wrote in 1973,
"To do work in computational mathematics is ... a commitment to a more demanding definition of what constitutes the
solution to a mathematical problem. When done properly,
it conforms to the highest standard of scientific research."
(see [73, Page 20]).
This had been long accepted among numerous scientific communities but was not at the time the prevalent view among applied probabilists. Neuts's program of research led in a few years to the celebrated
"matrix-geometric distribution" and "phase-type processes," brought
nowadays under the umbrella title of "matrix analytic methods."
The excitement one feels when dealing with that subject stems from
the synergy resulting from keeping algorithmic considerations in the
forefront when solving stochastic problems. The connection was already
noted in 1959 by Kemeny and Snell who wrote,
"One of the practical advantages of this new treatment of
the subject is that these elementary matrix operations can

easily be programmed for a high-speed computer. The authors have developed a pair of programs ... which will find a
number of interesting quantities for a given problem directly
from the transition matrix. These programs were invaluable
IX


www.pdfgrip.com

x

Preface
in the computation of examples and in the checking of conjectures for theorems.
A significant feature of the new approach is that it makes
no use of the theory of eigen-values. The authors found, in
each case, that the expressions in matrix form are simpler
than the corresponding expressions usually given in terms
of eigen-values. This is presumably due to the fact that the
fundamental matrices have direct probabilistic interpretations, while the eigen-values do not." (see [39, Page vj).

Matrix analytic methods are popular as modeling tools because they
give one the ability to construct and analyze, in a unified way and in
an algorithmically tractable manner, a wide class of stochastic models.
The methods are applied in several areas, of which the performance
analysis of telecommunication systems is one of the most notable at
the present time. The methods also offer to mathematicians the delight of discovering the stochastic process at work in the computational
procedure, as when one finds that the successive steps in an iterative
algorithm have a probabilistic significance.
This book presents the basic mathematical ideas and algorithms of
the matrix analytic theory. Our approach uses probabilistic arguments
to the fullest extent and allows us to show clearly the unity of argumentation in the whole theory. With this in mind, we have had to develop

new proofs for some of the results so as to shed the dependence on
the Perron-Probenius theory of finite dimensional matrices, fixed point
theorems of functional analysis, etc., which are prevalent in some of
the early texts on the subject. While most of these probabilistic proofs
have been published over the years, a few appear here for the first time.
The methods themselves are presented within the simpler framework of quasi-birth-and-death processes (QBDs). These are Markov
processes in two dimensions, the level and the phase, such that the
process does not jump across several levels in one transition. The advantage of working with QBDs, instead of the more general GI/M/1and M/G/1-type Markov chains, is that we may present the basic features of matrix analytic methods without being encumbered by side
issues which arise as soon as the processes are put into some application framework. The restriction to QBDs, however, is not unduly


www.pdfgrip.com

Preface

xi

limiting; we show in Chapter 13 that the theory for the more general
class of models can be deduced from the QBD analysis.
We separate as much as possible the elucidation of the structure
of some quantity from the determination of its numerical value. One
advantage in presenting the probabilistic results separate from the algorithms is that we make it clearly appear that the structural properties
do not depend on whether there are finitely or infinitely many values
for the phase dimension: only when doing actual matrix computations
does it become necessary to deal with a finite state space for the phase.
A major hurdle to overcome, in order to delve into matrix analytic
theory, is to start thinking in terms of blocks of states and transition
submatrices, instead of keeping track of individual states and scalar
transition probabilities. We gently lead the reader in that direction
by introducing one aspect of the theory at a time. To begin with,

we present several simple examples from queueing theory in the first
chapter. They are intended to help the reader become familiar with the
matrix notations which we use throughout the text, as well as to get
some feeling for the variety of models which are hidden by the general
block notations. Chapter 1 may safely be skipped by readers who wish
to jump immediately into the mathematics.
We next give two chapters on phase-type distributions and phasetype processes. Chapter 2 is the place where individual states lose
their identity and merely become members of a vague "phase set";
by drawing analogies with the exponential distribution, we show that
most of the results may be seen as matrix versions of familiar scalar
properties. In Chapter 3 we associate a counter to the phases, thereby
creating a rudimentary two-dimensional process which is the matrix
generalization of the Poisson process.
The structure of the stationary distribution of QBDs is determined
in Chapters 4 and 6. Chapter 4 deals with birth-and-death processes.
To present that material, we heavily make use of renewal theory; this
is an unusual approach since we do not rely as much as one usually
does on specific features of the birth-and-death models which lead to
an easier analysis. It is, however, that approach which we extend to
the general case, and we think it is advantageous to give it first in
Chapter 4, without the added complexity of having to deal, as in later
chapters, with matrices instead of scalar quantities.


www.pdfgrip.com

xii

Preface

The properties we analyze in the intermediary Chapter 5 are fundamental to our arguments and are repeatedly used in the remainder of
the text. In addition, together with Chapter 4, this chapter gives us the
opportunity to present in detail both discrete and continuous time processes. We show the strong similarity between the arguments which we
use in both cases, as well as the equations which are obtained. In later
chapters, this allows us to develop the theory for discrete time QBDs
and to skip the details of the derivation of the results for the continuous
time processes. Finally, we show how the application of theorems from
Markov chain theory may be used to interpret the Gaussian elimination
method for the solution of linear systems.
Chapter 8 is very important as it covers material where algorithmic
and probabilistic reasoning are most intimately connected. In three
steps, we take the reader from one of the simplest iterative procedures
to the fastest, relating the successive approximations to the dynamic
behavior of the stochastic process itself. The quadratic convergence of
the fastest algorithm is proved by a probabilistic argument. At present,
work is progressing on developing computational procedures based on
the spectral analysis of various component matrices. In Chapter 9 we
briefly give the argument showing why the spectral approach works
in principle. We do not elaborate on this material as probabilistic
reasoning is not very prominent and reported numerical experience is
scanty. Chapters 10 and 11 on QBDs with a finite number of levels
give us the opportunity to bring together material from several different
authors, about different problems, and to show how these results may
be interpreted in the light of the general properties in Chapter 5.
Lest a reader should leave this book feeling that the study of QBDs
is a closed subject, we conclude with five extremely short chapters on
various major extensions, ranging from the mostly algorithmic to the essentially structural: QBDs with level-dependent transitions, processes
skip-free in one direction, processes on a tree-like state space, product
form networks, and processes with a general state space for the phase
dimension.

We barely mention M/G/1- and GI/M/1-type Markov chains. This
glaring omission of an important subject might be explained away by
the fact that these may be viewed as special cases of QBDs, as we
prove in Chapter 13. More to the point, we find that these processes


www.pdfgrip.com

Preface

xiii

are better discussed in the context of the queueing or other applications
from which they stem; in them arise a number of questions, such as the
study of waiting time distributions, etc., which clearly are beyond our
present goal to describe the fundamental aspects of matrix analytic
methods.
We have presented the material in different short courses to various
audiences. It has been received well, both by newcomers to the area
and by those who have some familiarity with the subject matter. Our
nearly fanatical devotion to probabilistic reasoning makes it possible for
us to rely on basic properties for the most part, about Markov processes
and renewal theory. The prerequisites are advanced calculus and linear
algebra, at the level usually taught in undergraduate curricula, and a
course in stochastic processes at the level of Qinlar [12], Feller [20, 21],
Karlin and Taylor [36, 37], or Resnick [106]. In the few places where
more advanced material is required, in particular in Chapter 7, we
point the reader to specific references. Our bibliography does not form
an exhaustive survey of the field but only contains references cited in
the text.

Chapters 1-6, 8, 10, and 12 constitute the core of the methodology and may form the basis for a one-semester course at the senior
undergraduate or graduate level. Chapter 7 on ergodicity conditions
is essential but not easy, and an instructor might choose to walk the
students through the main ideas. Chapter 9 on spectral analysis is peripheral, except for its first two sections. The reader who has absorbed
the core material may work her way through Chapter 11 on passage
times without the need of an instructor. The last chapters are on extensions and areas of work that have opened up recently; they are more
appropriate for seminar topics after a basic course.
It is a pleasant duty to acknowledge the influence of our teachers,
who instilled in us the desire to pursue mathematical research and who
shaped our interest for probability theory and algorithmic thinking:
Guy Louchard and Jean Teghem of the Universite Libre de Bruxelles
and K. Balasubramanian and K. N. Venkataraman of the University of
Madras. Marcel Neuts has been a worthy mentor and role model to
both of us; this book grew out of the sapling he planted and nurtured
over many years.


www.pdfgrip.com

xiv

Preface

Many colleagues and students have read the manuscript at several stages of its development and have pointed out errors and possible improvements. Our heartfelt thanks go to Allan T. Andersen,
Nigel Bean, Nadjet Benseba, Brigitte Bertrand, Olivier De Deckere,
Ed Kao, Bernard Larue, Herlinde Leemans, David Lucantoni, Marcel
F. Neuts, Bo Priis Nielsen, Colm O'Cinneide, Marie-Ange Remiche,
Volker Schmidt, and Peter Taylor.
A special mention must be made of Don Gaver, the series editor, who
encouraged us and acted decisively at the right time and of Pascaline

Browaeys, who made this project her own and tirelessly prodded us
back to work whenever she felt we were straying away.
The first author has found much stimulation in the scholarly atmosphere of the Universite Libre de Bruxelles and benefited much from
visits to Bellcore, Purdue University, University of Delaware, University
of Adelaide, and the Naval Post Graduate School in Monterey.
The second author owes the benefit of higher education to the willingness of his mother and her parents to endure abject poverty for a
longer period. Instrumental to his progress have also been Charles D.
Pack and Patricia Wirth, two perceptive managers who understand the
value and power of innovation. He also thanks Bellcore and AT&T for
providing an environment where he could indulge in scholarly pursuits
and enjoy their fruition in meaningful practical uses.
Our families have provided an enviable refuge from worldly pressures
and constant understanding at those not infrequent times when we are
there but in body. To Monique and Soundaram, to Michael, Cindy,
Priya, and Prem, this book is dedicated.

G. LATOUCHE
V. RAMASWAMI


www.pdfgrip.com

Part I
Quasi-Birth-and-Death
Processes

i


www.pdfgrip.com

This page intentionally left blank


www.pdfgrip.com

Chapter 1
Examples
We begin by describing several examples of quasi-birth-and-death processes (QBDs). These examples serve various purposes. First, they
are intended to help the reader acquire some feeling for the range of
applications of QBDs. Our second purpose is to motivate some of the
terminology and notations which will be used throughout the text. Finally, these examples will be used as illustrations in later chapters.
Our presentation here will not be very formal. The reader who already has some familiarity with the subject may safely skip this chapter;
all the definitions introduced here will be restated later with precision.

1.1

The M/M/1 Queue

A birth-and-death process on the nonnegative integers is a Markov
process in which the only allowed transitions are from the state n to
the next higher state n + 1 for all n > 0 and from n to n — I for n > I.
The canonical example of homogeneous birth-and-death processes is
the M/M/1 queue. This is a single server queueing system with infinite
waiting room and is customarily described by the diagram of Figure
1.1. Customers join the system at the renewal epochs of a Poisson process with parameter A. They enter a waiting room if there are other
customers already present, or immediately begin service if no other
customer is present. Customers are served in the order of arrival. The
durations of services are independent and identically distributed (i.i.d.)
3

www.pdfgrip.com

4

Chapter 1. Examples

Figure 1.1: Diagrammatic representation of the M/M/1 queue.
The arrival rate of new customers is A; the service rate is //.

random variables, independent of the arrival process. The service time
distribution is exponential with parameter //, i.e., with probability density f(x) = p,e~^x for x > 0.
We denote by N(t) the number of customers present in the system
at time £, either waiting or being served. The process {N(t), t > 0} is a
continuous time Markov chain on the state space {0,1,2,...}. The possible transitions, and the corresponding instantaneous rates, are given
in the table below.

Prom
To Rate
n
n-l
for n > 1
M
for n > 0
n
n+1
A

Suppose time t is such that N(t) = n. Then the probability that

N(t + h) = n + 1, due to an arrival in (£, t + /i), is Xh + o(ti). If n > 1,
then the server is busy at time t and the probability that a service is
completed is (j,h + o(h).
Another characterization sometimes used for Markov processes is
the state transition diagram. We present that of the M/M/1 queue in
Figure 1.2.
A third characterization is in terms of the infinitesimal generator
Q, the matrix such that q^ is the instantaneous transition rate from i
to j for i j £ j and such that qn =• — £vj# %• It is well known that |^|
is the parameter of the exponentially distributed sojourn time in i. For


www.pdfgrip.com

1.2. The M/M/1 Queue in a Random Environment

5

Figure 1.2: State transition diagram for the M/M/1 queue.
The arrows represent the possible transitions and are labeled
with the corresponding instantaneous rates.

the M/M/1 queue, we have that

The most important feature here is that Q is a tridiagonal matrix: the
elements of the upper diagonal are all equal; so are the elements of
the lower diagonal; as for the elements on the main diagonal, with the
exception of the upper left corner, the rest are equal.
The singularity of the upper left element reflects the fact that state
0 is a boundary state: we see in Figure 1.2 that 0 is the only state from

which it is impossible to move to the left.
Remark 1.1.1 Unless otherwise stated, sample paths are right continuous, i.e., "state at time t" means "state at time i+," so that if t is
an epoch of transition, then the state at t is the new state entered into.

1.2

The M/M/1 Queue in a Random Environment

A simple example of a QBD is the M/M/1 queue in a Markovian environment (Neuts [79, Chapter 6]). In short, this system behaves like
an M/M/1 queue, but the arrival and service rates vary over time. In


www.pdfgrip.com

Chapter 1. Examples

6

addition to the arrival and service processes, one defines a so-called environmental process {E(t), t > 0} on a finite state space {1,2,..., ra}
with instantaneous transition rates s^;, 1 < i =^ j' < m.
The environment controls the arrival and service processes as follows. Suppose time t is such that E(t) = j. Then the arrival rate is Xj
and the service rate is /ij, provided that the server is busy at time t.
Thus, the whole system is a two-dimensional continuous time Markov
chain {(N(t), E(t)), t > 0} on the state space {(n, i)] n > 0,1 < i < m},
where N(t) is the number of customers present and E(t) is the state of
the environment at time t.
Changes of state occur when the environment changes, when a new
customer arrives, or when a service is completed. The possible transitions, and the corresponding instantaneous rates, are given in the table
below.
Prom

(n, i)
(n,i)
(n,i)

To
(n- l,i)
(nj)
(n + l,t)

Rate

m

Sij

Ai

for n > 1
for n > 0, i ^ j
for n > 0

It would be unnecessarily confusing to depict the state transition
diagram in full generality, and therefore we show in Figure 1.3 a particular example only. In that example, there are m = 3 states in the
environment and the instantaneous rates 521 and 513 are equal to zer
In order to display the infinitesimal generator Q of the system, it is
necessary to define a linear ordering of the states. Of particular interest
is the lexicographical ordering {(0,1), ( 0 , 2 ) , . . . , (0, m), (1,1), (1,2),...,
(1, m), (2,1), ( 2 , 2 ) , . . . , (2, m ) , . . . , (n, 1), (n, 2 ) , . . . , (n, m),...}: we firs
enumerate all the states with 0 customers, then all the states with 1
customer, then all the states with 2 customers, etc.

We shall call by level the subset of all states corresponding to a fixed
number of customers in the system. The levels appear as columns in the
diagram of Figure 1.3 while the rows in that diagram correspond to the
various environmental states. The lexicographical ordering corresponds
to the enumeration of the states level by level.
The infinitesimal generator Q corresponding to the diagram of Figure 1.3 is given below. We have labeled each row and column by the


www.pdfgrip.com

1.2. The M/M/1 Queue in a Random Environment

7

Figure 1.3: State transition diagram for the M/M/1 queue in
a random environment. There are 3 environmental states. The
arrows represent the possible transitions; for this example, it is
assumed that $21 and 813 are equal to 0.

corresponding state for easy reference:

i2 - AI, $ = -523 - A2, ql = -531 - 532 - A3, ql = ~Mi 23 - A2, and ql = -0.3 - s3i - s32 - A3. Note that
23, and —s3i — s32 are the diagonal elements of the infinitesimal
generator S of the environmental Markov process {E(t),t > 0}.


www.pdfgrip.com

Chapter 1. Examples

8

The important feature here is that the matrix Q is block-tridiagonal.
For general values of the size ra and the generator S of {E(t), t > 0},
the generator Q of the M/M/1 queue in a random environment is given
by

where S, A, and M are matrices of order ra (the number of environmental states), S is the generator of the environmental process, A and
M are diagonal matrices with A« = A,, and MH = fa for 1 < i < m.
Notice how similar the two structures (1.1) and (1.2) are: the matrix
(1.2) is block-tridiagonal and the blocks on the upper diagonal are all
equal, as are the blocks on the lower diagonal and the blocks on the
main diagonal, with the exception of the block in the upper left corner.
This is the characteristic of the Markov processes which we shall
study in later chapters.

1.3

Phase-Type Queues

These queues constitute a generalization of the M/M/1 queue, where
the service time and interarrival time distributions are not restricted
to the exponential; instead, they are assumed to be of phase-type
(Neuts [79, Chapter 3]). Phase-type distributions (abbreviated as PH
distributions) will be defined with precision in Chapter 2; we now give
an informal description only and two specific examples.
Assume that services are comprised of a number of operations, numbered 1 to m. Each operation i has a random duration, exponentially
distributed, with parameter ^; when that operation is completed, the
service proceeds to another operation j with probability pij, until eventually the whole service is completed. Thus, two services may differ by
the operations which are performed, as well as by the durations of the

operations. The total service duration, from start to finish, is said to
be of phase-type] the individual operations are called phases.


www.pdfgrip.com

1.3. Phase-Type Queues

9

Figure 1.4: Diagrammatic representation of an M/PH/1
queue. The arrival rate is A. The service time distribution has
two phases: the first phase has parameter /x; the second phase
has parameter pf and is performed with probability q.

The M/PH/1 Queue
In our first example, we assume that there are two service phases. The
first is exponentially distributed with parameter /z; when it is completed, the work performed is inspected. With probability p, the work
is found to be satisfactory and the customer departs. With probability
q = I — p, the customer needs additional work, which is exponentially
distributed with parameter //; at the end of this second operation, the
customer automatically departs. If we further assume that customers
arrive according to a Poisson process with rate A, then we have a particular example of the M/PH/1 queue, which may be described by the
diagram of Figure 1.4.
The diagram is to be read as follows: at any given time, there may
be at most one customer in the dashed box. When a customer's service
begins, the customer enters node 1; upon leaving node 1, he leaves the
system with probability p, and with probability q he enters node 2,
from which he leaves the system. The times spent in the nodes 1 and
2 are exponentially distributed with respective parameters p, and //.

With this description, the queue may be represented as a continuous
time Markov process on the state space ^(0) U £(1) U 1(2) U ..., where
^(0) = {0} and t(ri) = {(n, 1), (n, 2)} for n > 1; state 0 corresponds to
an empty system, and for n > 1, the state (n, j) signifies that there are
n customers in the queue, n — I of whom are in the waiting room, and
the customer in service is in the node j.


www.pdfgrip.com

Chapter 1. Examples

10

Figure 1.5: State transition diagram for an M/PH/1 queue.
The arrival rate is A. The service time distribution has two
phases: the first phase has parameter /^; the second phase has
parameter // and is performed with probability q.

Changes of state occur when there is a new arrival, when a service
is completed, or when a customer in service at node 1 moves to node
2. These transitions are given in the table below. The state transition
diagram is shown in Figure 1.5.
Prom
0
(1,1)
(n,l)
(n,l)
(1,2)
(n,2)

(nJ)

To
Rate
A
(1,1)
0
p,p
(n,2)
HQ for n > 1
(n - 1, 1) HP for n > 2
0
I*
for n > 2
(n - 1, 1)
//
A
f o r n > 1, .7 = 1,2
(n + l,j)

A few words of comment are in order here. The Markov process leaves
the state 0 with instantaneous rate A when a new customer arrives;
this new customer immediately begins its service in node 1. When the
Markov process is in the state (n, 2) for some n > 1, an end of service
occurs at the instantaneous rate ^', and if n > 2, a new customer begins
its service in node 1. When the Markov process is in the state (n, 1) for
some n > 1, the customer being served enters node 2 with probability
q\ with probability p it leaves the system and a new service may begin.
If we order the states by level (that is, by the number of customers)
and, within a level, by the label of the service node, we find that the