Thực hiện chất lượng dịch vụ trong các mạng IP (P4)

4
Traffic Engineering for
Multi-service IP Networks
Traffic engineering is a set of techniques and tools that can be
used to optimize the performance of an operational IP network.
An overarching concept, traffic engineering, broadly interpreted,
includes other topics discussed in this chapter: routing control
mechanisms such as MPLS, and an advanced configuration in the
form of policy-based management. These techniques can also be
used without deploying the complete traffic engineering frame-
work, but their optimal use benefits from it. For example, we shall
see that most efficient use of routing control is possible with proper
measurement technologies in the network.
Traffic engineering is also a cornerstone for the management of
a multi-service IP network in a cost-efficient manner. Supporting
services in the network in a commercial environment requires that
the status of the network is monitored, optimized and controlled.
The traffic engineering framework uses service level management
and monitoring techniques explained later in Chapters 6 and 7.
Especially in the presence of growing traffic volumes, the effec-
tiveness of traffic engineering also has implications on network
planning and – when used – admission control decisions, which
is why it can be said that traffic engineering also lays the foun-
dation for the application of techniques in Chapter 5 and Chap-
ter 8. Finally, traffic engineering typically makes use of the service
Implementing Service Quality in IP Networks Vilho R
¨
ais
¨
anen
 2003 John Wiley & Sons, Ltd ISBN: 0-470-84793-X

98 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS
quality support mechanisms described in Chapter 3. The relation
to end user service quality discussed in Chapter 2 is that ideally
traffic engineering is a set of network-side techniques providing
engineered quality support level for individual services, as the
name of the concept implies.
4.1 TRAFFIC ENGINEERING
The traffic engineering (TE) overview [RFC3272] of IETF’s Traffic
Engineering working group (tewg) is a good description of the
typical traffic engineering tasks, even though true multi-service
support is only finding its way to the Internet. IETF’s traffic engi-
neering overview is used as a frame of reference for the rest of the
topics in this chapter.
The TE overview draft defines traffic engineering as the part
of network engineering concerned with performance evaluation
and performance optimization issues. Further, traffic engineering
is defined [RFC3272] to include
the application of technology and scientific principles to
the measurement, characterization, modelling, and control of
Internet traffic.
An important area is the enhancement of performance of oper-
ational networks, both in terms of resource usage and from the
viewpoint of traffic requirements. The obvious goal is to imple-
ment the service quality support with as small an amount of
valuable network resources as possible. Examples of the resources
involved include leased link bandwidth, CPU usage, and buffer
space. Routing control is singled out as a means of controlling
resource use over-multiple hops. Compared with traditional IP
routing protocol-based methods, the target of traffic engineering-
oriented routing control can be network-wide traffic distribution.

Traffic requirements, manifested to the end user in the form of
“emergent properties” of network performance, are to be opti-
mized subject to the constraints of the economic realm. Emergent
properties presumably correspond roughly to the end user quality
experienced discussed in Chapter 2. Further, traffic engineering is
intended to help in identifying and structuring goals and priori-
ties in enhancing the end user service experience. Reformulated,
4.1 TRAFFIC ENGINEERING 99
traffic engineering is a systematic means of analysing the network
state, drawing conclusions from it, and potentially performing a
network configuration. As is often the case with complex real-
time systems, creation of proper procedures to cope with data
acquisition and control procedures is at least half the work. Traffic
engineering is perceived as important already in the present-day
best-effort Internet, and it is even more so in the multi-service
Internet of tomorrow that is in the making at the moment.
Some reasons leading to the need for parameter optimiza-
tion are:
• change in local traffic volume, such as addition of a new major
customer in a PoP;
• change in the resourcing situation, such as change in provider
for leased line capacity;
• change in traffic aggregate distribution (locally or globally), for
example, due to adoption of a new service type;
• exceptional conditions such as malfunctioning of a router.
Other focuses areas of traffic engineering are improvement and
maintenance of reliable network operations in the face of excep-
tional situations, for example, resulting from equipment failure.
This requires the ability to detect network fault conditions and
the ability to act based on this information. In the case of a multi-

service network, fault conditions pertain to a service quality sup-
port mechanism in the network. Related definitions and proce-
dures for the former will be discussed in Chapter 7. Fallback to
back-up route for a traffic aggregate in case of detected link fail-
ure is an example of an action taken as a result of detecting an
exceptional condition in a network.
In what follows, we shall take a look at the context of traffic engi-
neering, the traffic engineering process, acquiring information from
the network, analysing this information, and means of enhancing
performance in a DiffServ network. The discussion is loosely based
on a recent RFC on traffic engineering [RFC3272], but is amended
for application in a multi-service network as appropriate. A sum-
mary of service quality support aggregate level measurement meth-
ods has been added.
100 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS
4.1.1 Context of traffic engineering
The appropriate methodology used by traffic engineering can be
derived from the context of traffic engineering [RFC3272]. The
entire context of Internet traffic engineering is defined to include
the following four sub-contexts:
• Network context defining the different situations in which traffic
engineering needs to be applied. The network context includes
network structure, network policies, network characteristics, net-
work constraints, network quality attributes, and network opti-
mization criteria. In a multi-service IP network, the additional task
of creating and managing adequate set of service quality support
classes is needed.
• Problem context defines the concrete issues traffic engineering is
applied to. Problem context includes the identification, abstrac-
tion of relevant features, representation, formulation, specifi-

cation of solution requirements, and specification of desirable
features for the solutions. Congestion is named as an example
of a typical problem to be solved on the traditional Internet; in a
multi-service IP network, sub-optimal performance of a service
quality support class could be a further problem.
• Solution context defines how issues defined by problem con-
text are addressed. Solution context includes analysis, evalua-
tion of alternatives, prescription, and resolution. Examples of
solving the congestion problem at different timescales, reac-
tive/proactive as well as supply/demand alternatives are pro-
vided. In principle this phase is similar for best-effort Internet
and multi-service IP network, being only more complex in the
latter case.
• Implementation and operational context defines the methodological
application of solution context, including planning, organiza-
tion, and execution.
The network context must be able to express the essential factors of
traffic engineering in such a way that the problem context can be
applied to it. A multi-service network with heterogeneous service
quality support requirements can be represented as an aggregation
of network resources, traffic distribution offered, and the service
quality support mechanisms. Network resources include network
elements and the capacity and topology of the interconnecting
4.1 TRAFFIC ENGINEERING 101
network. The properties and configurability of network resources
are important. Service quality support mechanisms in a DiffServ
network include edge router functions, core router functions, and
optionally also routing control and admission control means. The
inter-operation of different service quality support mechanisms
is part of the network context, too. Specific to a DiffServ-based

multi-service network, the effectiveness of network utilization can
be taken to be part of the network context.
Problem context addresses abstraction, representation, and mea-
surement of network status in a manner that is relevant for traf-
fic engineering. The problem to be solved needs to be explicitly
defined, some kind of acceptance criteria must be defined for good
solutions, and requirements for solution context should be defined.
The requirements, in general, can relate to the following:
• service quality support aggregates;
• measurement methods;
• analysis methods;
• network reconfiguration means.
The different factors in the above list are often interdependent.
For example, unless there is an efficient means of reconfiguring
the network, there is less benefit in making detailed analyses of
the network state.
The network status measurement may encompass multiple levels
of abstraction, ranging from individual resources to Autonomous
Domain (AD)-wide representations. We shall return to this issue in
Chapters 6 and 7. The development of suitable performance opti-
mization means is an important issue. For a multi-service network
in general, service quality support aggregate characteristics such
as delays and packet loss measures are important. On the network
level, the utilization level of the network is vital.
Examples of solution context include specific policies, techniques
and methodologies, parameters, and guidelines that are imple-
mented to perform the task specified within the problem context.
The application of measurement tools and methodologies, as well
as routing control, are listed as typical important solution con-
text components.

Implementation and operative context are more administrative,
organizational, and executive in nature than the previous steps. This
102 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS
aspect of traffic engineering is not central to this book, but some
related aspects will pop up sporadically in the following chapters.
The authors of [RFC3272] further note that the application of
the traffic engineering context – especially with congestion – can
be anticipatory or corrective. In the former case, the choice of
good measurement, analysis, and performance optimization tools
is especially important.
4.1.2 The traffic engineering process
The traffic engineering process is defined [RFC3272] as consisting
of the following parts:
• Definition of relevant control policies. This can be considered as
being outside of the actual traffic engineering loop, controlling
its progress. Naturally control policies can be and are adjusted
based on observed network performance.
• Feedback mechanisms for acquiring measurement data from pro-
duction network.
• Analysis of network state and characterization of traffic workload.
• Performance optimization of the network.
Thetrafficengineering processisiterative, as illustratedin Figure 4.1.
For readers familiar with the philosophy of science, the picture is
reminiscent of the idealized depiction of the induction/deduction
sequence in science [Old86]. An example of application of traffic
engineering in a large-scale production network can be found in
[FGL + 00].
The first phase is the definition of the control policies. Inputs
used for devising these may include the required service quality
support of different traffic types – related to business models of

the network operator – and optimization algorithms. For example,
the network may have a busy hour configuration as well as an
off-peak hour configuration, which are different from each other.
Some service quality support aggregates can be defined as more
important than others when the triggering and execution of cor-
rective actions are being considered.
According to [RFC3272], feedback from the network – the sec-
ond stage in traffic engineering process – does not have to be
limited to data obtained from the network only, but can also
4.1 TRAFFIC ENGINEERING 103
Definition of
control
policies
OptimizeAnalyse
Feedback
from
network
Network
Measure
Configure
Figure 4.1 Traffic engineering process of [RFC3272]  International Tele-
communication Union
use synthetic workloads in case immediate measurement results
are not available. Synthetization of measurement results can be
based on extrapolation from earlier data, for instance. In the case
of a multi-service network, synthetization of performance data
is often more challenging than for a single-service network, but
this scheme may be necessary to better understand the status
of the network. We shall discuss measurement means briefly in
Section 4.1.3 and on a more general level in Chapter 7.

Analysis can be either reactive or proactive. In reactive mode,
analysis attempts to identify possible locations in the network hav-
ing sub-optimal performance, to trace the root cause, and may also
test different ways of solving the problem. In proactive mode,
the goal is to identify optimization targets to prevent future per-
formance problems. The proactive mode is often important in
understanding and anticipating the effect of traffic volume growth
and changing traffic mix to the configuration of a multi-service
IP network. Possible tools for performing analysis include mod-
elling, simulation, and traffic matrices. Traffic analysis does not
have to produce the actual solution, but can also be limited to a
descriptive mode.
The optimization phase includes the means of selecting the
actual method to be used for optimizing network performance.
In this phase, the network operator can greatly benefit from using
a proactive network performance analysis mode, if the analysis
means are good enough. Optimally, the analysis machinery
104 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS
can be used for benchmarking potential network configurations
in advance.
Automation of the above process is a challenge in the general
case. It requires finding the correct optimization tasks to be auto-
mated, and defining a correct interface for human supervision and
intervention. At the same time, instabilities due to control mecha-
nisms must be avoided.
4.1.3 Obtaining performance data from the
network and analysing it
Feedback from the network can be obtained from multiple sources
and on different levels of abstraction (see Figure 4.2). Starting
with the most detailed method, individual network elements can

be queried for information. Network elements may also provide
notification mechanisms for informing the network management
system of predefined conditions transpiring in the network. In
some cases, individual network elements may have information
about a larger part of the network than the network element itself.
An example of this is the routing table of a router running a Link
State type routing protocol such as OSPF.
For most characteristics, obtaining information on higher abstrac-
tion level than that of individual network elements is made possible
by a Network Management System (NMS). Forming of higher-level
characteristics can be achieved, for example, by applying thresh-
olding to information collected from individual network elements,
as will be discussed in Chapter 7. To give an example, with such
a method one can see the most heavily loaded routers in a net-
work domain.
The management system can typically provide averages and
tendency analyses in addition to an abstracted momentary net-
work status. For a multi-service network, the performance of the
network can be monitored on the level of traffic aggregates for
a network domain. Finally, the feedback can be in the form of
service quality. This requires a way of estimating service level
performance. The abstraction levels are compared in Table 4.1. The
subject of performance measurements will be covered more thor-
oughly in Chapter 7.
Typically, network elements such as routers provide a set
of counters accessible through a management interface, for
4.1 TRAFFIC ENGINEERING 105
AD
NE
NE

NE
Endpoint
AD
Endpoint
Aggregate performance
Service performance
Figure 4.2 The relations between performance measures
Note
: The management system provides information about performance
of individual network elements within an AD, whereas aggregate perfor-
mance and service performance relate to measurements spanning multiple
elements. An AD may have multiple aggregate performance measures, for
example, relating to different DSCPs
Table 4.1 A comparison of network monitoring levels
Monitoring level Typical measured characteristics
Per network element Overall load and per- traffic aggregate statistics.
IP management system Network-wide data, averages, trend analysis.
Aggregate performance Delay, jitter, packet loss, and available bandwidth in
anetworkdomain.
Service level Service Level Specification components (see
Chapter 6).
example, Simple Network Management Protocol (SNMP). The
contents of the accessible information are defined in Management
Information Bases (MIBs). Some routers may use vendor-specific,
non-standard interfaces.
Two types of measurements relevant to traffic engineering will
be discussed in this chapter for purposes of concreteness. Traffic
aggregate performance measurements (Section 4.1.3.1) data is rele-
vant for optimizing the parameters of a DiffServ network domain.
They can also be used as an input for routing control. Traffic load

measurements are typically used for routing control and capacity
expansion need evaluation purposes.
4.1.3.1 Traffic aggregate performance measurements
Aggregate level performance here means network performance
of a service quality support aggregate, such as the behavioural
aggregate of DiffServ, over multiple IP hops. Thus, they include
106 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS
the effect of multiple network elements as a part of the
measurement definition. In view of performance data relating
to a particular service quality support aggregate, aggregate level
performance measurement means may also require less processing
than performance estimation obtained by combining data from
individual network elements. Generally, three different types of
methods can be used for aggregate level performance assessment:
1. passive (non-intrusive) measurements;
2. active (intrusive) measurements;
3. piggybacking measurements.
The measurement data obtained by one or more of these mecha-
nisms can be processed in different ways to obtain relevant char-
acteristics. One example of such processed characteristics is packet
loss correlation modelling [Cla01, TIPHON-5] which can be used
to estimate the quality of traffic aggregate used for transporting
VoIP or multimedia streaming. More generally, ITU-T’s recom-
mendation [I.380] lists the following measurable quantities: packet
transfer delay, delay variation, packet error ratio, and packet loss
ratio. Availability is characterized by the PIA and PIU parameters
discussed in Chapter 2.
A generic issue related to measurements, proper measurement
methodology, including adequate sampling, is very important in
obtaining meaningful results [RFC2330, RFC3432]. Measurement

results must form a representative sample of network behaviour
in order for optimization techniques to be applicable. To this end,
the measurement point placement scenarios need to be defined
carefully [I.380]. Likewise, the treatment of error conditions such
as packets with bit errors need to be defined. Let us take a look
at the aggregate performance measurement methods next.
4.1.3.1.1 Passive measurements
Passive measurements do not require additional traffic to be injected
into the operational network. Instead, the normal traffic in an opera-
tional network can be monitored. One or more measurement points
canbeusedinasinglenetworkdomainpermonitoredtraffictype.
With one measurement point, traffic load and protocol statistics can
be monitored. In the case the monitored streams include timestamp-
ing and sequence numbering information, which is the case for the
RTP, also delay jitter and packet loss measurements can be made,
4.1 TRAFFIC ENGINEERING 107
but it must be noted that the results may include the effects of such
elements as TCP/IP stack implementation of the sending host oper-
ating system scheduling, and access network [RFC3432]. One way of
circumventing this is to combine passive and active measurements
by measuring a test stream such that the properties of the test stream
generator are well known.
With two measurement points, delay measurements are possi-
ble without timestamps, provided that a sufficiently high degree
of synchronization can be implemented between the measurement
points. Two-point delay variation measurements are described in
[I.380]. Controlled delay measurements with only a single mea-
surement point are in principle possible when a timestamped mea-
surement stream is sent by the endpoint. In addition to suffering
from the sending terminal-related problems listed above, it is also

required that the host clock of the sending terminal is correctly
synchronized with the measuring point, and that timestamp mark-
ing can be trusted. Passive measurements have been subject to
standardization at least in IETF [RFC1757, RFC2021, RFC3432] and
ETSI’s IP telephony project TIPHON [TIPHON-5].
Passive measurements do not add further traffic into the net-
work, but require more processing than active measurements, for
example. They may also require transfitting of large amounts of
data. Passive measurements in the network core require high-
performance probes. Privacy and possible effects of encryption
are important issues for passive measurements [CDF + 00].
4.1.3.1.2 Active measurements
Active measurements are based on controlled transmission test
traffic streams through the measured network. Such measurements
can be implemented relatively straightforwardly, and have thus
been used relatively early to assess the suitability of the Internet
for transporting real-time streams [BCG95]. The measurement
packets typically carry timestamps and sequence numbers to
accommodate latency and loss measurements. The obvious
downside of the method is the network capacity consumed
by measurement streams. Active measurements are typically
most useful in high-capacity parts of a network domain in
monitoring service quality levels there, but can also be used for
troubleshooting purposes [RR00].
At best, active measurements can provide for controlled mea-
surements, when the operating system and TCP/IP stack-related
108 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS
properties of the measuring hosts are well known. Active mea-
surements have been subject to standardization in IETF’s IP Per-
formance Metrics working group (IPPM WG), ITU-T [I380], and

ETSI [TIPHON-5]. To give an example of IETF work, the IPPM
working group has defined measurement metrics for delay mea-
surements using packets sent at Poisson-distributed intervals for
one-way [RFC2679] and round-trip [RFC2681] measurements. An
alternative approach suitable for measuring network performance
for periodic streams is described in [RFC3432]. The real differ-
ence between the Poisson sampling measurement and the periodic
stream measurement is only in sampling methodology.
With proper measurement packet format, delay, delay jitter,
packet loss and packet loss correlation performance for a traffic
aggregate can be directly measured with active measurements
[TIPHON-5]. Further, the IPPM working group is in the process
of defining a reordering metric. Furthermore, available bottleneck
bandwidth for the traffic aggregate in question can be estimated
using the packet pair method (see, for example, [LB99, HCY + 01]).
With active measurement, the type of measurement pack-
ets – population of interest – needs to be defined [I.380, RFC3432].
Spurious outcomes, or reception of packets matching the filter of
measurement packets which, however, are not part of the mea-
surement stream, need to be defined as well.
4.1.3.1.3 Piggybacking
Piggybacking is at the moment mostly a research-oriented
approach, where timestamping and sequence numbering infor-
mation are attached to traffic payload [JH00]. Overheads from the
measurement as well as required processing in the elements insert-
ing and removing measurement fields to payload data packets
need to be taken into account.
4.1.3.1.4 Comparison of aggregate performance measurement
methods
A comparison of aggregate performance measurement technolo-

gies is provided in Table 4.2.
In summary, active measurements generate more traffic in the
network, but require less processing than passive measurements.
Passive measurements require more processing, and are most suit-
able for access networks. Active measurements can be used for
service level monitoring. Active and passive measurements can
4.1 TRAFFIC ENGINEERING 109
Table 4.2 Comparison of aggregate measurement methods
Method Delay Delay variation Packet loss Throughput
Passive measurements (1) (2) (2) X
Active measurements X X X (3)
Piggybacking X (4) X (3)
Notes: (1) reliable measurement possible for two measurement points; (2) possible with a
single measurement point for streams with timestamps and sequence numbers, see text for
other factors; (3) indirectly possible, see text for details; (4) see text for details.
be combined by applying passive monitoring to the streams used
for active measurements.
4.1.3.1.5 Relation of aggregate measurement to end-to-end
service level performance
Service level measurements, in general, can be implemented in the
following ways:
• using endpoint feedback;
• estimating from aggregate measurements;
• estimating from per-network element data.
A communication endpoint may be able to give feedback relat-
ing to service quality. An example of this is the transmission of
RTCP messages for real-time streams between endpoints using
RTP, with RTCP messages yielding information about delay jitter
and packet loss. Another example of endpoint could be SDP rene-
gotiation of multi-speed codec bitrate in SIP environment. These

examples show that the endpoint feedback or use of per-element
data does not necessarily provide service level performance infor-
mation directly. Furthermore, the endpoint performance is affected
by other factors, including those of the endpoint itself. Subse-
quently, it may be difficult to map endpoint performance to opti-
mization actions in the network.
Aggregate performance measurement yield information that is
network-specific, yet relates to end-to-end service performance is
shown in Figure 4.2. According to ITU-T’s definition [I.350], ser-
vice performance is measured at service access points, whereas
network performance is measured at connection element bound-
aries. What this definition boils down to is that domain-internal
110 TRAFFIC ENGINEERING FOR MULTI-SERVICE IP NETWORKS
network performance measurements often have different scope
than service level measurements.
Drawing conclusions of performance optimization actions based
on aggregate performance results requires the link between end-to-
end service performance and aggregate performances to be known.
OnetoolforthisistheuseofQoSbudgets[TIPHON-3].Asim-
ple application of QoS budgets to providing the aforementioned
link can be found in [LRR01]. In general, the link between net-
work domain-level performance and end-to-end performance can
be considered to be defined within a SLA between network oper-
ator and the provider and/or the user of the service. QoS budgets
for different service quality characteristics can be considered to be
a part of the SLA, as will be discussed in Chapter 5.
4.1.3.2 Obtaining data relevant for routing control
The data most important for routing control are load levels on
individual links and traffic aggregate performance characteristics.
Obtaining accurate information about traffic load in IP networks

where routing is based purely on IP routing protocols is important
mainly because of two factors: the non-connection-oriented nature
of the Internet Protocol, and the distributed nature of IP routing
protocols. As a result of these two factors, traffic volumes on a
particular link in an AD may vary as a result of the interplay of
BGPs and IGPs [FGL + 00b]. Further potential reasons for traffic
volume variability include burstiness of traffic sources, the oper-
ation of load-sharing schemes, and routing fallback to resiliency
routes. Obtaining load information is also important as a basis for
performing MPLS-based traffic engineering.
Traffic load level on a link not only affects the mean queueing
delay, but may also increase the variability of queueing delays. The
properties of the flows transported in a particular traffic aggregate
as well as the implementation of scheduling in individual network
elements affect the functional form of the dependence between link
loading level, average queueing delay, and variability of queue-
ing. Queueing theory can be used to estimate delay and delay
variation effects in an individual router when the distributions for
arrival process (offered traffic) and service time (packet lengths)
are known. The distributions, in general, can depend according to
thetimeofdayoflongertemporalcycles.
In a multi-service network such as a DiffServ domain, the load
estimation methods used should be applicable to the estimation