Advanced Computer Architecture - Lecture 42: Networks and clusters

CS 704
Advanced Computer Architecture

Lecture 42
Networks and Clusters
(Networks Topology and Internetworking .. Cont’d)

Prof. Dr. M. Ashraf Chughtai


Today’s Topics
Recap:

Switch Topologies .. Cont’d
 Centralized Switch Topology
 Distributed Switch Topology

Cluster
Summary

MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

2


Recap: Lecture 41
Last time we discussed:
 The formation of generic interconnection

networks and their categorization;
 The networks communication model,
performance, media, software, protocols,
subnet and networks topologies
Here, we noticed that a generic interconnection
network comprises:
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

3


Recap: Lecture 41
–
–
–
–

Computer nodes (host or end system)
H/W and S/W interface
Links to the interconnection network and
Communication subnet
The interconnections are classified based
on the number of processors or nodes
and the distance between them as:

– Local Area Network-LAN
– Wide Area Network-WAN

– System Area Network-SAN
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

4


Recap: Lecture 41
The interconnect communication model
shows that two machines are connected via
two unidirectional wires with a FIFO (queue) at
the end to hold the data
The communication software separates the
header and trailer from the message and
identifies the request, reply, their
acknowledgments and error checking codes
The communication protocols suggest the
sequence of steps to reliable communication
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

5


Recap: Lecture 41
The network performance that defines the

latency of the message as the sum of the:
Sender overhead, time to flight, receiver
overhead and the ratio of the message size to
the bandwidth
We also discussed the properties and
performance of interconnect network media or
link – the unshielded twisted pair (UTP), coaxial
cable and fiber optics
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

6


Recap: Lecture 41
At the end we discussed the formation of busbased and switch-based communication
subnets and introduced the network
topologies
Here, we observed that the bus-based LAN or
Ethernet is the simplest way to interconnect
more than two computers sharing a single
media
However, the interconnect sharing media are
challenging as it requires coordination and …
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

7


Recap: Lecture 41
… arbitration when more than one computer
needs the same media simultaneously
Alternative to sharing media is to use a switch
to provide a dedicated line to all destinations
in order; and facilitates point-to-point
communication much faster than the shared
media
A switch provides unidirectional interconnection of input to any one of multiple
output terminals
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

8


Recap: Lecture 41
The switch that facilitates unidirectional interconnection of every processor to all the
processors in the network is referred to as the
non-blocking switch
The Crossbar switch is typical example of nonblocking switch; an is employed in the centralized
switching topology
Last time we discussed the crossbar topology in
detail and noticed that a crossbar uses n2

switches to interconnect n processors in a
network
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

9


Recap: Lecture 41
Here the routing, to establish interconnection
between two node at a time, depends on the
addressing style
i.e., source-based routing where message
specifies the path to the destination or
destination-based routing where the message
simply contains the destination address and a
program running in the switch selects the port
to take for a given destination
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

10


Multistage Interconnect Network
Today, continuing our discussion on the

centralized switching topologies, we will
discuss an intermediate class of network
interconnect which lies between crossbar and
bus-based networks
This interconnect topology is referred to as the
Multistage network topology
A centralized multistage network, shown here,
is built from number of large switch boxes,
placed at multiple stages to interconnect all of
the nodes
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

11


Multistage Interconnection Topology

Each stage contains number of
small crossbar switches and
allows the straight or cross
connections through the
switch, as shown
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

12


Multistage Interconnection Topology
The number of stages are related to the
number of nodes and the size of the crossbar
switch
Consequently, its performance and cost are
more scalable than bus-based networks
The number of identical stages (Ns) in the
network having n nodes and switches of size m
x m, in each stage, is given as:

Ns = log m n
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

13


Multistage Interconnection Topology
And, the number of switches per stage is n/m
Thus, the total number of switches used in
multistage network of n nodes is n/m log m n

i.e., its cost is
O(n log n) as compared O(n2 ) for crossbar
To understand the design and working of

multistage networks, let us consider Omega
Network, depicted here, as a typical
implementation of multistage network
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

14


Omega Topology
Here, 8 nodes
(processors), are
addressed using 3bit code and 3
stages of 2x2
crossbar switches

000
001

000
001

010
011

010
011

100
101

100
101

110
111

110
111

number of identical stages [log2 8] = 3
And, switches per stage [n/m] = 8/2 =4
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

15


Omega Topology: Multistage Interconnect
000
001

S2

S1

S0

000
001

010
011

010
011

100
101

100
101

110
111

110
111

 let us see how the switches at each stage
operate to establish connection
 Note that for the 8-nodes Omega Network the
node address is of 3 bits, which is equal to
number of stages of the switch
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

16


Omega Network: Example
Here, the 3-bit code a2a1a0 represents 3 stages
of the network, as stage S2S1S0, from left to
right
To find the connection pattern XOR the source
and destination, e.g.,
 Src (010)  dest (110) then XOR results
 100  Cross (S2) Straight (S1) Straight (S0)
The switch connections are shown Green
Circles
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

17


Omega Network: Example
Thus, the generalized rule to find the switch
connection can be summarized as
 For the stage i
IF

the source and destination differ in ith bit
THEN

ELSE

connection Cross the switch in
the ith stage”
Connection is Straight in the ith stage”

MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

18


Characteristics of Omega
There exist an single path from source to
destination, thus contrary to the non-blocking
crossbar network, the omega network is
blocking network
This is shown here as:
- the path 010  110 (red) and
- the path 110  100 (blue)
have blockage as the S2 for 110 has to wait till
010 has passed otherwise it results in
collision

MAC/VU-Advanced

Computer Architecture

Lecture 42 Networks and Clusters (2)

19


Omega Network Characteristics
000
001

S2

S1

S0

…. Cont’d
000
001

010
011

010
011

100
101

100
101

110
111

110
111

However, in order to minimize collisions and to
improve fault tolerance to achieve high
reliability and dependability extra pathways
can be added
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

20


S1

Butterfly Network
Alternative to the
Omega Topology
of multistage
switching, is
Butterfly Network
shown here

000
001

S2

S2 S1

S0

000
001

010
011

010
011

100
101

100
101

110
111

110
111

Here, irrespective of the source address, for
the destination a2a1a0, the ith stage switch
sends to:
Upper port if ai = 0 and to
Lower port if ai = 1

MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

21


Distributed Switch Networks
So far we have been discussing the Centralized
switching topologies
The distributed switching network is one where
the switches are distributed throughout the
network and they allow interconnection of one
node to:
 either all the nodes
 or to a limited number of nodes

A
C
B

MAC/VU-Advanced

Computer Architecture

D
Lecture 42 Networks and Clusters (2)

22


Distributed Switch Networks
A network where each
node interconnects all
nodes of the network is
called, Fully connected
network

.. Cont’d

B
C

A

D

There exist different interconnects for
distributed switch networks
Before discussing these interconnects, let us
understand the parameters of interconnect
performance measure
MAC/VU-Advanced

Computer Architecture

Lecture 42 Networks and Clusters (2)

23


Interconnect Performance Measure
Criteria
Latency: Number of Links and must be small
Bandwidth: The number of messages or the
length of massages; it should be large
Node Degree: Number of links connected to a
node
Diameter: Maximum distance between any two
processors, i.e., the number of nodes between
source and destination; this is in deed the
measure of maximum latency
MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

24


Performance Measure Criteria

… Cont’d

Bisect: The imaginary line that divides the
interconnect into roughly two equal parts, each
having half the nodes
Bisection Bandwidth: Sum of the bandwidth of
lines crossing the imaginary bisection line
It measures the volume of communication
allowed between any two halves of network
with equal number of nodes

MAC/VU-Advanced
Computer Architecture

Lecture 42 Networks and Clusters (2)

25