Advanced Computer Architecture - Lecture 40: Input/Output systems

CS 704
Advanced Computer Architecture

Lecture 40
Input Output Systems
(RAID and I/O System Design)

Prof. Dr. M. Ashraf Chughtai


Today’s Topics
Recap:

Redundant Array of Inexpensive
Disks
I/O Benchmarks
I/O System Design
Conclusion

MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

2


Recap: I/O device’s performance
Last time we compared the performance of disk
storage and flash memory
We noticed that flash is six times faster than

the disk for read and the disk is six time faster
than the flash for data write
– Then we discussed the trends in I/O inter-

connects as: the networks, channels and
backplanes
– The networks offer message-based narrow-

pathway for distributed processors over long
distance
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

3


Recap: I/O Interconnects
The backplanes offer memory-mapped wide
pathway for centralized processing over short
distance
The interconnects are implemented via buses
The buses are classified in two major
categories as the I/O bus and CPU-Memory bus
The channels are implemented using I/O buses
and backplanes using CPU-Memory buses
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

4


Recap: I/O buses
Then we discussed the bus transition protocols
which specify the sequence of events and
timing requirements in transferring information
as synchronous or asynchronous
communication
We also discussed bus arbitration protocols ―
the protocols to reserve the bus by a device
that wishes to communicates when multiple
devices need the bus access
Here, we noticed that the bus arbitration
schemes usually try to balance two factors:
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

5


Recap: I/O System
 Bus-priority: the device with highest priority
should be serviced first
 Fairness: every device that want to use the bus
is guaranteed to get the bus eventually

The three bus arbitration schemes are:
 Daisy Chain Arbitration
 Centralized Parallel Arbitration
 Distributed Arbitration

MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

6


Storage I/O Performance
Now having discussed the basic types of
storage devices and the ways to interconnect
them to the CPU, we are going to look into the
ways to evaluate the performance of storage
I/O systems
We know that if a storage device crashes then
prime objective of a storage device should be
to remember the original information to make
storage device reliable

The reliability of a system can be improved
by using the following four methods
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

7


Reliability Improvement
 Fault Avoidance – prevent fault occurrence by
construction
 Fault Tolerance – providing service complying
with the service specification
by redundancy
 Error Removal – minimizing the presence of
errors by verification
 Error Forecasting – to estimate the presence,
creation and consequence
of errors by evaluation
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

8


Reliability, availability and dependability
The performance of storage I/Os is measured
in terms of its reliability, availability and
dependability
These terminologies have been defined by
Laprie; in the paper entitled
‘Dependable Computing and Fault Tolerance:

Concepts and Terminology;
published in the Digest of papers of 15th
Annual Symposium on Fault Tolerant
Computing (1985)
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

9


Dependability
Laprie defined dependability as the quality of
delivered service such that reliance can
justifiably be placed on this service;
where the service delivered by a system is its
observed actual behavior and the system
failure occurs when actual behavior deviates
from the specified behavior
Note that a user perceives a system alternating
between two states of delivered service; these
states are:
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

10

Dependability
 Service Accomplishment – service is
delivered as specified and
 Service Interruption – delivered service is
different from the specified service
Quantifying the transitions between service
accomplishment and service interruption is
the measure of the dependability
The dependability is measured in terms of
the measure of:
 module reliability, which is the measure
of the continuous service
accomplishment;

MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

11


Measuring Reliability
 and, module availability, which is the
measure of the swinging between the
accomplishment and interruption states
of delivered service
Now before we discuss the reliable and
dependable designs of the storage I/O let us

understand the terminologies used to measure
reliability, availability and dependability
The reliability of a module is the measure of
the time to failure from a reference initial
instant
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

12


Measuring Reliability … Cont’d
In other words we can say the Mean Time To
Failure (MTTF) of a storage module, a disk, is
the measure of reliability; and
The reciprocal of the MTTF is the rate of
failure; and
the service interruption is measured as the
Mean Time To Repair (MTTR)
Now let us understand, with the help of an
example, how can we use these terminologies
to measure the availability of a disk subsystem
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

13

Measuring Reliability: Example
Consider a disk subsystem comprising the
following component






10 disks
1SCSI controller
1 SCSI cable
1 power supply
1 fan
For the given values of MTTF of each
component; find the system failure rate and
hence the system MTTF
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

14


Reliability Example … Cont’d
10 disks, each with MTTF
1SCSI controller with MTTF

1 SCSI cable with MTTF
1 power supply with MTTF
1 fan with MTTF

= 1,000,000 Hrs
= 500,000 Hrs
= 1,000,000 Hrs
= 200,000 Hrs
= 200,000 Hrs

Solution:
System Failure Rate = 10 (1/1,000,000) +1/500,000 +
1/1,000,000 +
1/200,000 +1/200,000
= 23 /1,000,000 Hrs
System MTTF = 1/Failure Rate = 1,000,000/23
MAC/VU-Advanced
Computer Architecture

years

= 43,500 Hrs = 5
Lecture 40 Input / Output System (3)

15


Availability
The availability of a module is the measure of
the service accomplishment with respect to

the swinging between the two states of
accomplishment and interruption
The module availability, therefore can be
quantified as the ratio of the MTTF and Mean
Time Between Failure – MTBF (which is equal
to the sum of MTTF and MTTR); i.e.,
Availability = MTTF / (MTTF +MTTR)
= MTTF / MTBF
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

16


Network Attached Storages and Reliability
Last time we discussed the disk storages and
their interface with the processor using
channel and backplane interconnects; and
talked about the impact of disk storages and
interconnects on the overall performance of
the complete computing system
Today we will discuss the network
interconnects to interface multiple processers
located at a long distance and need high
performance storage service
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

17


Network Attached Storages and Reliability
A network provides well defined physical
and logical interfaces; i.e., interconnect
separate CPU and storage system
The networks are capable of sustaining
high bandwidth transfer and their file-server
Operating system supports remote file
access
Hence, the network attached storages are
more vulnerable to the reliability and their
dependability is very high
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

18


Network Attached Storage
Decreasing Disk Diameters
14" » 10" » 8" » 5.25" » 3.5" » 2.5" » 1.8" » 1.3" » . . .
high bandwidth disk systems based on arrays of disks

Network provides

well defined physical
and logical interfaces:
separate CPU and
storage system!

High
HighPerformance
Performance
Storage
StorageService
Service
on
onaaHigh
HighSpeed
Speed
Network
Network

Network File Services
OS structures
supporting remote
file access

3 Mb/s » 10Mb/s » 50 Mb/s » 100 Mb/s » 1 Gb/s » 10 Gb/s
networks capable of sustaining high bandwidth transfers
Increasing Network Bandwidth
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

19


Network Attached Storages and Reliability
So to improve both the availability and
performance of storage system, disk arrays
are introduced, which contain many low cost
disks
The throughput of disk arrays is improved by
having high bandwidth disk system which
employ many small disk drives; and
The throughput of a disk array is increased by
having many small arms on small (3.00” – 1.8”)
disk drives rather than one long arm on a
larger disk (14” – 24”); and
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

20


Manufacturing Advantages of Disk Arrays
Disk Product Families
Conventional: 4 disk designs

3.5”

5.25”

10”

14”
High End

Low End

Disk Array: 1 disk design
3.5”
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

21


Replace Small # of Large Disks with Large #
of Small Disks! (1988 Disks)
IBM 3390 (K)

IBM 3.5" 0061

x70

20 GBytes

320 MBytes

23 GBytes

97 cu. ft.

0.1 cu. ft.

11 cu. ft.

3 KW

11 W

1 KW

15 MB/s

1.5 MB/s

120 MB/s

I/O Rate

600 I/Os/s

55 I/Os/s

3900 IOs/s

MTTF

250 KHrs

50 KHrs

??? Hrs

Cost

$250K

$2K

$150K

Data Capacity
Volume
Power
Data Rate

large data and I/O rates
Disk Arrays have potential for

high MB per cu. ft., high MB per KW
reliability?

MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

22


Network Attached Storages and Reliability
Simply spreading the data over many disk
forces access to may several disks and hence
improve the throughput

The drawback to an array with more devices is
that dependability and hence the reliability
decreases – generally N devices have 1/N
reliability
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

23


Array Reliability: Example
Reliability of N disks = Reliability of 1 Disk ÷ N
Disk system MTTF

= 50,000 Hours ÷ 70

disks
= 700 hours
Drops from 6 years to 1 month!

However, the dependability can be improved by
adding redundant disks to the array to tolerate
faults
Arrays (without redundancy) too unreliable to be
useful!
MAC/VU-Advanced
Computer Architecture

Lecture 40 Input / Output System (3)

24


Subsystem Organization
host

host
adapter

single board
disk
controller

array
controller

manages interface
to host, DMA

single board

disk
controller

control, buffering,
parity logic

single board
disk
controller

physical device
control

single board
disk
controller

striping software off-loaded from
host to array controller
no applications modifications
no reduction of host performance

MAC/VU-Advanced
Computer Architecture

often piggy-backed
in small format devices

Lecture 40 Input / Output System (3)

25