Advanced Computer Architecture - Lecture 25: Memory hierarchy design

CS 704
Advanced Computer Architecture

Lecture 25
Memory Hierarchy Design
(Storage Technologies Trends and Caching)

Prof. Dr. M. Ashraf Chughtai


Today’s Topics
Recap: Processor Performance
What is outside the processor
Storage Technologies
RAM and Enhanced DRAM
Disk Storage
Summary
MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

2


Recap: Processor Performance
Data path and control design of
processor
Hardware-based and software-based
techniques to expose the ILP

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

3


What is outside processor?
performance of a computer

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

4


What is Outside the Processor? … Cont’d
– what is outside the processor?
– how the outside systems influence the
performance of processors?
Whatever is outside the processor is
referred to as the I/O system
The I/O systems include:
– Memory System
– Buses and controllers
MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

5


Memory System: An introduction
Memory system design for high
performance computers
Design goal of memory system for high
performance computers is to present
the user with as much memory as is
available

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

6


Memory Hierarchy: An introduction
Fast memory is expensive and cheap
memory is slow, therefore a memory
hierarchy is organized into several
levels
cost as low as of the cheapest
memory and
– speed as fast as of the fastest

memory
–

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

7


Memory Hierarchy System
Processor
Control
Memory

Speed:
Size:
Cost:

Memory

Memory

Datapath

Memory
Memory

Slowest

Fastest

Biggest

Smallest

Lowest

Highest

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

8


MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

9


Principle of Memory hierarchy
“The principle of locality”,
where each type of the module is

located in the memory system?

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

10


Principle of Locality
Advantage of the principle of
locality
An average access speed that is
very close to the speed that is
offered by the fastest technology
and
– Cost that is very close to the
cheapest technology
MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

11


Storage Type in Memory System
The semiconductor memories such as
registers, Static RAM and Dynamic RAM fast

in access speed but are expensive
Used in small size and placed either inside
or closest to the processor
Secondary storage devices which offer huge
storage at lowest cost per bit are placed
farthest away from the processor
MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

12


Levels of the Memory Hierarchy
100s Bytes, <1 ns/
$.1-.01/bit

G Bytes, 1-10ms
1-0.1 1 cents
Infinite, sec
0.1 - .010 C
-6
MAC/VU-Advanced
Computer Architecture

prog./compiler
1-8 bytes

Cache

Blocks

M Bytes, 10ns100ns
$.001-.0001

faster

Registers
Instr. Operands

K Bytes, 1-10 ns
$.01-.001/bit

Upper Level

Staging
Transfer Unit

Capacity, Access Time, Cost

cache control
8-128 bytes

Main Memory
Pages

OS
512-4K bytes

Disk

Files
Tape
Lec. 25 – Memory Hierarchy Design (1)

user/operator
Mbytes

Larger

Lower Level
13


MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

14


MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

15


Memory Hierarchy Pyramid

Smaller,
faster,
and
costlier
(per byte)
storage
devices

Larger,
slower,
and
cheaper
(per byte)
storage
devices

L0:
registers

CPU registers hold words
retrieved from L1 cache.

L1: on-chip L1
cache (SRAM)
L2:

L3:

L4:

L5:

MAC/VU-Advanced
Computer Architecture

off-chip L2
cache (SRAM)
main memory
(DRAM)

L1 cache holds cache lines
retrieved from the L2 cache
memory.
L2 cache holds cache
lines retrieved from
main memory.

local secondary storage
(local disks)

remote secondary storage
(distributed file systems, Web servers)

Lec. 25 – Memory Hierarchy Design (1)

Main memory holds disk
blocks retrieved from local
disks.

Local disks hold

files retrieved
from disks on
remote network
servers.

16


MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

17


Storage Systems
Classification of memory systems based on
different attributes and their design
Attributes:
– Material – Semiconductor, magnetic, optical
– Accessing – Random, Sequential, Hybrid
– Store/Retrieve – ROM, RMM and RWM

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

18

Random-Access Memory (RAM)
Key features
– RAM is packaged as a chip.
– Basic storage unit is a cell (one bit per cell).
– Multiple RAM chips form a memory

Types of RAM
– Static RAM (SRAM)
– Dynamic RAM (DRAM)
MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

19


Static RAM: Basic Cell
6-Transistor SRAM Cell
0

0

bit

word
word
(row select)

1

1

bit

Write:
1. Drive bit lines (bit=1, bit=0)
2.. Select row
Read:
1. Pre-charge bit and bit to Vdd
2. Select row
3. Cell pulls one line low
4. Sense amp on column detects
difference between bit and bit
MAC/VU-Advanced
Computer Architecture

bit

bit
replaced with pullup
to save area

Lec. 25 – Memory Hierarchy Design (1)

20

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

21


MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

22


MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

23


SRAM Organization: 16-word x 4-bit
Din 3

Din 2

Din 1

Din 0
Precharge

Wr Driver &
­ Precharger+

SRAM
Cell

:

SRAM
Cell
SRAM
Cell

:

Wr Driver &
­ Precharger+
SRAM
Cell
SRAM
Cell

SRAM
Cell
SRAM
Cell

:

SRAM
Cell
­ Sense Amp+

SRAM
Cell
­ Sense Amp+

SRAM
Cell
­ Sense Amp+

Dout 3

Dout 2

Dout 1

MAC/VU-Advanced
Computer Architecture

Wr Driver &
­ Precharger+
Word 0

Word 1

:

Address Decoder

SRAM
Cell

Wr Driver &
­ Precharger+

WrEn

A0
A1
A2
A3

Word 15

SRAM
Cell
­ Sense Amp+ Q: Which is longer:
Dout 0

Lec. 25 – Memory Hierarchy Design (1)

word line or 
bit line?

24

MAC/VU-Advanced
Computer Architecture

Lec. 25 – Memory Hierarchy Design (1)

25