Advanced Computer Architecture - Lecture 30: Memory hierarchy design

CS 704
Advanced Computer Architecture

Lecture 30
Memory Hierarchy Design
Cache Performance Enhancement

(Reducing Miss Rate)

Prof. Dr. M. Ashraf Chughtai


Today’s Topics
Recap: Reducing Miss Penalty
Classification of Cache Misses
Reducing Cache Miss Rate
Summary

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

2


Recap: Improving Cache Performance
─
─
─

The miss penalty
The miss rate
The miss Penalty or miss rate via
Parallelism
─ The time to hit in the cache

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

3


Recap: Reducing Miss Penalty
– Multilevel Caches
– Critical Word first and Early Restart
– Priority to Read Misses Over writes
– Merging Write Buffers
– Victim Caches

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

4


Recap: Reducing Miss Penalty

‘Multi level caches’
‘The more the merrier

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

5


Recap: Reducing Miss Penalty
“ Critical Word First and Early Restart’
intolerance
Reduces miss-penalty

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

6


Recap: Reducing Miss Penalty
‘Priority to read miss over the write
miss’
Favoritism

MAC/VU-Advanced

Computer Architecture

Lecture 30 Memory Hierarchy (6)

7


Recap: Reducing Miss Penalty
‘Merging write-buffer,’
Acquaintance
Victim cache
Salvage

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

8


Recap: Reducing Miss Penalty
Reduces miss penalty
Multi level caches
Reduces miss rate
Cache-misses
Methods to reduce the miss rate

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

9


Cache Misses

– Compulsory Misses
(cold start or first reference misses)

– Capacity Misses
– Conflict Misses
(collision or interference misses)
MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

10


Cache Misses - Classification
Compulsory Miss
Block must be brought into the cache
Capacity Miss
Capacity misses occur in Fully Associative
cache

MAC/VU-Advanced

Computer Architecture

Lecture 30 Memory Hierarchy (6)

11


Cache Misses - Classification
Conflict Miss
Many blocks map to the same address or set

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

12


Absolute Miss Rate

Cache Misses - Classification

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

13

Percent Miss Rate

Cache Misses - Classification

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

14


Reducing Miss Rate
1.
2.
3.
4.
5.

Larger Block Size
Larger Caches
Higher Associativity
Way Prediction and Pseudo-associativity
Compiler Optimization

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

15


1: Larger

Block Size

Reduce the miss rate
Spatial locality
Larger block have maximum number of data
or instructions

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

16


1: Larger

Block Size

In small cache , larger blocks may increase

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

17


1: Larger

Block Size
Cache: 1K

7.25%

Cache: 4K

64K

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

256K

16K

18


1:

Larger Block Size

Assumption
80 clock cycles of over head
Delivers 16 bytes every 2 clock cycle
Hit time of 1 clock cycle

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

19


1:

Larger Block Size: Solution

Average memory access time
= Hit time + Miss Rate x Miss Penalty
4KB cache, the miss rate = 7.24%
Miss penalty =
80 +4 = 84 clocks

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

20


1:

Larger Block Size: Solution
Cache: 1K

16K

64K

256K

7.25%

Cache: 4K

, Average Memory Access Time =

1 +(7.25% x 84) = 7.082 Clock cycles
MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

21


1:

Larger Block Size: Solution
Copy table 5.18 pp 428

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

22


1:

Larger Block Size: Solution

– Latency and bandwidth of the lower level
memory
– High latency and high bandwidth

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

23


2: Large

Cache Size

Reduce Capacity misses
In 2001
2nd – level and 3rd-level caches
Drawback
Longer hit time
Higher cost (access time)

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

24


Associativity

Absolute Miss Rate

3: Higher

MAC/VU-Advanced
Computer Architecture

Lecture 30 Memory Hierarchy (6)

25