Slides kiến trúc máy tính nhóm 7

Multiprocessors
Nhóm 7
1.Đỗ Luật Khoa
2.Lương Quang Tùng
3.Trần Thanh Phương
4.Phan Thanh Duy
5.Đặng Thanh Hùng
6.Thái Tiểu Minh
7.Nguyễn Thị Thanh Xuân


Multiprocessor system


What is multiprocessor system?
 A multiprocessor is a tightly coupled
computer system having two or more
processing units (Multiple Processors) each
sharing main memory and peripherals, in
order to simultaneously process programs
(theo wikipedia).

 Why multiprocessor systems?
 To get high performance and to reduce energy
consumption


Flynn Classification ?
 Flynn [1966] proposed a simple model of categorizing all computers
that is still useful today
 He looked at the parallelism in the instruction and data streams

May 20, 1934 (age )
New York City


What category can it be in the Flynn
Classification?
 The four classifications defined by Flynn are
based upon the number of concurrent
instruction (or control) and data streams
available in the architecture:


Category…
 Single instruction stream, single data stream (SISD)—
This category is the uniprocessor.
 Single instruction stream, multiple data streams (SIMD)—
The same instruction is executed by multiple processors
using different data streams.
 Multiple instruction streams, single data stream (MISD)—
No commercial multiprocessor of this type has been built
to date
 Multiple instruction streams, multiple data streams
(MIMD)—Each processor fetches its own instructions and
operates on its own data


Category…
 Because the MIMD model can exploit
thread-level parallelism, it is the

architecture of choice for generalpurpose multiprocessors.
 MIMDs offer flexibility.
 MIMDs can build on the cost-performance
advantages of off-the-shelf processors.


MIMD Model
 Each processor is executing its own instruction stream
 In many cases, each processor executes a different
process.
 A process is a segment of code that may be run
independently contains all the information necessary to
execute that program on a processor.
 Thread  multiple processors executing a single
program and sharing the code and most of their address
space.
 MIMD multiprocessor with n processors, we must usually
have at least n threads or processes to execute.
 Thread-level parallelism is identified at a high level by the
software system  threads consist of hundreds to
millions of instructions that may be executed in parallel


MIMD Model
 MIMD multiprocessors:
 Have 2 classes, depending on the number of processors
involved.
 which in turn dictates a memory organization and interconnect
strategy

 centralized shared-memory architectures
 most a few dozen processor chips (and less than 100 cores) in
2006
 For multiprocessors with small processor counts
 By using multiple point-to-point connections, or a switch, and
adding additional memory banks
 Because there is a single main memory that has a symmetric
relationship to all processors and a uniform access time from
any processor  called symmetric (shared-memory)
multiprocessors (SMPs) or uniform memory access (UMA)


•

This type of symmetric shared-memory architecture is
currently by far the most popular organization


MIMD Model
 To support larger processor counts, memory
must be distributed among the processors
rather than centralized.
 Distributing the memory among the nodes
has two major benefits:
 First, it is a cost-effective way to scale the memory
bandwidth.
 Second, it reduces the latency for accesses to the
local memory

• The second group consists of multiprocessors with
physically distributed memory.


MIMD Model
 The key disadvantages for a
distributedmemory architecture are that:
 communicating data between processors
becomes somewhat more complex
 it requires more effort in the software to take
advantage of the increased memory bandwidth
afforded by distributed memories


Memory consistency model


What is a memory consistency model?
 A memory consistency model, or memory model, for a
shared-memory multiprocessor specifies how memory
behaves with respect to read and write operations
from multiple processors.
 A memory consistency model is a set of rules which
specify when a written value by one processor can be
read by another processor .
 The memory consistency model affects:
 System implementation : Hardware , OS , languages ,
compilers
 Programming correctness
 Performance

 There are several different consistency models.


Sequential Consistency
 Lamport’s definition: A multiprocessor system
is sequentially consistent if the result of any
execution is the same as if the operations of
all the processors were executed in some
sequential order, and the operations of each
individual processor appear in this sequence
in the order specified by its program.
 There are two aspects to sequential consistency:
 (1) Maintaining program order among operations from
individual processors.
 (2) Maintaining a single sequential order among
operations from all processors


Sequential Consistency

Conceptual representation of sequential consistency with several processors sharing
a common logical memory


Sample programs to illustrate sequential
consistency

P1

P2

P1

P2

P1

P2

P3

a1: A=1;
b1: B=1;

a2: u=B;
b2: v=A;

a1: A=1;
b1: u=B;

a2:B=1;
b2:v=A;

a1: A=1;

a2:u=A;
b2: B=1;

a3:v=B;

b3:w=A;

(a) result (u,v)=(1,1).

(b) result (u,v)=(0,0)

(c) result (u,v,w)=(1,1,0)

P1

P2

P3

P4

a1: A=1;

a2:u=A;
b2: v=B;

a3:B=1;

a4:w=B;
b4:x=A;

(d) result (u,v,w,x)=(1,0,1,0)


Relaxed Consistency Models: The Basics

 The basic idea behind relaxed memory models is to enable the use of more optimizations by eliminating some of
the constraints that sequential consistency places on the overlap and reordering of memory operations.

Sequential consistency
Sequential consistency requires the
illusion of program order and atomicity
to be maintained for all operations

Relaxed consistency model
Relaxed models typically allow certain
memory operations to execute out of
program order or non-atomically

Several of the relaxed memory consistency models:
• Relaxing the Write to Read Program Order(total store ordering or
• processor consistency).
• Relaxing the Write to Write Program Order(partial store order).
• Relaxing the Read to Read and Read to Write Program Order(weak ordering,
the PowerPC consistency model, and release consistency)


Relaxing the Write to Read Program Order

SPARC V8 Total Store Ordering (TSO) [SFC91,SUN91]


Relaxing the Write to Read Program Order

the outcome (u,v,w,x)=(1,1,0,0)

the outcome (u,v,w,x)=(1,2,0,0)


Relaxing the Write to Read Program Order


Conditions for Processor Consistency (PC)
Memory sub-operations must execute in a sequential order that satisfies the
following conditions:
(a) sub-operations appear in this sequence in the order specified by the
program order requirement of Figure ,
(b) the order among sub-operations satisfies the coherence requirement,
and
(c) a read sub-operation issued by R(i) returns the value of either the last
write sub-operation W(i) to the same
location that appears before the read in this sequence or the last write suboperation to the location that is before the read in program order, whichever
occurs later in the execution sequence
P1

P2

P3

P1

P2

P3

P4

a1: A=1;

a2:u=A;
b2: B=1;

a3:v=B;
b3:w=A;

a1: A=1;

a2:u=A;
b2: v=B;

a3:B=1;

a4:w=B;
b4:x=A;

(c) the outcome (u,v,w)=(1,1,0)

(d) the outcome (u,v,w,x)=(1,0,1,0)


Relaxing the Write to Write Program Order


Partial store ordering (PSO) model example

PSO allows writes to different locations to complete out

of program order, it also allows the outcomes (u,v)=(0,0)
or (0,1) or (1,0).