Shared Memory Machine Architectures

Entire address space defined by the memories is accessible from any CPU, and each CPU has an associated cache

Dance Hall Architecture

CPUs on one side and memory on other side of an interconnection network

Symmetric Multiprocessor (SMP)

Interconnection network here is a system bus that connects all CPUs to talk to main memory
Symmetric as access time of any of the CPUs to memory is the same

Distributed Shared Memory (DSM)

A piece of memory is associated with each CPU which it can quickly access
At the same time, each CPU can have slower access to all other memories through the interconnection network

When using multiprocessors, a process may update a hot memory location. In this case, other processors should now reference this value. This can be done through enforcing a memory consistency model.

For memory consistency models, it should enforce only expected outcomes. If there is an outcome resulting from out-of-order updates, this should be disallowed

Shared Memory and Caches

Cache associated with CPU serves same purpose in multi-processor and uni-processor
CPU first looks for data in the cache (much faster ~2 cycles), then the main memory (slower, ~100 cycles)

Issues with Private Caches (Cache Coherence Problem)

Some memory location $y$ may be a hot memory location in all the private caches
If $P1$ updates $y$ ($y$ → $y'$), we need to ensure that all other CPUs see this new value
Hardware and software has to agree on the memory consistency model

Consistency Model Quiz

If initially $a=b=0$, and we have the following processes, where the instructions for each process execute top-down

| --- | --- | --- |

What are the possible values for $d$ and $c$?

$c=d=0$
$c=d=1$
$c=1,d=0$
$c=0,d=1$

<aside> 📌 SUMMARY: Shared memory architecture includes Dance Hall, SMP and DSM. When dealing with shared memory, we have to ensure cache coherence since multiple processes may be referring to the same memory location. For this, we have memory consistency models which are presented to the programmer, and cache coherence (NCC or CC) to enforce these models. Furthermore for CC architectures, we can have Write-Invalidate or Write-Update. Increasing the number of processors may not equate to proportional gain due to overheads from maintaining cache coherence as well. Having less memory sharing leads to less problems.

</aside>

Date: January 19, 2025

Topic: Shared Memory Machines

Recall

Notes