Introduction to memory hierarchy, types of memories, cache design, & secondary storage

1. Memory Organization
CS2052 Computer Architecture
Computer Science & Engineering
University of Moratuwa
Dilum Bandara
Dilum.Bandara@uom.lk

2. Outline
 Memory hierarchy
 Types of memory
 Cache memory
 Secondary/Permanent storage
2

3. Blocks of a Microprocessor
3
Literal
Address
Operation
Program
Memory
Instruction
Register
STACK Program Counter
Instruction
Decoder
Timing, Control and Register selection
Accumulator
RAM &
Data
Registers
ALU
IO
IO
FLAG &
Special
Function
Registers
Clock
Reset
Interrupts
Program Execution Section Register Processing Section
Set up
Set up
Modify
Address
Internal data bus
Source: Makis Malliris & Sabir Ghauri, UWE

4. Accessing Memory
4
Memory
Address Bus
Control Bus
RD/WR signals
Data Bus

5. Reading From & Writing to Memory
 Reading from memory
 Address of memory location to read is placed on
Address Bus
 Read Signal (RD) in control bus is activated
 Data is fetched (read) from Data Bus
 Writing to memory
 Address of memory location to write is placed on
Address Bus
 Data is placed on Data Bus
 Write Signal (WR) in control bus is activated
5

6. Connecting Memory & CPU
6
CPU
ROM
RAM 1
RAM 2
Memory
Controller

7. Traditional Memory Hierarchy
7
Secondary Storage
Main
Memory
Reg.S
p
e
e
d
C
o
s
t
S
i
z
e

8. Memory Hierarchy
 Memory has to be organized in such a way that
its slowness doesn’t reduce performance of
overall system
 Some memory types are fast but expensive
 Registers, Static RAM
 Some other types are cheap but slow
 Dynamic RAM
 Solution
 Hierarchical memory system
 Limited capacity of fast but expensive memory types
 Larger capacity of slow but cheap memory types 8

9. Memory Hierarchy (Cont.)
 Objective is to have a memory system
 with a sufficient speed
 with a sufficient capacity
 as cheap as possible
9

10. Processor Memory Gap
10
Gap grew 50%
per year
Source: Computer Architecture, A Quantitative Approach by John L. Hennessy and David A. Patterson

11. Extended Memory Hierarchy
11Source: http://www.ts.avnet.com/uk/products_and_solutions/storage/hierarchy.html

12. Modern Memory Hierarchy
12
Source: http://blog.teachbook.com.au/index.php/2012/02/memory-hierarchy/

13. Types of Memories
 ROM – Read Only Memory
 Permanent memory
 Can only be read, can’t be written to
 RWM – Read Write Memory
 Traditionally known as RAM (Random Access
Memory)
 Contents are erased when power is disconnected
 Permanent storage
 A.k.a secondary storage
 Hard disks, Solid State Drives (SSDs), CD-ROM, tape
drives
13

14. Types of ROMs
 ROM – Read Only Memory
 Contents are written at the time of manufacture
 Stores initial start-up programs
 Not economical to produce in small quantities
 PROM – Programmable Read Only Memories
 Same as ROMS, but contents can be written once,
using special equipment
 Used in embedded systems
14

15. Types of ROMs (Cont.)
 UVEPROM – UV Erasable PROM
 Same as PROM but contents can be erased by
shining an UV light on to the IC
 Require special equipment to program
 EEPROM – Electrical Erasable PROM
 Same as UVEPROM except that contents can be
erased by applying a special high voltage to some of
the signals
 Used in embedded systems, microcontrollers
 Can be (re)written several times
15

16. Types of ROMs (Cont.)
 FLASH memory
 Special type of EEPROM that can be erased or
programmed while in the circuit
 Once programmed contents remains unchanged –
even after a power failure
 Can be (re)written many times
 Can be written in blocks of memory
 Commonly used in modern PCs
 BIOS, USB memory stick (flash drive), memory cards
16

17. Types of RAMs
 Static RAM vs. Dynamic RAM
17
Transistors Capacitors
Developed using Silicon & other
semiconductors
Developed using Silicon & other
semiconductors
High-speed switching Slower performance
Will retain state forever
Automatically discharges after
sometime, need refreshing
Reliable Less reliable
Low density High density
High power consumption Low power consumption
High cost (per bit) Low cost (per bit)

18. Types of RAMs (Cont.)
 SRAM (Static RAM)
 More reliable but expensive
 Typically used for cache memories
 More expensive & consumes more power
 DRAM (Dynamic RAM)
 Contents are stored as charges in a small capacitor
 Capacitor must be re-charged from time to type
 Bulk of PC memory is made out of DRAM
18

19. Memory Modules
19
Source: Upgrading & Repairing a PC by Scott Mueller
SIMM
DIMM

20. DRAM Optimizations
20
Source: Computer Architecture, A Quantitative Approach by John L. Hennessy and David A. Patterson

21. Principle of Locality
 Programs tend to reuse data & instructions that
are close to each other or they have used
recently
 Temporal locality
 Recently referenced items are likely to be referenced
in the near future
 A block tend to be accessed again & again
 Spatial locality
 Items with nearby addresses tend to be referenced
close together in time
 Near by blocks tend to be accessed
21

22. Locality – Example
sum = 0;
for (i = 0; i < n; i++)
sum += a[i];
return sum;
 Data
 Access array elements in succession – Spatial locality
 Reference sum each iteration – Temporal locality
 Instructions
 Reference instructions in sequence – Spatial locality
 Cycle through loop repeatedly – Temporal locality
22
a[0] a[1] a[2] a[3] … …

23. Cache Memory
 Small amount of memory that is faster than RAM
 Slower than registers
 Built using SRAM
 Range from few KB to few MB
 Use by CPU to store frequently used instructions
& data
 Spatial & temporal locality
 Use multiple levels of cache
 L1 Cache – Very fast, usually within CPU itself
 L2 Cache – Slower than L1, but faster than RAM
 Today there’s even L3 Cache
23

24. Core i7 Die & Major Components
24
Source: Intel Inc.

25. L1 & L2 Cache
25
Source: www.edwardbosworth.com/CPSC2105/Lectures/Slides_06/Chapter_07/Pentium_Architecture.htm

26. Caching in Multi-Core Systems
26
Source: www.igvita.com/2010/08/18/multi-core-threads-message-passing/

27. Caching in Multi-Core Systems (Cont.)
27
Source: http://sips.inesc-id.pt/~nfvr/msc_theses/msc09e/

28. Increasing Cache Performance
 Large cache capacity
 Multiple-levels of cache
 Prefetching
 Fully associative cache
28

29. Perfecting – Example
 Which of the following code is faster?
sum = 0;
for (i = 0; i < n; i++)
for (j = 0; j < m; j++)
sum += a[i][j];
return sum;
sum = 0;
for (j = 0; j < m; j++)
for (i = 0; i < n; i++)
sum += a[i][j];
return sum;
29
1,1 1,2 1,3 1,4 2,1 2,2 2,3 2,4 3,1 …

30.  A collection of words are
called a block
 Multiple blocks are moved
between levels in cache
hierarchy
 Blocks are tagged with
memory address
 Tags are searched parallel
Cache Blocks
30
Source:
http://archive.arstechnica.com/paedia/c/cachi
ng/m-caching-5.html

31. Cache Associativity
 Defines where blocks can be placed in a cache
31
Source: Computer Architecture, A Quantitative Approach by John L. Hennessy and David A. Patterson

32. Intel Pentium 4 vs. AMD Opteron Memory
Hierarchy
32
CPU Pentium 4 (3.2 GHz) Opteron (2.8 GHz)
Instruction
Cache
Trace Cache 8K
micro-ops
2-way associative,
64 KB, 64B block
Data Cache 8-way associative, 16
KB, 64B block,
inclusive in L2
2-way associative,
64 KB, 64B block,
exclusive to L2
L2 Cache 8-way associative, 2
MB, 128B block
16-way associative,
1 MB, 64B block
Prefetch 8 streams to L2 1 stream to L2
Memory 200 MHz x 64 bits 200 MHz x 128 bits

33. Cache Replacement Policies
 When cache is full some of the cached blocks
need to be removed before bringing new ones in
 If cached blocks are dirty (written/updated), then they
need to be written to RAM
 Cache replacement policies
 Random
 Least Recently Used (LRU)
 Need to track last access time
 Least Frequently Used (LFU)
 Need to track no of accesses
 First In First Out (FIFO)
33

34. Cache Read Operations
34

35. Cache Misses
 When required item is not found in cache
 Miss rate – fraction of cache accesses that result
in a failure
 Types of misses
 Compulsory – 1st access to a block
 Capacity – limited cache capacity force blocked to be
removed from a cache & later retrieved
 Conflict – if placement strategy is not fully associative
 Average memory access time
= Hit time + Miss rate x Miss penalty
35

36. Cache Misses – Example
 Consider a cache with a block size of 4 words
 It takes 1 clock cycle to access a word in cache
 It takes 15 clock cycles to access a word from RAM
 How much time will it take to access a word that’s not in cache?
 4 x 15 + 1 = 61 CC
 This is Miss Penalty
 What is the average memory access time if miss rate is 0.4?
 1 + 0.4 x 61 = 25.4 CC
 What is the average memory access time if miss rate is 0.1?
 1 + 0.1 x 61 = 7.1CC
36

37. Cache Misses – Example
 Assume 40% of the instructions are data accessing
instruction
 A hit take 1 clock cycle & miss penalty is 100 clock
cycles
 Assume instruction miss rate is 4% & data access miss
rate is 12%, what is the average memory access time?
 100% x (1 + 4% x 100) + 40% x (1 + 12% x 100)
= 1 x (1 + 4) + 0.4 x (1 + 12)
= 5 + 0.4 x 13
= 10.2 CC
37

38. Permanent Storage – Hard Disks
38
Source: Upgrading & Repairing a PC by Scott Mueller
• Rigid disk (aluminum or glass) with a
magnetic coating

39. Hard Disks (Cont.)
39
Source: www.dataclinic.it/data-
recovery/hard-disk-
functionality.htm

40. Cylinders, Tracks, & Sectors
 Data access time depends on
 Seek time – time to position head over desired track
 Rotational latency – time till desire track is under the
head
 Read time – actual time to read data
40

41. Classification of Hard Disks
 Disk capacity
 250 MB, 1GB, 60GB, 500GB, 1TB, 2TB
 Type of controller
 IDE, SCSI, SATA
 Speed
 3600rpm, 5,400rpm, 7,200rpm, 10,000rpm
41

42. Solid State Drive (SSD)
 No moving parts
 Permanent storage
 Based on flash memory technologies
 High speed, large capacity, & robust
 More expensive
 Used in tablets, thin laptops, laptops
42

43. SSD Layout
43
Source:
www.blog.solidstatediskshop.com/2
012/how-does-an-ssd-write-part-2/

44. 44Source:
www.resellerspanel.com

45. Optical Storage
 Make use of light instead of magnetism
 Different forms of optical storage
 CD-ROM
 CD-R – Recordable
 CD-RW –Rewritable
 DVD – digital versatile/video disk
 DVD-R/DVD-RW
 Blu-ray
45

46. Geometry of a CD
46
Pit Land
0 1

47. Components of a CD-ROM drive
47
Source: Upgrading & Repairing a PC by Scott Mueller