Briefly described about the main points of cache memory

1. 1
Contents
1. Introduction………………………………………………………………………………2
2. Placement Of Caches…………………………………………………………………3
3. Cache addressing………………………………………………………………………3
4. Working Principles of Cache Memory……………………………………….4
5. Levels of Cache Memory…………………………………………………………..5
6. Cache mapping…………………………………………………………………………6
7. Methods of Cache mapping……………………………………………………..6
7.1 Direct cache mapping
7.2 Associative cache mapping
7.3 Set Associative cache mapping
8.Interaction Policies with Main Memory…………………………………….9
9.Conclusion………………………………………………………………………………..12

2. 2
1.Introduction
What is Cache Memory?
Generally Cache memory is a small-sized type of volatile computer memory that provides high-
speed data access to a processor and stores frequently used computer programs, applications
and data. It stores and retains data only until a computer is powered up. Cache memory is the
fastest memory in a computer. It is typically integrated on the motherboard and directly
embedded on the processor or main random access memory (RAM)
Cache is a memory which holds the recently utilized data by the processor. A bloc
of memory cannot be placed randomly in the cache and is restricted to a single cache line by
the “Placement Policy”. In other words, Placement Policy determines where a
particular memory block can be placed when it goes into the cache
We know that Cache memory is also called CPU memory, is high-speed static random access
memory (SRAM) that a computer microprocessor can access more quickly than it can access
regular random access memory (RAM). This memory is typically integrated directly into the CPU
chip or placed on a separate chip that has a separate bus interconnect with the CPU. The
purpose of cache memory is to store program instructions and data that are used repeatedly in
the operation of programs or information that the CPU is likely to need next. The computer
processor can access this information quickly from the cache rather than having to get it from
computer's main memory. Fast access to these instructions increases the overall speed of the
program.

3. 3
2.Placement of Cache Memory
Cache is a memory which holds the recently utilized data by the processor. A block
of memory cannot be placed randomly in the cache and is restricted to a single cache line by
the “Placement Policy”. In other words, Placement Policy determines where a
particular memory block can be placed when it goes into the cache.
Caches in modern systems are located on the same chip as the cores. Typically, systems will
have three levels of caches. The first level of cache is the L1 which is close to each core and can
be accessed at least once a cycle and will respond to a read hit in a handful or less of cycles. L1
caches are generally divided between instruction (L1-I) and data (L1-D) accesses. L1 caches are
usually less than 64kB.
When read or write address is not found in the L1, a miss request will be made to the L2 cache
which is generally private or shared by a cluster of cores. L2 caches often hold both instructions
and data but are sometimes split where the L2-D is private to a core and the L2–I is shared
between a cluster of cores. This cache will generally have a hit latency that is under 15 cycles
(sometimes significantly under). When an access to the L2 misses, the L3 cache is accessed. L2
caches are usually around 256kB
The L3 cache is usually shared by all cores on a processor and is much larger than the aggregate
capacity of all L1 and L2. Some L3 are up to 48MB! The access latency of the L3 cache can be a
few tens of cycles. When a request misses in the L3, main memory is accessed with hundreds of
cycles of latency. Avoiding that latency is the whole purpose of caches.
3.Cache Addressing
Logical cache (virtual cache)
Logical cache stores Data using virtual addresses. Virtual addresses use same address space for
different applications and so Cache access faster, before MMU address translation.

4. 4
Physical cache
Physical caches use physical addresses, do not need flush-ing on a context switch and therefore
data is preserved within the cache. The disadvantage is that all accesses must go through the
memory management unit, thus incurring delays. Particular care must also be exercised when
pages are swapped to and from disk.
If the processor does not invalidate any associated cache entries, the cache contents will be
different from the main memory con-tents by virtue of the new page that has been swapped in.
4. Working Principles of Cache Memory
When an application starts or data is to be read/written or any operation is to be performed
then the data and commands associated with the specific operation are shifted from a slow
moving storage device (magnetic device – hard disk, optical device – CD drive etc.) to a faster
device. This faster device is RAM – Random Access Memory. This RAM is type of DRAM
(Dynamic Random Access Memory). RAM is placed here because it is a faster device, and
whenever data/ commands/instructions are needed by Processor, they provide them at a faster

5. 5
rate than slow storage devices. They serve as a cache memory for the storage devices. Although
they are much faster than slow storage device but the processor processes at much a faster
pace and they are not able to provide the needed data/instructions at that rate. So there is
need of a device that is faster than RAM which could keep up with the speed of processor
needs. Therefore the data required is transmitted to the next level of fast memory, which is
known as CACHE memory. CACHE is also a type of RAM, but it is Static RAM – SRAM. SRAM are
faster and costlier then DRAM because it has flip-flops (6 transistors) to store data unlike DRAM
which uses 1 transistor and capacitor to store data in form of charge. Moreover they need not
be refreshed periodically (because of bistable latching circuitry) unlike DRAM making it faster.
So whenever the processor needs to perform an action or execute any command then it first
checks the state of the data registers. If the required instruction/data is not present over there,
then it looks in the first level of cache memory – L1, and if there also data is not present it further
goes to second and further third level of cache memory. Whenever the data needed by processor
is not found in the cache it is known as CACHE MISS and it leads to delay in the execution thus
making the system slow. If the data is found in cache memory it is known as CACHE HIT.
If the data needed is not found in any of the cache memory, the processor checks in RAM. And if
this also fails then it goes to look onto the slower storage device.
5.Levels of Cache Memory
CPU cache is divided into three main ‘Levels’, L1, L2, and L3. The hierarchy here is again
according to the speed, and thus, the size of the cache.
L1 (Level 1) cache is the fastest memory that is present in a computer system. In terms of
priority of access, L1 cache has the data the CPU is most likely to need while completing a
certain task.As far as the size goes, the L1 cache typically goes up to 256KB. However, some
really powerful CPUs are now taking it close to 1MB. Some server chipsets (like Intel’s top-end
Xeon CPUs) now have somewhere between 1-2MB of L1 cache.
L1 cache is also usually split two ways, into the instruction cache and the data cache. The
instruction cache deals with the information about the operation that the CPU has to perform,
while the data cache holds the data on which the operation is to be performed.

6. 6
L2 (Level 2) cache is slower than L1 cache, but bigger in size. Its size typically varies between
256KB to 8MB, although the newer, powerful CPUs tend to go past that. L2 cache holds data
that is likely to be accessed by the CPU next. In most modern CPUs, the L1 and L2 caches are
present on the CPU cores themselves, with each core getting its own cache.
L3 (Level 3) cache is the largest cache memory unit, and also the slowest one. It can range
between 4MB to upwards of 50MB. Modern CPUs have dedicated space on the CPU die for the
L3 cache, and it takes up a large chunk of the space.
6.Cache Mapping
Cache mapping is the method by which the contents of main memory are brought into
the cache and referenced by the CPU. The mapping method used directly affects the
performance of the entire computer system.
Cache Mapping Function
Mapping functions are used as a way to decide which main memory block occupies which line
of cache. As there are less lines of cache than there are main memory blocks, an algorithm is
needed to decide this. One block from main memory maps into only one possible line
of cache memory.
Cache Mapping Needed for?
It holds frequently requested data and instructions so that they are immediately available to
the CPU when needed. Cache memory is used to reduce the average time to access data from
the Main memory. The cache is a smaller and faster memory which stores copies of the data
from frequently used main memory locations.
7.Methods Of Cache Mapping
The three different types of mapping used for the purpose of cache memory are :
1)Direct Mapping.
2) Associative Mapping.
3)Set Associative Mapping.

7. 7
7.1 Direct Mapping:
In Direct mapping, assign each memory block to a specific line in the cache. If a line is previously
taken up by a memory block when a new block needs to be loaded, the old block is trashed. An
address space is split into two parts index field and a tag field. Each location in RAM has one
specific place in cache where the data will be held. Consider the cache to be like an array. Part
of the address is used as index into the cache to identify where the data will held. Since a data
block from RAM can only be in one specific line in the cache, it must always replace the one
block that was already there. There is no need for a replacement algorithm.
A direct-mapping is the simplest approach: each main memory address maps to exactly
one cache block. For example, on the right is a 16-byte main memory and a 4-
byte cache . Memory locations 0, 4, 8 and 12 all map to cache block 0. Addresses 1, 5, 9 and
13 map to cache block 1 etc.
Advantages Of Direct Mapping
In Direct mapping each memory block is mapped to exactly one block in the cache. Advantages
of direct mapping are that it is simple technique and The mapping scheme is easy to

8. 8
implement. Replacement is straight forward. Does not require any search technique to find a
block in cache. Simple hardware and it’s cost is low.
Disadvantages Of Direct Mapping
Each block of main memory maps to a fixed location in the cache;, therefore, if two different
blocks map to the same location in cache and they are continually referenced, the two blocks
will be continually swapped in and out . each main memory maps to fixed location the scheme
can suffer from many addresses colliding to the same block, thus causing the cache line to be
repeatedly evicted, even though there may be empty blocks that aren't being used, or being
used with less frequency. Poor cache utilization.
7.2 Associative Cache Mapping
An Associative Mapping use an associative memory. This memory is being accessed using its
contents. Each line of cache memory will accommodate the address (main memory)and the
contents of that address from the main memory. That is why this memory also called Content
Addressable Memory (CAM). It allows each Block of main memory to be stored in the cache. A
number of hardware schemes have been developed for translating main memory addresses to
cache memory addresses. The user does not need to know about the address translation, which
has the advantage that cache memory enhancements can be introduced into a computer
without a corresponding need
for modifying application software. The choice of cache mapping scheme affects cost and
performance, and there is no single best method that is appropriate for all situations. In this
section, an associative mapping scheme is studied.

9. 9
Advantages of Associative Mapping
Any main memory block can be placed into any cache slot.Regardless of how irregular the data
and program references are, if a slot is available for the block, it can be stored in the cache.
Dis-advantage of Associative Memory
Considerable hardware overhead needed for cache bookkeeping. There must be a mechanism
for searching the tag memory in parallel.
Set Associative Mapping
This form of mapping is an enhanced form of direct mapping where the drawbacks of direct mapping
are removed. Set associative addresses the problem of possible thrashing in the direct mapping method.
It does this by saying that instead of having exactly one line that a block can map to in the cache, we will
group a few lines together creating a set.
Then a block in memory can map to any one of the lines of a specific set..Set-associative mapping allows
that each word that is present in the cache can have two or more words in the main memory for the
same index address. Set associative cache mapping combines the best of direct and associative cache
mapping techniques.

10. 10
8.Interaction Policies with Main Memory
Reads dominate processor cache accesses. All instruction accesses are reads, and most
instructions do not write to memory. The block can be read at the same time that the tag is
read and compared, so the block read begins as soon as the block address is available. If the
read is a miss, there is no benefit - but also no harm; just ignore the value read.
The read policies are:
Read Through - reading a word from main memory to CPU
No Read Through - reading a block from main memory to cache and then from cache to
CPU
Such is not the case for writes. Modifying a block cannot begin until the tag is checked to see if
the address is a hit. Also the processor specifies the size of the write, usually between 1 and 8
bytes; only that portion of the block can be changed. In contrast, reads can access more bytes
than necessary without a problem.
The write policies on write hit often distinguish cache designs:
Write Through - the information is written to both the block in the cache and to the block in
the lower-level memory.
Advantage
- read miss never results in writes to main memory
- easy to implement
- main memory always has the most current copy of the data (consistent)
Disadvantage
- write is slower
- every write needs a main memory access
- as a result uses more memory bandwidth
Write Back
The process of write backs are-
i. Updates initially made in cache only
ii. Update bit for cache slot is set when update occurs
iii. If block is to be replaced, write to main memory only if update bit is set
iv. Other caches get out of sync
v. I/O must access main memory through cache

11. 11
vi. N.B. 15% of memory references are writes
Advantage
- writes occur at the speed of the cache memory
- multiple writes within a block require only one write to main memory
- as a result uses less memory bandwidth
Disadvantage
- harder to implement
- main memory is not always consistent with cache
- reads that result in replacement may cause writes of dirty blocks to main
There are two common options on a write miss:
Write Allocate - the block is loaded on a write miss, followed by the write-hit action.
No Write Allocate - the block is modified in the main memory and not loaded into the cache.
Although either write-miss policy could be used with write through or write back, write-
back caches generally use write allocate (hoping that subsequent writes to that block will be
captured by the cache) and
write-through caches often use no-write allocate (since subsequent writes to that block will
still have to go to memory).
Write Through with Write Allocate:
on hits it writes to cache and main memory
on misses it updates the block in main memory and brings the block to the cache
Bringing the block to cache on a miss does not make a lot of sense in this combination
because the next hit to this block will generate a write to main memory anyway (according to
Write Through policy)
Write Through with No Write Allocate:
on hits it writes to cache and main memory;
on misses it updates the block in main memory not bringing that block to the cache;
Subsequent writes to the block will update main memory because Write Through policy is

12. 12
employed. So, some time is saved not bringing the block in the cache on a miss because it
appears useless anyway.
Write Back with Write Allocate:
on hits it writes to cache setting bit for the block, main memory is not updated;
on misses it updates the block in main memory and brings the block to the cache;
Subsequent writes to the same block, if the block originally caused a miss, will hit in the
cache next time, setting dirty bit for the block. That will eliminate extra memory accesses and
result in very efficient execution compared with Write Through with Write Allocate
combination.
Write Back with No Write Allocate:
on hits it writes to cache setting bit for the block, main memory is not updated;
on misses it updates the block in main memory not bringing that block to the cache;
Subsequent writes to the same block, if the block originally caused a miss, will generate
misses all the way and result in very inefficient execution.
The following applet shows the dynamics of interaction policies. By clicking on white squares of
the applet one can chose the policy to test depending on whether it is read or write, hit or miss.
On read hit there is not much choice of policies. The block is just read from cache to CPU.
The following applet will show you the dynamics of interaction policies.
9.Conclusion
So we can say that Cache memory is a high-speed random access memory which is used by a
system Central processing unit for storing the data/instruction temporarily. It decreases the
execution time by storing the most frequent and most probable data and instructions “closer”
to the processor, where the systems CPU can quickly get it.

13. 13