Downloaded 19 times
Uploaded byHeechul Yun
Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time Systems
This document discusses the limitations of cache partitioning for performance isolation in multicore real-time systems, revealing that contention for miss-status-holding registers can negate the benefits of partitioning. Through experiments on various multicore architectures, it was found that cache partitioning does not ensure performance isolation under several conditions, particularly in out-of-order cores. The authors propose an OS/architecture collaborative solution to enhance cache performance isolation by managing MSHR resources more effectively.
More Related Content
Similar to Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time Systems
![An Overview of [Linux] Kernel Lock Improvements -- Linuxcon NA 2014](https://cdn.slidesharecdn.com/ss_thumbnails/linuxcon-2014-locking-final-140826225054-phpapp01-thumbnail.jpg?width=640&height=640&fit=bounds)
Taming Non-blocking Caches to Improve Isolation in Multicore Real-Time Systems
- 1. Taming Non-blocking Cachesto Improve Isolation in Multicore Real-Time Systems Prathap Kumar Valsan, Heechul Yun, Farzad Farshchi University of Kansas 1
- 2. High-Performance Multicores for Real-TimeSystems • Why? – Intelligence more performance – Space, weight, power (SWaP), cost 2
- 3. Time Predictability Challenge 3 Core1Core2 Core3 Core4 Memory Controller (MC) Shared Cache • Shared hardware resource contention can cause significant interference delays • Shared cache is a major shared resource DRAM Task 1 Task 2 Task 3 Task 4
- 4. Cache Partitioning • Canbe done in either software or hardware – Software: page coloring – Hardware: way partitioning • Eliminate unwanted cache-line evictions • Common assumption – cache partitioning performance isolation • If working-set fits in the cache partition • Not necessarily true on out-of-order cores using non-blocking caches 4
- 5. Outline • Evaluate theisolation effect of cache partitioning – On four real COTS multicore architectures – Observed up to 21X slowdown of a task running on a dedicated core accessing a dedicated cache partition • Understand the source of contention – Using a cycle-accurate full system simulator • Propose an OS/architecture collaborative solution – For better cache performance isolation 5
- 6. Memory-Level Parallelism (MLP) •Broadly defined as the number of concurrent memory requests that a given architecture can handle at a time 6
- 7. Non-blocking Cache(*) • Canserve cache hits under multiple cache misses – Essential for an out-of-order core and any multicore • Miss-Status-Holding Registers (MSHRs) – On a miss, allocate a MSHR entry to track the req. – On receiving the data, clear the MSHR entry 7 cpu cpu miss hit miss Miss penalty Miss penalty stall only when result is needed Multiple outstanding misses (*) D. Kroft. “Lockup-free instruction fetch/prefetch cache organization,” ISCA’81
- 8. Non-blocking Cache • #ofcache MSHRs memory-level parallelism (MLP) of the cache • What happens if all MSHRs are occupied? – The cache is locked up – Subsequent accesses---including cache hits---to the cache stall – We will see the impact of this in later experiments 8
- 9. COTS Multicore Platforms Cortex-A7Cortex-A9 Cortex-A15 Nehalem Core 4core @ 1.4GHz In-order 4core @ 1.7GHz Out-of-order 4core @ 2.0GHz Out-of-order 4core @ 2.8GHz Out-of-order LLC (shared) 512KB 1MB 2MB 8MB 9 • COTS multicore platforms – Odroid-XU4: 4x Cortex-A7 and 4x Cortex-A15 – Odroid-U3: 4x Cortex-A9 – Dell desktop: Intel Xeon quad-core (Nehalem)
- 10. Identified MLP • LocalMLP – MLP of a core-private cache • Global MLP – MLP of the shared cache (and DRAM) 10 (See paper for our experimental identification method) Cortex-A7 Cortex-A9 Cortex-A15 Nehalem Local MLP 1 4 6 10 Global MLP 4 4 11 16
- 11. Cache Interference Experiments •Measure the performance of the ‘subject’ – (1) alone, (2) with co-runners – LLC is partitioned (equal partition) using PALLOC (*) • Q: Does cache partitioning provide isolation? 11 DRAM LLC Core1 Core2 Core3 Core4 subject co-runner(s) (*) Heechul Yun, Renato Mancuso, Zheng-Pei Wu, Rodolfo Pellizzoni. “PALLOC: DRAM Bank-Aware Mem ory Allocator for Performance Isolation on Multicore Platforms.” RTAS’14
- 12. IsolBench: Synthetic Workloads •Latency – A linked-list traversal, data dependency, one outstanding miss • Bandwidth – An array reads or writes, no data dependency, multiple misses • Subject benchmarks: LLC partition fitting 12 Working-set size: (LLC) < ¼ LLC cache-hits, (DRAM) > 2X LLC cache misses
- 13. Latency(LLC) vs. BwRead(DRAM) •No interference on Cortex-A7 and Nehalem • On Cortex-A15, Latency(LLC) suffers 3.8X slowdown – despite partitioned LLC 13
- 14. BwRead(LLC) vs. BwRead(DRAM) •Up to 10.6X slowdown on Cortex-A15 • Cache partitioning != performance isolation – On all tested out-of-order cores (A9, A15, Nehalem) 14
- 15. BwRead(LLC) vs. BwWrite(DRAM) •Up to 21X slowdown on Cortex-A15 • Writes generally cause more slowdowns – Due to write-backs 15
- 16. EEMBC and SD-VBS •X-axis: EEMBC, SD-VBS (cache partition fitting) – Co-runners: BwWrite(DRAM) • Cache partitioning != performance isolation 16 Cortex-A7 (in-order) Cortex-A15 (out-of-order)
- 17. MSHR Contention • Shortageof cache MSHRs lock up the cache • LLC MSHRs are shared resources – 4cores x L1 cache MSHRs > LLC MSHRs 17 Cortex-A7 Cortex-A9 Cortex-A15 Nehalem Local MLP 1 4 6 10 Global MLP 4 4 11 16 (See paper for our experimental identification method)
- 18. Outline • Evaluate theisolation effect of cache partitioning • Understand the source of contention – Effect of LLC MSHRs – Effect of L1 MSHRs • Propose an OS/architecture collaborative solution 18
- 19. Simulation Setup • Gem5full-system simulator – 4 out-of-order cores (modeling Cortex-A15) • L1: 32K I&D, LLC (L2): 2048K – Vary #of L1 and LLC MSHRs • Linux 3.14 – Use PALLOC to partition LLC • Workload – IsolBench, EEMBC, SD-VBS, SPEC2006 19 DRAM LLC Core1 Core2 Core3 Core4 subject co-runner(s)
- 20. Effect of LLC(Shared) MSHRs • Increasing LLC MSHRs eliminates the MSHR contention • Issue: scalable? – MSHRs are highly associative (must be accessed in parallel) 20 L1:6/LLC:12 MSHRs L1:6/LLC:24 MSHRs
- 21. Effect of L1(Private) MSHRs • Reducing L1 MSHRs can eliminate contention • Issue: single thread performance. How much? – It depends. L1 MSHRs are often underutilized – EEMBC: low, SD-VBS: medium, SPEC2006: high 21 EEMBC, SD-VBS IsolBench,SPEC2006
- 22. Outline • Evaluate theisolation effect of cache partitioning • Understand the source of contention • Propose an OS/architecture collaborative solution 22
- 23. OS Controlled MSHRPartitioning 23 • Add two registers in each core’s L1 cache – TargetCount: max. MSHRs (set by OS) – ValidCount: used MSHRs (set by HW) • OS can control each core’s MLP – By adjusting the core’s TargetCount register Block Addr.Valid Issue Target Info. Block Addr.Valid Issue Target Info. Block Addr.Valid Issue Target Info. MSHR 1 MSHR 2 MSHR n TargetCount ValidCount Last Level Cache (LLC) Core 1 Core 2 Core 3 Core 4 MSHRs MSHRs MSHRs MSHRs MSHRs
- 24. OS Scheduler Design •Partition LLC MSHRs by enforcing • When RT tasks are scheduled – RT tasks: reserve MSHRs – Non-RT tasks: share remaining MSHRs • Implemented in Linux kernel – prepare_task_switch() at kernel/sched/core.c 24
- 25. Case Study • 4RT tasks (EEMBC) – One RT per core – Reserve 2 MSHRs – P: 20,30,40,60ms – C: ~8ms • 4 NRT tasks – One NRT per core – Run on slack • Up to 20% WCET reduction – Compared to cache partition only 25
- 26. Conclusion • Evaluated theeffect of cache partitioning on four real COTS multicore architectures and a full system simulator • Found cache partitioning does not ensure cache (hit) performance isolation due to MSHR contention • Proposed low-cost architecture extension and OS scheduler support to eliminate MSHR contention • Availability – https://github.com/CSL-KU/IsolBench – Synthetic benchmarks (IsolBench),test scripts, kernel patches 26
- 27. Exp3: BwRead(LLC) vs.BwRead(LLC) • Cache bandwidth contention is not the main source – Higher cache-bandwidth co-runners cause less slowdowns 27
- 28. EEMBC, SD-VBS Workload •Subject – Subset of EEMBC, SD-VBS – High L2 hit, Low L2 miss • Co-runners – BwWrite(DRAM): High L2 miss, write-intensive 28
Editor's Notes
- #2 This talk is about improving isolation in modern multicore real-time systems focusing on shared cache.
- #3 High-performance multicore processors are increasingly demanded in many modern real-time systems to satisfy the ever increasing performance need while reducing size, weight, and power consumption.
- #4 Unfortunately, it is well know that providing timing predictability is very difficult in multicore processors due to contention in shared hardware resources. In this work, we focus on shared cache.
- #5 Cache partitioning is a well known technique to improve isolation for real-time systems. And it can be done either in software (by the OS) or in hardware. Either way, the goal of partitioning is to eliminate the unwanted cache-line evictions by providing isolated space per core/task. Most literature equate cache partitioning and cache performance isolation. However, as we will show in this work, partitioning cache doesn’t necessarily provide performance isolation in modern high-performance multicore architectures.
- #6 This talk is composed of three parts. First, we evaluate the isolation effect cache partitioning on four real COTS multicore architectures. Surprisingly, we observe up to 21X slowdown of a task running on a dedicated core, accessing dedicated cache partition, and almost all memory accesses are cache hits. We then further investigate and better understand the source of the contention using a cycle accurate full system simulator. Finally, based on the understanding, we propose an OS and architecture collaborative solution to to provide true cache performance isolation.
- #8 Before we begin, let me briefly introduce essential background on non-blocking caches. A non-blocking cache can continue to serve cache-hit requests under multiple cache misses. Supporting multiple outstanding misses is important to leverage memory-level parallelism. And it is especially important for out-of-order cores, but in-order multicore also can benefit from using non-blocking shared caches. In a non-blocking cache, there are special hardware registers known as miss-status-holing registers (MSHRs). When a cache miss occurs, the cache controller allocates a MSHR entry to track the outgoing cache-miss and it is cleared when the data is delivered from the lower-level memory hierarch,
- #9 The number of MSHRs essentially determines the parallelism of the cache. Now, because MSHRs are limited and getting the data from DRAM could take 100s of cpu cycles, it is possible that MSRHs of a cache are fully occupied with outstanding misses. If that happens, the cache lock-up and all the subsequent accesses to the cache will be stalled.
- #10 For experiments, we use four COTS multicore architectures. One in-order and three out-of-order architectuers.
- #12 On the platforms, we perform a set of cache-interference experiments. The basic setup is as follows. We ran a subject benchmark on one core, and we increase the number of co-runners on the other cores. I would like to stress that the LLC is partitioned on a per-core basis. So, what we would like to see is whether cache portioning provide cache performance isolation.
- #13 First, we use a benchmark suite called IsolBench that consists of two synthetic benchmarks latency and bandwidth. The latency benchmark is a simple linked-list traversal program. Due to data dependency, only one outstanding miss can be generated at a time. The bandwidth benchmark is a simple array access program. As there’s no data dependency, multiple concurrent accesses can be made in an out-of-order core. We created 6 workloads with varying their working-set sizes. Here, LLC denotes less than the given cache partiion while (DRAM) denotes twice size of the the LLC. In all workloads, the subject benchmarks always fit in their cache partitions. In other words, accesses are cache hits. So, ideally, cache partitioning should provide performance isolation regardless of the co-runners.
- #17 We also used real benchmarks as subjects and get the similar results. That is isolation is provided in in-order cores but not so in out-of-order cores
- #18 We attribute this to the MSHR contention problem. This table shows the experimentally obtained core local and global memory-level parallelism of the tested platforms. Detailed methods to obtain MLP is in paper. These numbers were also validated from the manuals of the processors. What this table suggests is that the MSHRs in the LLCs are smaller than the sum of L1 MSHRs on the tested out-of-order cores. As I mentioned before, then the shortage of the LLC MSHRs can lock-up the LLC, even if the cache space is partitioned.
- #19 Next, we validate the MSHR contention problem using a cycle-accurate full system simulator. We also perform a set of sensitive analysis to better understand the effects of MSHRs in the local and shared caches.
- #28 At first, we suspected that this may be because of shared bus that connects the LLC and the cores. But we found that it was not the case from next experiment. In this experiment, we chaged the corunners’ working-set sizes to fit in their cache partitions. Because now that co-runners’ memory accesses are also cache hits, the cache bandwidth demand, and therefore the shared bus contention would have been increased. Yet, the slowdown becomes much smaller than the previous experiment where cache bandwidth usages were much lower.