SlideShare a Scribd company logo
© Copyright 2020 Xilinx
Xen Cache Coloring &
Real-Time
Stefano Stabellini
Marco Solieri
Luca Miccio
Giulio Corradi
© Copyright 2020 Xilinx
Xen in Embedded
Consolidation and Componentization
Isolation
 Security
 Reliability
 Safety
 Interference& Real-Time
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
sched=null
sched=credit
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
sched=null
sched=credit
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
sched=null
sched=rtds
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Xen Cache Coloring
CPUs clusters often share L2 cache
Interference via L2 cache affects performance
 App0 running on CPU0 can cause cache entries
evictions, which affect App1 running on CPU1
 App1 running on CPU1 could miss a deadline due to
App0’s behavior
 It can happen betweenApps running on the same OS &
between VMs on the same hypervisor
Hypervisor Solution: Cache Partitioning,
AKA Cache Coloring
 Each VM gets its own allocation of cache entries
 No shared cache entries between VMs
 Allows real-time apps to run with deterministic IRQ
latency
Core
1
Core
2
Core
3
Core
4
L2
D
D
R
© Copyright 2020 Xilinx
Colors Calculation
Detect Way Size
 via registers
 via xen command line
Calculate colors masks and total number of available colors
 Xilinx ZynqMP: way size is 65536
 Order 16 -> color bitmask 0xF000 -> 16 colors
D
D
R
© Copyright 2020 Xilinx
Colored Allocation
New colored memory allocator
 In alternative to Buddy
 Pages are stored by color in separate lists
Can be used for Xen memory as well as Domain memory
© Copyright 2020 Xilinx
Coloring Configuration
Xen command line
Dom0-less DomUs
xl domU config files
way_size=65536 xen_colors=0 dom0_colors=1-6
fdt set /chosen/domU0 colors <0x0 0x80>
colors=["10-11"]
© Copyright 2020 Xilinx
Benchmarking
 Hardware: Xilinx MPSoC
 Workload: Bare Metal application
 Stress: Bare Metal application
 Test configuration:
 Measured load: 1500 sums
 Internal load: 8000 sums every 250us
 Interrupt triggered every 8ms
 Samples:1000
 Colored configuration:
 Xen: ["0,1"]
 Motor Control DomU: ["4,7"]
 Stress DomU: ["8,11"]
 Stress2 DomU:["12,15"]
 Linux stress:["2,3"]
 Linux stress command: stress -m 1 --vm-bytes 8M --vm-stride 64 --vm-keep -t 3600 &
© Copyright 2020 Xilinx
L1 vs L2 cache
L1 cache can hide L2 cache interference effects in simple test scenarios
Make sure not use the L1 cache by accident during the measurements
© Copyright 2020 Xilinx
Benchmark #1: Motor Control Execution Time
Measure the time it takes to run a typical motor control execution routine, with
and without interference
© Copyright 2020 Xilinx
Results
No interference:
results for reference
Lower is better
© Copyright 2020 Xilinx
Results
Adding interference:
Xen Coloring
has the best execution time
© Copyright 2020 Xilinx
Results
Increasing interference:
• Xen Coloring is stable
• Other results worsen
© Copyright 2020 Xilinx
Results
Increasing interference
again:
• Xen Coloring is stable
• Other results worsen still
© Copyright 2020 Xilinx
Results
Comparing
Xen vs. Xen Colored
Lower is better
© Copyright 2020 Xilinx
Benchmark #2: interrupt response time
Measure the difference between the time when the interrupt service routine
runs and the time the interrupt is set to fire, with and without interference
Timer in Programmable Logic
© Copyright 2020 Xilinx
Results
No interference:
results for reference
Lower is better
© Copyright 2020 Xilinx
Results
Adding interference:
• Xen without Coloring
worsen
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
Adding more interference:
• Xen without Coloring
worsen again
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
Adding even more
interference:
• Xen without Coloring
worsen still
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
All results together
Xen Coloring provides
stable IRQ latency even
under severe levels of
interference
© Copyright 2020 Xilinx
Conclusions
Xen Cache Coloring offers
 Much lower execution times under stress
 Much lower IRQ latency under stress
Greatly improves Determinism -- measurements have far lower variance
Xen Cache Coloring effectively reduces the effects of cache interference
Enable deployments of real-time and non-real-time workloads on a single SoC
© Copyright 2020 Xilinx
Status
All patches online at:
 https://github.com/Xilinx/xen.git xilinx/release-2020.1
TODO
 Upstreaming
 Linux Dom0 not 1:1 mapped, i.e. PV drivers support
© Copyright 2020 Xilinx
Xen Cache Coloring: Demo
Xen
[0]
Baremetal
Latency
Measurements
[7]
Baremetal Memory
Stress
[10 11]
Dom0
[1 2 3 4 5 6]
© Copyright 2020 Xilinx
Baremetal Latency Measurements
© Copyright 2020 Xilinx
Baremetal Memory Stress
© Copyright 2020 Xilinx
Thank You

More Related Content

What's hot

VMware Horizon - news
VMware Horizon - newsVMware Horizon - news
VMware Horizon - news
Jürgen Ambrosi
 
Linux et les systèmes embarqués
Linux et les systèmes embarquésLinux et les systèmes embarqués
Linux et les systèmes embarqués
Stéphane Legrand
 
OMG DDS: The Data Distribution Service for Real-Time Systems
OMG DDS: The Data Distribution Service for Real-Time SystemsOMG DDS: The Data Distribution Service for Real-Time Systems
OMG DDS: The Data Distribution Service for Real-Time Systems
Angelo Corsaro
 
OpenShift 4 installation
OpenShift 4 installationOpenShift 4 installation
OpenShift 4 installation
Robert Bohne
 
Blockchain in IoT and Other Considerations by Dinis Guarda
Blockchain in IoT and Other Considerations by Dinis GuardaBlockchain in IoT and Other Considerations by Dinis Guarda
Blockchain in IoT and Other Considerations by Dinis Guarda
Dinis Guarda
 
Etude de la virtualisation : Réseau & Cloisonnement
Etude de la virtualisation : Réseau & CloisonnementEtude de la virtualisation : Réseau & Cloisonnement
Etude de la virtualisation : Réseau & Cloisonnement
Antoine Benkemoun
 
Open stack
Open stackOpen stack
Open stack
svm
 
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse HypervisorHKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
Linaro
 
Uncover the mysteries of infrastructure as code (iac)!
Uncover the mysteries of infrastructure as code (iac)!Uncover the mysteries of infrastructure as code (iac)!
Uncover the mysteries of infrastructure as code (iac)!
Prashant Kalkar
 
What is Ethereum 2.0
What is Ethereum 2.0 What is Ethereum 2.0
What is Ethereum 2.0
Vikram Khanna
 
vpn-site-a-site-avec-des-routeurs-cisco
 vpn-site-a-site-avec-des-routeurs-cisco vpn-site-a-site-avec-des-routeurs-cisco
vpn-site-a-site-avec-des-routeurs-cisco
Camara Assane
 
Implémentation des principaux algorithmes de chiffrements en Java
Implémentation des principaux algorithmes de chiffrements en JavaImplémentation des principaux algorithmes de chiffrements en Java
Implémentation des principaux algorithmes de chiffrements en Java
Ibn Tofail University
 
Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU
Andrés Leonardo Martinez Ortiz
 
cours-supcom-virt.pptx
cours-supcom-virt.pptxcours-supcom-virt.pptx
cours-supcom-virt.pptx
Manellansari
 
Understanding Proof of Work (PoW) and Proof of Stake (PoS) Algorithms
Understanding Proof of Work (PoW) and Proof of Stake (PoS) AlgorithmsUnderstanding Proof of Work (PoW) and Proof of Stake (PoS) Algorithms
Understanding Proof of Work (PoW) and Proof of Stake (PoS) Algorithms
Gautam Anand
 
Oracle OCI APIs and SDK
Oracle OCI APIs and SDKOracle OCI APIs and SDK
Oracle OCI APIs and SDK
Phil Wilkins
 
Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020
Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020
Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020
Smarter.World
 
Introduction to Filecoin
Introduction to Filecoin   Introduction to Filecoin
Introduction to Filecoin
Vanessa Lošić
 
solidity programming.pptx
solidity programming.pptxsolidity programming.pptx
solidity programming.pptx
RohiniBagul4
 
Open shift 4-update
Open shift 4-updateOpen shift 4-update
Open shift 4-update
SaeidVarmazyar
 

What's hot (20)

VMware Horizon - news
VMware Horizon - newsVMware Horizon - news
VMware Horizon - news
 
Linux et les systèmes embarqués
Linux et les systèmes embarquésLinux et les systèmes embarqués
Linux et les systèmes embarqués
 
OMG DDS: The Data Distribution Service for Real-Time Systems
OMG DDS: The Data Distribution Service for Real-Time SystemsOMG DDS: The Data Distribution Service for Real-Time Systems
OMG DDS: The Data Distribution Service for Real-Time Systems
 
OpenShift 4 installation
OpenShift 4 installationOpenShift 4 installation
OpenShift 4 installation
 
Blockchain in IoT and Other Considerations by Dinis Guarda
Blockchain in IoT and Other Considerations by Dinis GuardaBlockchain in IoT and Other Considerations by Dinis Guarda
Blockchain in IoT and Other Considerations by Dinis Guarda
 
Etude de la virtualisation : Réseau & Cloisonnement
Etude de la virtualisation : Réseau & CloisonnementEtude de la virtualisation : Réseau & Cloisonnement
Etude de la virtualisation : Réseau & Cloisonnement
 
Open stack
Open stackOpen stack
Open stack
 
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse HypervisorHKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
 
Uncover the mysteries of infrastructure as code (iac)!
Uncover the mysteries of infrastructure as code (iac)!Uncover the mysteries of infrastructure as code (iac)!
Uncover the mysteries of infrastructure as code (iac)!
 
What is Ethereum 2.0
What is Ethereum 2.0 What is Ethereum 2.0
What is Ethereum 2.0
 
vpn-site-a-site-avec-des-routeurs-cisco
 vpn-site-a-site-avec-des-routeurs-cisco vpn-site-a-site-avec-des-routeurs-cisco
vpn-site-a-site-avec-des-routeurs-cisco
 
Implémentation des principaux algorithmes de chiffrements en Java
Implémentation des principaux algorithmes de chiffrements en JavaImplémentation des principaux algorithmes de chiffrements en Java
Implémentation des principaux algorithmes de chiffrements en Java
 
Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU Tensorflow 2.0 and Coral Edge TPU
Tensorflow 2.0 and Coral Edge TPU
 
cours-supcom-virt.pptx
cours-supcom-virt.pptxcours-supcom-virt.pptx
cours-supcom-virt.pptx
 
Understanding Proof of Work (PoW) and Proof of Stake (PoS) Algorithms
Understanding Proof of Work (PoW) and Proof of Stake (PoS) AlgorithmsUnderstanding Proof of Work (PoW) and Proof of Stake (PoS) Algorithms
Understanding Proof of Work (PoW) and Proof of Stake (PoS) Algorithms
 
Oracle OCI APIs and SDK
Oracle OCI APIs and SDKOracle OCI APIs and SDK
Oracle OCI APIs and SDK
 
Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020
Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020
Dell Technologies Portfolio On One Single Page - POSTER - v4b June 2020
 
Introduction to Filecoin
Introduction to Filecoin   Introduction to Filecoin
Introduction to Filecoin
 
solidity programming.pptx
solidity programming.pptxsolidity programming.pptx
solidity programming.pptx
 
Open shift 4-update
Open shift 4-updateOpen shift 4-update
Open shift 4-update
 

Similar to Xen Cache Coloring: Interference-Free Real-Time System

Cache coloring Xen Summit 2020
Cache coloring Xen Summit 2020Cache coloring Xen Summit 2020
Cache coloring Xen Summit 2020
Stefano Stabellini
 
Rendering in the Cloud
Rendering in the CloudRendering in the Cloud
Rendering in the Cloud
Benjamin Shrive
 
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
Edge AI and Vision Alliance
 
Cit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_houseCit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_house
Erica Price
 
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
KTN
 
Encoding at Scale for Live Video Streaming
Encoding at Scale for Live Video StreamingEncoding at Scale for Live Video Streaming
Encoding at Scale for Live Video Streaming
Ray Adensamer
 
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
Edge AI and Vision Alliance
 
VDI-POC
VDI-POCVDI-POC
VDI-POC
Robert Jones
 
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
HostedbyConfluent
 
Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...
Principled Technologies
 
Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)
Intel® Software
 
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Lee Bushen
 
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
Edge AI and Vision Alliance
 
Novosco: Citrix tips and best practices
Novosco: Citrix tips and best practicesNovosco: Citrix tips and best practices
Novosco: Citrix tips and best practices
Novosco
 
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
Edge AI and Vision Alliance
 
Citrix Portfolio Updates
Citrix Portfolio UpdatesCitrix Portfolio Updates
Citrix Portfolio Updates
Pawel Serwan
 
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
Edge AI and Vision Alliance
 
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
The Linux Foundation
 
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Codemotion
 
Dcp
DcpDcp

Similar to Xen Cache Coloring: Interference-Free Real-Time System (20)

Cache coloring Xen Summit 2020
Cache coloring Xen Summit 2020Cache coloring Xen Summit 2020
Cache coloring Xen Summit 2020
 
Rendering in the Cloud
Rendering in the CloudRendering in the Cloud
Rendering in the Cloud
 
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
 
Cit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_houseCit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_house
 
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
 
Encoding at Scale for Live Video Streaming
Encoding at Scale for Live Video StreamingEncoding at Scale for Live Video Streaming
Encoding at Scale for Live Video Streaming
 
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
 
VDI-POC
VDI-POCVDI-POC
VDI-POC
 
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
 
Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...
 
Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)
 
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
 
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
 
Novosco: Citrix tips and best practices
Novosco: Citrix tips and best practicesNovosco: Citrix tips and best practices
Novosco: Citrix tips and best practices
 
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
 
Citrix Portfolio Updates
Citrix Portfolio UpdatesCitrix Portfolio Updates
Citrix Portfolio Updates
 
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
 
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
 
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
 
Dcp
DcpDcp
Dcp
 

More from Stefano Stabellini

Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Stefano Stabellini
 
Safety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen HypervisorSafety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen Hypervisor
Stefano Stabellini
 
Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022
Stefano Stabellini
 
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
Stefano Stabellini
 
Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...
Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...
Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...
Stefano Stabellini
 
ELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for EmbeddedELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for Embedded
Stefano Stabellini
 
RunX ELCE 2020
RunX ELCE 2020RunX ELCE 2020
RunX ELCE 2020
Stefano Stabellini
 
RunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edgeRunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edge
Stefano Stabellini
 
System Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperSystem Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and Lopper
Stefano Stabellini
 
Dom0less - Xen Developer Summit 2019
Dom0less  - Xen Developer Summit 2019Dom0less  - Xen Developer Summit 2019
Dom0less - Xen Developer Summit 2019
Stefano Stabellini
 
Xen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systemsXen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systems
Stefano Stabellini
 
Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)
Stefano Stabellini
 
Xen Project for ARM Servers
Xen Project for ARM ServersXen Project for ARM Servers
Xen Project for ARM Servers
Stefano Stabellini
 
Xen and OpenStack
Xen and OpenStackXen and OpenStack
Xen and OpenStack
Stefano Stabellini
 
XDS15: Project Raisin
XDS15: Project RaisinXDS15: Project Raisin
XDS15: Project Raisin
Stefano Stabellini
 
OpenStack and Xen
OpenStack and XenOpenStack and Xen
OpenStack and Xen
Stefano Stabellini
 

More from Stefano Stabellini (16)

Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdf
 
Safety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen HypervisorSafety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen Hypervisor
 
Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022Xen in Safety-Critical Systems - Critical Summit 2022
Xen in Safety-Critical Systems - Critical Summit 2022
 
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
 
Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...
Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...
Static Partitioning with Xen, LinuxRT, and Zephyr: A Concrete End-to-end Exam...
 
ELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for EmbeddedELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for Embedded
 
RunX ELCE 2020
RunX ELCE 2020RunX ELCE 2020
RunX ELCE 2020
 
RunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edgeRunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edge
 
System Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperSystem Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and Lopper
 
Dom0less - Xen Developer Summit 2019
Dom0less  - Xen Developer Summit 2019Dom0less  - Xen Developer Summit 2019
Dom0less - Xen Developer Summit 2019
 
Xen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systemsXen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systems
 
Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)
 
Xen Project for ARM Servers
Xen Project for ARM ServersXen Project for ARM Servers
Xen Project for ARM Servers
 
Xen and OpenStack
Xen and OpenStackXen and OpenStack
Xen and OpenStack
 
XDS15: Project Raisin
XDS15: Project RaisinXDS15: Project Raisin
XDS15: Project Raisin
 
OpenStack and Xen
OpenStack and XenOpenStack and Xen
OpenStack and Xen
 

Recently uploaded

42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
vaishalijagtap12
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
kalichargn70th171
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
sandeepmenon62
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
gapen1
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
Yara Milbes
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Vince Scalabrino
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
Alina Yurenko
 
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdfSoftware Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
kalichargn70th171
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
michniczscribd
 
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
dhavalvaghelanectarb
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 
Microsoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptxMicrosoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptx
jrodriguezq3110
 
Computer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdfComputer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdf
chandangoswami40933
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
ervikas4
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
ThousandEyes
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Paul Brebner
 
Cost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App DevelopmentCost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App Development
Softradix Technologies
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
widenerjobeyrl638
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
ICS
 

Recently uploaded (20)

42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert42 Ways to Generate Real Estate Leads - Sellxpert
42 Ways to Generate Real Estate Leads - Sellxpert
 
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
A Comprehensive Guide on Implementing Real-World Mobile Testing Strategies fo...
 
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptxOperational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
Operational ease MuleSoft and Salesforce Service Cloud Solution v1.0.pptx
 
bgiolcb
bgiolcbbgiolcb
bgiolcb
 
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
如何办理(hull学位证书)英国赫尔大学毕业证硕士文凭原版一模一样
 
The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024The Rising Future of CPaaS in the Middle East 2024
The Rising Future of CPaaS in the Middle East 2024
 
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery FleetStork Product Overview: An AI-Powered Autonomous Delivery Fleet
Stork Product Overview: An AI-Powered Autonomous Delivery Fleet
 
Going AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applicationsGoing AOT: Everything you need to know about GraalVM for Java applications
Going AOT: Everything you need to know about GraalVM for Java applications
 
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdfSoftware Test Automation - A Comprehensive Guide on Automated Testing.pdf
Software Test Automation - A Comprehensive Guide on Automated Testing.pdf
 
Beginner's Guide to Observability@Devoxx PL 2024
Beginner's  Guide to Observability@Devoxx PL 2024Beginner's  Guide to Observability@Devoxx PL 2024
Beginner's Guide to Observability@Devoxx PL 2024
 
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024Flutter vs. React Native: A Detailed Comparison for App Development in 2024
Flutter vs. React Native: A Detailed Comparison for App Development in 2024
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 
Microsoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptxMicrosoft-Power-Platform-Adoption-Planning.pptx
Microsoft-Power-Platform-Adoption-Planning.pptx
 
Computer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdfComputer Science & Engineering VI Sem- New Syllabus.pdf
Computer Science & Engineering VI Sem- New Syllabus.pdf
 
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptxMigration From CH 1.0 to CH 2.0 and  Mule 4.6 & Java 17 Upgrade.pptx
Migration From CH 1.0 to CH 2.0 and Mule 4.6 & Java 17 Upgrade.pptx
 
Assure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyesAssure Contact Center Experiences for Your Customers With ThousandEyes
Assure Contact Center Experiences for Your Customers With ThousandEyes
 
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
Why Apache Kafka Clusters Are Like Galaxies (And Other Cosmic Kafka Quandarie...
 
Cost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App DevelopmentCost-Effective Strategies For iOS App Development
Cost-Effective Strategies For iOS App Development
 
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
美洲杯赔率投注网【​网址​🎉3977·EE​🎉】
 
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA ComplianceSecure-by-Design Using Hardware and Software Protection for FDA Compliance
Secure-by-Design Using Hardware and Software Protection for FDA Compliance
 

Xen Cache Coloring: Interference-Free Real-Time System

  • 1. © Copyright 2020 Xilinx Xen Cache Coloring & Real-Time Stefano Stabellini Marco Solieri Luca Miccio Giulio Corradi
  • 2. © Copyright 2020 Xilinx Xen in Embedded Consolidation and Componentization Isolation  Security  Reliability  Safety  Interference& Real-Time
  • 3. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU
  • 4. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU sched=null sched=credit vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU
  • 5. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU sched=null sched=credit vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU
  • 6. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU sched=null sched=rtds vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU
  • 7. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 8. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 9. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 10. © Copyright 2020 Xilinx Xen Cache Coloring CPUs clusters often share L2 cache Interference via L2 cache affects performance  App0 running on CPU0 can cause cache entries evictions, which affect App1 running on CPU1  App1 running on CPU1 could miss a deadline due to App0’s behavior  It can happen betweenApps running on the same OS & between VMs on the same hypervisor Hypervisor Solution: Cache Partitioning, AKA Cache Coloring  Each VM gets its own allocation of cache entries  No shared cache entries between VMs  Allows real-time apps to run with deterministic IRQ latency Core 1 Core 2 Core 3 Core 4 L2 D D R
  • 11. © Copyright 2020 Xilinx Colors Calculation Detect Way Size  via registers  via xen command line Calculate colors masks and total number of available colors  Xilinx ZynqMP: way size is 65536  Order 16 -> color bitmask 0xF000 -> 16 colors D D R
  • 12. © Copyright 2020 Xilinx Colored Allocation New colored memory allocator  In alternative to Buddy  Pages are stored by color in separate lists Can be used for Xen memory as well as Domain memory
  • 13. © Copyright 2020 Xilinx Coloring Configuration Xen command line Dom0-less DomUs xl domU config files way_size=65536 xen_colors=0 dom0_colors=1-6 fdt set /chosen/domU0 colors <0x0 0x80> colors=["10-11"]
  • 14. © Copyright 2020 Xilinx Benchmarking  Hardware: Xilinx MPSoC  Workload: Bare Metal application  Stress: Bare Metal application  Test configuration:  Measured load: 1500 sums  Internal load: 8000 sums every 250us  Interrupt triggered every 8ms  Samples:1000  Colored configuration:  Xen: ["0,1"]  Motor Control DomU: ["4,7"]  Stress DomU: ["8,11"]  Stress2 DomU:["12,15"]  Linux stress:["2,3"]  Linux stress command: stress -m 1 --vm-bytes 8M --vm-stride 64 --vm-keep -t 3600 &
  • 15. © Copyright 2020 Xilinx L1 vs L2 cache L1 cache can hide L2 cache interference effects in simple test scenarios Make sure not use the L1 cache by accident during the measurements
  • 16. © Copyright 2020 Xilinx Benchmark #1: Motor Control Execution Time Measure the time it takes to run a typical motor control execution routine, with and without interference
  • 17. © Copyright 2020 Xilinx Results No interference: results for reference Lower is better
  • 18. © Copyright 2020 Xilinx Results Adding interference: Xen Coloring has the best execution time
  • 19. © Copyright 2020 Xilinx Results Increasing interference: • Xen Coloring is stable • Other results worsen
  • 20. © Copyright 2020 Xilinx Results Increasing interference again: • Xen Coloring is stable • Other results worsen still
  • 21. © Copyright 2020 Xilinx Results Comparing Xen vs. Xen Colored Lower is better
  • 22. © Copyright 2020 Xilinx Benchmark #2: interrupt response time Measure the difference between the time when the interrupt service routine runs and the time the interrupt is set to fire, with and without interference Timer in Programmable Logic
  • 23. © Copyright 2020 Xilinx Results No interference: results for reference Lower is better
  • 24. © Copyright 2020 Xilinx Results Adding interference: • Xen without Coloring worsen • Xen Coloring is stable
  • 25. © Copyright 2020 Xilinx Results Adding more interference: • Xen without Coloring worsen again • Xen Coloring is stable
  • 26. © Copyright 2020 Xilinx Results Adding even more interference: • Xen without Coloring worsen still • Xen Coloring is stable
  • 27. © Copyright 2020 Xilinx Results All results together Xen Coloring provides stable IRQ latency even under severe levels of interference
  • 28. © Copyright 2020 Xilinx Conclusions Xen Cache Coloring offers  Much lower execution times under stress  Much lower IRQ latency under stress Greatly improves Determinism -- measurements have far lower variance Xen Cache Coloring effectively reduces the effects of cache interference Enable deployments of real-time and non-real-time workloads on a single SoC
  • 29. © Copyright 2020 Xilinx Status All patches online at:  https://github.com/Xilinx/xen.git xilinx/release-2020.1 TODO  Upstreaming  Linux Dom0 not 1:1 mapped, i.e. PV drivers support
  • 30. © Copyright 2020 Xilinx Xen Cache Coloring: Demo Xen [0] Baremetal Latency Measurements [7] Baremetal Memory Stress [10 11] Dom0 [1 2 3 4 5 6]
  • 31. © Copyright 2020 Xilinx Baremetal Latency Measurements
  • 32. © Copyright 2020 Xilinx Baremetal Memory Stress
  • 33. © Copyright 2020 Xilinx Thank You