SlideShare a Scribd company logo
© Copyright 2020 Xilinx
Xen Cache Coloring &
Real-Time
Stefano Stabellini
Marco Solieri
Luca Miccio
Giulio Corradi
© Copyright 2020 Xilinx
Xen in Embedded
Consolidation and Componentization
Isolation
 Security
 Reliability
 Safety
 Interference& Real-Time
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
sched=null
sched=credit
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
sched=null
sched=credit
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
© Copyright 2020 Xilinx
Xen Schedulers
CPU CPU
CPU CPU
sched=null
sched=rtds
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
vCPU vCPU
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Xen Cache Coloring
CPUs clusters often share L2 cache
Interference via L2 cache affects performance
 App0 running on CPU0 can cause cache entries
evictions, which affect App1 running on CPU1
 App1 running on CPU1 could miss a deadline due to
App0’s behavior
 It can happen betweenApps running on the same OS &
between VMs on the same hypervisor
Hypervisor Solution: Cache Partitioning,
AKA Cache Coloring
 Each VM gets its own allocation of cache entries
 No shared cache entries between VMs
 Allows real-time apps to run with deterministic IRQ
latency
Core
1
Core
2
Core
3
Core
4
L2
D
D
R
© Copyright 2020 Xilinx
Colors Calculation
Detect Way Size
 via registers
 via xen command line
Calculate colors masks and total number of available colors
 Xilinx ZynqMP: way size is 65536
 Order 16 -> color bitmask 0xF000 -> 16 colors
D
D
R
© Copyright 2020 Xilinx
Colored Allocation
New colored memory allocator
 In alternative to Buddy
 Pages are stored by color in separate lists
Can be used for Xen memory as well as Domain memory
© Copyright 2020 Xilinx
Coloring Configuration
Xen command line
Dom0-less DomUs
xl domU config files
way_size=65536 xen_colors=0 dom0_colors=1-6
fdt set /chosen/domU0 colors <0x0 0x80>
colors=["10-11"]
© Copyright 2020 Xilinx
Benchmarking
 Hardware: Xilinx MPSoC
 Workload: Bare Metal application
 Stress: Bare Metal application
 Test configuration:
 Measured load: 1500 sums
 Internal load: 8000 sums every 250us
 Interrupt triggered every 8ms
 Samples:1000
 Colored configuration:
 Xen: ["0,1"]
 Motor Control DomU: ["4,7"]
 Stress DomU: ["8,11"]
 Stress2 DomU:["12,15"]
 Linux stress:["2,3"]
 Linux stress command: stress -m 1 --vm-bytes 8M --vm-stride 64 --vm-keep -t 3600 &
© Copyright 2020 Xilinx
L1 vs L2 cache
L1 cache can hide L2 cache interference effects in simple test scenarios
Make sure not use the L1 cache by accident during the measurements
© Copyright 2020 Xilinx
Benchmark #1: Motor Control Execution Time
Measure the time it takes to run a typical motor control execution routine, with
and without interference
© Copyright 2020 Xilinx
Results
No interference:
results for reference
Lower is better
© Copyright 2020 Xilinx
Results
Adding interference:
Xen Coloring
has the best execution time
© Copyright 2020 Xilinx
Results
Increasing interference:
• Xen Coloring is stable
• Other results worsen
© Copyright 2020 Xilinx
Results
Increasing interference
again:
• Xen Coloring is stable
• Other results worsen still
© Copyright 2020 Xilinx
Results
Comparing
Xen vs. Xen Colored
Lower is better
© Copyright 2020 Xilinx
Benchmark #2: interrupt response time
Measure the difference between the time when the interrupt service routine
runs and the time the interrupt is set to fire, with and without interference
Timer in Programmable Logic
© Copyright 2020 Xilinx
Results
No interference:
results for reference
Lower is better
© Copyright 2020 Xilinx
Results
Adding interference:
• Xen without Coloring
worsen
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
Adding more interference:
• Xen without Coloring
worsen again
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
Adding even more
interference:
• Xen without Coloring
worsen still
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
All results together
Xen Coloring provides
stable IRQ latency even
under severe levels of
interference
© Copyright 2020 Xilinx
Conclusions
Xen Cache Coloring offers
 Much lower execution times under stress
 Much lower IRQ latency under stress
Greatly improves Determinism -- measurements have far lower variance
Xen Cache Coloring effectively reduces the effects of cache interference
Enable deployments of real-time and non-real-time workloads on a single SoC
© Copyright 2020 Xilinx
Status
All patches online at:
 https://github.com/Xilinx/xen.git xilinx/release-2020.1
TODO
 Upstreaming
 Linux Dom0 not 1:1 mapped, i.e. PV drivers support
© Copyright 2020 Xilinx
Xen Cache Coloring: Demo
Xen
[0]
Baremetal
Latency
Measurements
[7]
Baremetal Memory
Stress
[10 11]
Dom0
[1 2 3 4 5 6]
© Copyright 2020 Xilinx
Baremetal Latency Measurements
© Copyright 2020 Xilinx
Baremetal Memory Stress
© Copyright 2020 Xilinx
Thank You

More Related Content

What's hot

Securing Prometheus exporters using HashiCorp Vault
Securing Prometheus exporters using HashiCorp VaultSecuring Prometheus exporters using HashiCorp Vault
Securing Prometheus exporters using HashiCorp Vault
Bram Vogelaar
 
Introduction to CMake
Introduction to CMakeIntroduction to CMake
Introduction to CMake
Dimitrios Platis
 
Dom0less - Xen Developer Summit 2019
Dom0less  - Xen Developer Summit 2019Dom0less  - Xen Developer Summit 2019
Dom0less - Xen Developer Summit 2019
Stefano Stabellini
 
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse HypervisorHKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
Linaro
 
Improving hyperconverged performance
Improving hyperconverged performanceImproving hyperconverged performance
Improving hyperconverged performance
Denis Chapligin
 
Db2 migration -_tips,_tricks,_and_pitfalls
Db2 migration -_tips,_tricks,_and_pitfallsDb2 migration -_tips,_tricks,_and_pitfalls
Db2 migration -_tips,_tricks,_and_pitfalls
sam2sung2
 
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Henning Jacobs
 
WorkShop: Introducción a GIT
WorkShop: Introducción a GITWorkShop: Introducción a GIT
WorkShop: Introducción a GIT
Keopx
 
Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...
Michelle Holley
 
Writing a fast HTTP parser
Writing a fast HTTP parserWriting a fast HTTP parser
Writing a fast HTTP parser
fukamachi
 
XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...
XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...
XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...
The Linux Foundation
 
Pgsodium's Features: those not provided by pgcrypto and integration with rem...
 Pgsodium's Features: those not provided by pgcrypto and integration with rem... Pgsodium's Features: those not provided by pgcrypto and integration with rem...
Pgsodium's Features: those not provided by pgcrypto and integration with rem...
EDB
 
[Public] gerrit concepts and workflows
[Public] gerrit   concepts and workflows[Public] gerrit   concepts and workflows
[Public] gerrit concepts and workflows
Yanbin Kong
 
Configurable horizontal pod autoscaler
Configurable horizontal pod autoscalerConfigurable horizontal pod autoscaler
Configurable horizontal pod autoscaler
Paul Guth
 
Derbycon - The Unintended Risks of Trusting Active Directory
Derbycon - The Unintended Risks of Trusting Active DirectoryDerbycon - The Unintended Risks of Trusting Active Directory
Derbycon - The Unintended Risks of Trusting Active Directory
Will Schroeder
 
Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!
Krishna-Kumar
 
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, XilinxXPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
The Linux Foundation
 
I2C Subsystem In Linux-2.6.24
I2C Subsystem In Linux-2.6.24I2C Subsystem In Linux-2.6.24
I2C Subsystem In Linux-2.6.24
Varun Mahajan
 
(Ab)Using GPOs for Active Directory Pwnage
(Ab)Using GPOs for Active Directory Pwnage(Ab)Using GPOs for Active Directory Pwnage
(Ab)Using GPOs for Active Directory Pwnage
Petros Koutroumpis
 
Docker swarm
Docker swarmDocker swarm

What's hot (20)

Securing Prometheus exporters using HashiCorp Vault
Securing Prometheus exporters using HashiCorp VaultSecuring Prometheus exporters using HashiCorp Vault
Securing Prometheus exporters using HashiCorp Vault
 
Introduction to CMake
Introduction to CMakeIntroduction to CMake
Introduction to CMake
 
Dom0less - Xen Developer Summit 2019
Dom0less  - Xen Developer Summit 2019Dom0less  - Xen Developer Summit 2019
Dom0less - Xen Developer Summit 2019
 
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse HypervisorHKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
HKG18- 115 - Partitioning ARM Systems with the Jailhouse Hypervisor
 
Improving hyperconverged performance
Improving hyperconverged performanceImproving hyperconverged performance
Improving hyperconverged performance
 
Db2 migration -_tips,_tricks,_and_pitfalls
Db2 migration -_tips,_tricks,_and_pitfallsDb2 migration -_tips,_tricks,_and_pitfalls
Db2 migration -_tips,_tricks,_and_pitfalls
 
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
Optimizing Kubernetes Resource Requests/Limits for Cost-Efficiency and Latenc...
 
WorkShop: Introducción a GIT
WorkShop: Introducción a GITWorkShop: Introducción a GIT
WorkShop: Introducción a GIT
 
Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...Accelerating Virtual Machine Access with the Storage Performance Development ...
Accelerating Virtual Machine Access with the Storage Performance Development ...
 
Writing a fast HTTP parser
Writing a fast HTTP parserWriting a fast HTTP parser
Writing a fast HTTP parser
 
XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...
XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...
XPDDS18: Design and Implementation of Automotive: Virtualization Based on Xen...
 
Pgsodium's Features: those not provided by pgcrypto and integration with rem...
 Pgsodium's Features: those not provided by pgcrypto and integration with rem... Pgsodium's Features: those not provided by pgcrypto and integration with rem...
Pgsodium's Features: those not provided by pgcrypto and integration with rem...
 
[Public] gerrit concepts and workflows
[Public] gerrit   concepts and workflows[Public] gerrit   concepts and workflows
[Public] gerrit concepts and workflows
 
Configurable horizontal pod autoscaler
Configurable horizontal pod autoscalerConfigurable horizontal pod autoscaler
Configurable horizontal pod autoscaler
 
Derbycon - The Unintended Risks of Trusting Active Directory
Derbycon - The Unintended Risks of Trusting Active DirectoryDerbycon - The Unintended Risks of Trusting Active Directory
Derbycon - The Unintended Risks of Trusting Active Directory
 
Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!Kubernetes Application Deployment with Helm - A beginner Guide!
Kubernetes Application Deployment with Helm - A beginner Guide!
 
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, XilinxXPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, Xilinx
 
I2C Subsystem In Linux-2.6.24
I2C Subsystem In Linux-2.6.24I2C Subsystem In Linux-2.6.24
I2C Subsystem In Linux-2.6.24
 
(Ab)Using GPOs for Active Directory Pwnage
(Ab)Using GPOs for Active Directory Pwnage(Ab)Using GPOs for Active Directory Pwnage
(Ab)Using GPOs for Active Directory Pwnage
 
Docker swarm
Docker swarmDocker swarm
Docker swarm
 

Similar to Xen Cache Coloring: Interference-Free Real-Time System

Rendering in the Cloud
Rendering in the CloudRendering in the Cloud
Rendering in the Cloud
Benjamin Shrive
 
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
Edge AI and Vision Alliance
 
Cit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_houseCit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_house
Erica Price
 
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
KTN
 
Encoding at Scale for Live Video Streaming
Encoding at Scale for Live Video StreamingEncoding at Scale for Live Video Streaming
Encoding at Scale for Live Video Streaming
Ray Adensamer
 
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
Edge AI and Vision Alliance
 
VDI-POC
VDI-POCVDI-POC
VDI-POC
Robert Jones
 
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
HostedbyConfluent
 
Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...
Principled Technologies
 
Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)
Intel® Software
 
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Lee Bushen
 
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
Edge AI and Vision Alliance
 
Novosco: Citrix tips and best practices
Novosco: Citrix tips and best practicesNovosco: Citrix tips and best practices
Novosco: Citrix tips and best practices
Novosco
 
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
Edge AI and Vision Alliance
 
Citrix Portfolio Updates
Citrix Portfolio UpdatesCitrix Portfolio Updates
Citrix Portfolio Updates
Pawel Serwan
 
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
Edge AI and Vision Alliance
 
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
The Linux Foundation
 
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Codemotion
 
Dcp
DcpDcp
HDX 3D
HDX 3DHDX 3D

Similar to Xen Cache Coloring: Interference-Free Real-Time System (20)

Rendering in the Cloud
Rendering in the CloudRendering in the Cloud
Rendering in the Cloud
 
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
 
Cit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_houseCit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_house
 
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
 
Encoding at Scale for Live Video Streaming
Encoding at Scale for Live Video StreamingEncoding at Scale for Live Video Streaming
Encoding at Scale for Live Video Streaming
 
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
“Vitis and Vitis AI: Application Acceleration from Cloud to Edge,” a Presenta...
 
VDI-POC
VDI-POCVDI-POC
VDI-POC
 
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
Uncover the Root Cause of Kafka Performance Anomalies, Daniel Kim & Antón Rod...
 
Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...
 
Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)Introduction to Software Defined Visualization (SDVis)
Introduction to Software Defined Visualization (SDVis)
 
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
Citrix Desktop Master Class – New in XA/XD 7.9, Skype for Business, HDX Raspb...
 
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
“State-of-the-art Model Quantization and Optimization for Efficient Edge AI,”...
 
Novosco: Citrix tips and best practices
Novosco: Citrix tips and best practicesNovosco: Citrix tips and best practices
Novosco: Citrix tips and best practices
 
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
“Vision and AI DSPs for Ultra-High-End and Always-On Applications,” a Present...
 
Citrix Portfolio Updates
Citrix Portfolio UpdatesCitrix Portfolio Updates
Citrix Portfolio Updates
 
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
 
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
 
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
 
Dcp
DcpDcp
Dcp
 
HDX 3D
HDX 3DHDX 3D
HDX 3D
 

More from Stefano Stabellini

Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdf
Stefano Stabellini
 
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
Stefano Stabellini
 
ELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for EmbeddedELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for Embedded
Stefano Stabellini
 
RunX ELCE 2020
RunX ELCE 2020RunX ELCE 2020
RunX ELCE 2020
Stefano Stabellini
 
RunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edgeRunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edge
Stefano Stabellini
 
System Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperSystem Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and Lopper
Stefano Stabellini
 
Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)
Stefano Stabellini
 
Xen Project for ARM Servers
Xen Project for ARM ServersXen Project for ARM Servers
Xen Project for ARM Servers
Stefano Stabellini
 
Xen and OpenStack
Xen and OpenStackXen and OpenStack
Xen and OpenStack
Stefano Stabellini
 
XDS15: Project Raisin
XDS15: Project RaisinXDS15: Project Raisin
XDS15: Project Raisin
Stefano Stabellini
 
OpenStack and Xen
OpenStack and XenOpenStack and Xen
OpenStack and Xen
Stefano Stabellini
 

More from Stefano Stabellini (11)

Xen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdfXen Safety Embedded OSS Summit April 2024 v4.pdf
Xen Safety Embedded OSS Summit April 2024 v4.pdf
 
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
 
ELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for EmbeddedELC21: VM-to-VM Communication Mechanisms for Embedded
ELC21: VM-to-VM Communication Mechanisms for Embedded
 
RunX ELCE 2020
RunX ELCE 2020RunX ELCE 2020
RunX ELCE 2020
 
RunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edgeRunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edge
 
System Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperSystem Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and Lopper
 
Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)
 
Xen Project for ARM Servers
Xen Project for ARM ServersXen Project for ARM Servers
Xen Project for ARM Servers
 
Xen and OpenStack
Xen and OpenStackXen and OpenStack
Xen and OpenStack
 
XDS15: Project Raisin
XDS15: Project RaisinXDS15: Project Raisin
XDS15: Project Raisin
 
OpenStack and Xen
OpenStack and XenOpenStack and Xen
OpenStack and Xen
 

Recently uploaded

GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
Green Software Development
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
Green Software Development
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
sjcobrien
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
Quickdice ERP
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
Hornet Dynamics
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
Marcin Chrost
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
Peter Muessig
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
Rakesh Kumar R
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
dakas1
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
Rakesh Kumar R
 
fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.
AnkitaPandya11
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
Philip Schwarz
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
Alina Yurenko
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
mz5nrf0n
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
TaghreedAltamimi
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
XfilesPro
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
kalichargn70th171
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
Patrick Weigel
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
Rakesh Kumar R
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
dakas1
 

Recently uploaded (20)

GreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-JurisicGreenCode-A-VSCode-Plugin--Dario-Jurisic
GreenCode-A-VSCode-Plugin--Dario-Jurisic
 
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, FactsALGIT - Assembly Line for Green IT - Numbers, Data, Facts
ALGIT - Assembly Line for Green IT - Numbers, Data, Facts
 
Malibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed RoundMalibou Pitch Deck For Its €3M Seed Round
Malibou Pitch Deck For Its €3M Seed Round
 
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian CompaniesE-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
E-Invoicing Implementation: A Step-by-Step Guide for Saudi Arabian Companies
 
E-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet DynamicsE-commerce Development Services- Hornet Dynamics
E-commerce Development Services- Hornet Dynamics
 
Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !Enums On Steroids - let's look at sealed classes !
Enums On Steroids - let's look at sealed classes !
 
UI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design SystemUI5con 2024 - Bring Your Own Design System
UI5con 2024 - Bring Your Own Design System
 
How to write a program in any programming language
How to write a program in any programming languageHow to write a program in any programming language
How to write a program in any programming language
 
一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理一比一原版(USF毕业证)旧金山大学毕业证如何办理
一比一原版(USF毕业证)旧金山大学毕业证如何办理
 
What next after learning python programming basics
What next after learning python programming basicsWhat next after learning python programming basics
What next after learning python programming basics
 
fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.fiscal year variant fiscal year variant.
fiscal year variant fiscal year variant.
 
Hand Rolled Applicative User Validation Code Kata
Hand Rolled Applicative User ValidationCode KataHand Rolled Applicative User ValidationCode Kata
Hand Rolled Applicative User Validation Code Kata
 
All you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVMAll you need to know about Spring Boot and GraalVM
All you need to know about Spring Boot and GraalVM
 
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
原版定制美国纽约州立大学奥尔巴尼分校毕业证学位证书原版一模一样
 
Lecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptxLecture 2 - software testing SE 412.pptx
Lecture 2 - software testing SE 412.pptx
 
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
Everything You Need to Know About X-Sign: The eSign Functionality of XfilesPr...
 
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf8 Best Automated Android App Testing Tool and Framework in 2024.pdf
8 Best Automated Android App Testing Tool and Framework in 2024.pdf
 
WWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders AustinWWDC 2024 Keynote Review: For CocoaCoders Austin
WWDC 2024 Keynote Review: For CocoaCoders Austin
 
Fundamentals of Programming and Language Processors
Fundamentals of Programming and Language ProcessorsFundamentals of Programming and Language Processors
Fundamentals of Programming and Language Processors
 
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
一比一原版(UMN毕业证)明尼苏达大学毕业证如何办理
 

Xen Cache Coloring: Interference-Free Real-Time System

  • 1. © Copyright 2020 Xilinx Xen Cache Coloring & Real-Time Stefano Stabellini Marco Solieri Luca Miccio Giulio Corradi
  • 2. © Copyright 2020 Xilinx Xen in Embedded Consolidation and Componentization Isolation  Security  Reliability  Safety  Interference& Real-Time
  • 3. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU
  • 4. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU sched=null sched=credit vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU
  • 5. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU sched=null sched=credit vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU
  • 6. © Copyright 2020 Xilinx Xen Schedulers CPU CPU CPU CPU sched=null sched=rtds vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU vCPU
  • 7. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 8. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 9. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 10. © Copyright 2020 Xilinx Xen Cache Coloring CPUs clusters often share L2 cache Interference via L2 cache affects performance  App0 running on CPU0 can cause cache entries evictions, which affect App1 running on CPU1  App1 running on CPU1 could miss a deadline due to App0’s behavior  It can happen betweenApps running on the same OS & between VMs on the same hypervisor Hypervisor Solution: Cache Partitioning, AKA Cache Coloring  Each VM gets its own allocation of cache entries  No shared cache entries between VMs  Allows real-time apps to run with deterministic IRQ latency Core 1 Core 2 Core 3 Core 4 L2 D D R
  • 11. © Copyright 2020 Xilinx Colors Calculation Detect Way Size  via registers  via xen command line Calculate colors masks and total number of available colors  Xilinx ZynqMP: way size is 65536  Order 16 -> color bitmask 0xF000 -> 16 colors D D R
  • 12. © Copyright 2020 Xilinx Colored Allocation New colored memory allocator  In alternative to Buddy  Pages are stored by color in separate lists Can be used for Xen memory as well as Domain memory
  • 13. © Copyright 2020 Xilinx Coloring Configuration Xen command line Dom0-less DomUs xl domU config files way_size=65536 xen_colors=0 dom0_colors=1-6 fdt set /chosen/domU0 colors <0x0 0x80> colors=["10-11"]
  • 14. © Copyright 2020 Xilinx Benchmarking  Hardware: Xilinx MPSoC  Workload: Bare Metal application  Stress: Bare Metal application  Test configuration:  Measured load: 1500 sums  Internal load: 8000 sums every 250us  Interrupt triggered every 8ms  Samples:1000  Colored configuration:  Xen: ["0,1"]  Motor Control DomU: ["4,7"]  Stress DomU: ["8,11"]  Stress2 DomU:["12,15"]  Linux stress:["2,3"]  Linux stress command: stress -m 1 --vm-bytes 8M --vm-stride 64 --vm-keep -t 3600 &
  • 15. © Copyright 2020 Xilinx L1 vs L2 cache L1 cache can hide L2 cache interference effects in simple test scenarios Make sure not use the L1 cache by accident during the measurements
  • 16. © Copyright 2020 Xilinx Benchmark #1: Motor Control Execution Time Measure the time it takes to run a typical motor control execution routine, with and without interference
  • 17. © Copyright 2020 Xilinx Results No interference: results for reference Lower is better
  • 18. © Copyright 2020 Xilinx Results Adding interference: Xen Coloring has the best execution time
  • 19. © Copyright 2020 Xilinx Results Increasing interference: • Xen Coloring is stable • Other results worsen
  • 20. © Copyright 2020 Xilinx Results Increasing interference again: • Xen Coloring is stable • Other results worsen still
  • 21. © Copyright 2020 Xilinx Results Comparing Xen vs. Xen Colored Lower is better
  • 22. © Copyright 2020 Xilinx Benchmark #2: interrupt response time Measure the difference between the time when the interrupt service routine runs and the time the interrupt is set to fire, with and without interference Timer in Programmable Logic
  • 23. © Copyright 2020 Xilinx Results No interference: results for reference Lower is better
  • 24. © Copyright 2020 Xilinx Results Adding interference: • Xen without Coloring worsen • Xen Coloring is stable
  • 25. © Copyright 2020 Xilinx Results Adding more interference: • Xen without Coloring worsen again • Xen Coloring is stable
  • 26. © Copyright 2020 Xilinx Results Adding even more interference: • Xen without Coloring worsen still • Xen Coloring is stable
  • 27. © Copyright 2020 Xilinx Results All results together Xen Coloring provides stable IRQ latency even under severe levels of interference
  • 28. © Copyright 2020 Xilinx Conclusions Xen Cache Coloring offers  Much lower execution times under stress  Much lower IRQ latency under stress Greatly improves Determinism -- measurements have far lower variance Xen Cache Coloring effectively reduces the effects of cache interference Enable deployments of real-time and non-real-time workloads on a single SoC
  • 29. © Copyright 2020 Xilinx Status All patches online at:  https://github.com/Xilinx/xen.git xilinx/release-2020.1 TODO  Upstreaming  Linux Dom0 not 1:1 mapped, i.e. PV drivers support
  • 30. © Copyright 2020 Xilinx Xen Cache Coloring: Demo Xen [0] Baremetal Latency Measurements [7] Baremetal Memory Stress [10 11] Dom0 [1 2 3 4 5 6]
  • 31. © Copyright 2020 Xilinx Baremetal Latency Measurements
  • 32. © Copyright 2020 Xilinx Baremetal Memory Stress
  • 33. © Copyright 2020 Xilinx Thank You