SlideShare a Scribd company logo
1 of 31
Download to read offline
© Copyright 2020 Xilinx
Xen Cache Coloring &
Real-Time
Stefano Stabellini
Marco Solieri
Luca Miccio
Giulio Corradi
© Copyright 2020 Xilinx
Xen in Embedded
Consolidation and Componentization
Isolation
 Security
 Reliability
 Safety
 Interference& Real-Time
© Copyright 2020 Xilinx
Reference Example
Static Configuration – 2-4 VMs
No multitenancy – All VMs owned by the same user
Direct assignment of resources
No PV drivers
 Maybe except for PV Console
 A shared page to exchange data between VMs
Baremetal DomU driving a soft-logic block with latency requirements
© Copyright 2020 Xilinx
sched=null vwfi=native
CPU CPU
CPU CPU
sched=null
vCPU vCPU
vCPU vCPU
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Shared L2 cache
L2
Core
1
Core
2
Core
3
Core
4
DDR
L1 L1 L1 L1
© Copyright 2020 Xilinx
Xen Cache Coloring
CPUs clusters often share L2 cache
Interference via L2 cache affects performance
 App0 running on CPU0 can cause cache entries
evictions, which affect App1 running on CPU1
 App1 running on CPU1 could miss a deadline due to
App0’s behavior
 It can happen betweenApps running on the same OS &
between VMs on the same hypervisor
Hypervisor Solution: Cache Partitioning,
AKA Cache Coloring
 Each VM gets its own allocation of cache entries
 No shared cache entries between VMs
 Allows real-time apps to run with deterministic IRQ
latency
Core
1
Core
2
Core
3
Core
4
L2
D
D
R
© Copyright 2020 Xilinx
Colors Calculation
Detect Way Size
 via registers
 via xen command line
Calculate colors masks and total number of available colors
 Xilinx ZynqMP: way size is 65536
 Order 16 -> color bitmask 0xF000 -> 16 colors
D
D
R
© Copyright 2020 Xilinx
Colored Allocation
New colored memory allocator
 In alternative to Buddy
 Pages are stored by color in separate lists
Can be used for Xen memory as well as Domain memory
Xen relocates itself in colored memory
© Copyright 2020 Xilinx
Coloring Configuration
Xen command line
Dom0-less DomUs
xl domU config files
way_size=65536 xen_colors=0 dom0_colors=1-6
fdt set /chosen/domU0 colors <0x0 0x80>
colors=["10-11"]
© Copyright 2020 Xilinx
Benchmarking
 Hardware: Xilinx MPSoC
 Workload: Bare Metal application
 Stress: Bare Metal application
 Test configuration:
 Measured load: 1500 sums
 Internal load: 8000 sums every 250us
 Interrupt triggered every 8ms
 Samples:1000
 Colored configuration:
 Xen: ["0,1"]
 Motor Control DomU: ["4,7"]
 Stress DomU: ["8,11"]
 Stress2 DomU:["12,15"]
 Linux stress:["2,3"]
 Linux stress command: stress -m 1 --vm-bytes 8M --vm-stride 64 --vm-keep -t 3600 &
© Copyright 2020 Xilinx
L1 vs L2 cache
L1 cache can hide L2 cache interference effects in simple test scenarios
Make sure not use the L1 cache by accident during the measurements
© Copyright 2020 Xilinx
Benchmark #1: Motor Control Execution Time
Measure the time it takes to run a typical motor control execution routine, with
and without interference
© Copyright 2020 Xilinx
Results
No interference:
results for reference
Lower is better
© Copyright 2020 Xilinx
Results
Adding interference:
Xen Coloring
has the best execution time
© Copyright 2020 Xilinx
Results
Increasing interference:
• Xen Coloring is stable
• Other results worsen
© Copyright 2020 Xilinx
Results
Increasing interference
again:
• Xen Coloring is stable
• Other results worsen still
© Copyright 2020 Xilinx
Results
Comparing
Xen vs. Xen Colored
Lower is better
© Copyright 2020 Xilinx
Benchmark #2: interrupt response time
Measure the difference between the time when the interrupt service routine
runs and the time the interrupt is set to fire, with and without interference
Timer in Programmable Logic
© Copyright 2020 Xilinx
Results
No interference:
results for reference
Lower is better
© Copyright 2020 Xilinx
Results
Adding interference:
• Xen without Coloring
worsen
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
Adding more interference:
• Xen without Coloring
worsen again
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
Adding even more
interference:
• Xen without Coloring
worsen still
• Xen Coloring is stable
© Copyright 2020 Xilinx
Results
All results together
Xen Coloring provides
stable IRQ latency even
under severe levels of
interference
© Copyright 2020 Xilinx
Conclusions
Xen Cache Coloring offers
 Much lower execution times under stress
 Much lower IRQ latency under stress
Greatly improves Determinism -- measurements have far lower variance
Xen Cache Coloring effectively reduces the effects of cache interference
Enable deployments of real-time and non-real-time workloads on a single SoC
© Copyright 2020 Xilinx
Status
All patches online at:
 https://github.com/Xilinx/xen.git xilinx/release-2020.1
TODO
 Upstreaming
 Linux Dom0 not 1:1 mapped, i.e. PV drivers support
© Copyright 2020 Xilinx
Xen Cache Coloring: Demo
Xen
[0]
Baremetal
Latency
Measurements
[7]
Baremetal Memory
Stress
[10 11]
Dom0
[1 2 3 4 5 6]
© Copyright 2020 Xilinx
Baremetal Latency Measurements
© Copyright 2020 Xilinx
Baremetal Memory Stress
© Copyright 2020 Xilinx
Thank You

More Related Content

What's hot

Safety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen HypervisorSafety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen HypervisorStefano Stabellini
 
Rootlinux17: An introduction to Xen Project Virtualisation
Rootlinux17:  An introduction to Xen Project VirtualisationRootlinux17:  An introduction to Xen Project Virtualisation
Rootlinux17: An introduction to Xen Project VirtualisationThe Linux Foundation
 
Xen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systemsXen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systemsStefano Stabellini
 
Citrix XenDesktop and XenApp 7.5 Architecture Deployment
Citrix XenDesktop and XenApp 7.5 Architecture DeploymentCitrix XenDesktop and XenApp 7.5 Architecture Deployment
Citrix XenDesktop and XenApp 7.5 Architecture DeploymentHuy Pham
 
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022Stefano Stabellini
 
Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...
Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...
Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...Red Bend Software
 
What’s New in VMware vSphere 7?
What’s New in VMware vSphere 7?What’s New in VMware vSphere 7?
What’s New in VMware vSphere 7?Insight
 
Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)Stefano Stabellini
 
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...
OSSJP/ALS19:  The Road to Safety Certification: Overcoming Community Challeng...OSSJP/ALS19:  The Road to Safety Certification: Overcoming Community Challeng...
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...The Linux Foundation
 
In-depth Troubleshooting on NetScaler using Command Line Tools
In-depth Troubleshooting on NetScaler using Command Line ToolsIn-depth Troubleshooting on NetScaler using Command Line Tools
In-depth Troubleshooting on NetScaler using Command Line ToolsDavid McGeough
 
Presentation citrix desktop virtualization
Presentation   citrix desktop virtualizationPresentation   citrix desktop virtualization
Presentation citrix desktop virtualizationxKinAnx
 
Users and groups in Linux
Users and groups in LinuxUsers and groups in Linux
Users and groups in LinuxKnoldus Inc.
 
HCL Notes und Nomad Fehlerbehebung für Dummies
HCL Notes und Nomad Fehlerbehebung für DummiesHCL Notes und Nomad Fehlerbehebung für Dummies
HCL Notes und Nomad Fehlerbehebung für Dummiespanagenda
 
Citrix XenApp and XenDesktop 7.X
Citrix XenApp and XenDesktop 7.XCitrix XenApp and XenDesktop 7.X
Citrix XenApp and XenDesktop 7.XIzaak Salman
 
MemVerge: The Software Stack for CXL Environments
MemVerge: The Software Stack for CXL EnvironmentsMemVerge: The Software Stack for CXL Environments
MemVerge: The Software Stack for CXL EnvironmentsMemory Fabric Forum
 
Ibm spectrum scale fundamentals workshop for americas part 1 components archi...
Ibm spectrum scale fundamentals workshop for americas part 1 components archi...Ibm spectrum scale fundamentals workshop for americas part 1 components archi...
Ibm spectrum scale fundamentals workshop for americas part 1 components archi...xKinAnx
 
ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...
ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...
ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...The Linux Foundation
 

What's hot (20)

Safety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen HypervisorSafety-Certifying Open Source Software: The Case of the Xen Hypervisor
Safety-Certifying Open Source Software: The Case of the Xen Hypervisor
 
Rootlinux17: An introduction to Xen Project Virtualisation
Rootlinux17:  An introduction to Xen Project VirtualisationRootlinux17:  An introduction to Xen Project Virtualisation
Rootlinux17: An introduction to Xen Project Virtualisation
 
Xen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systemsXen on ARM for embedded and IoT: from secure containers to dom0less systems
Xen on ARM for embedded and IoT: from secure containers to dom0less systems
 
Citrix XenDesktop and XenApp 7.5 Architecture Deployment
Citrix XenDesktop and XenApp 7.5 Architecture DeploymentCitrix XenDesktop and XenApp 7.5 Architecture Deployment
Citrix XenDesktop and XenApp 7.5 Architecture Deployment
 
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022System Device Tree and Lopper: Concrete Examples - ELC NA 2022
System Device Tree and Lopper: Concrete Examples - ELC NA 2022
 
Xen Memory Management
Xen Memory ManagementXen Memory Management
Xen Memory Management
 
Xen Debugging
Xen DebuggingXen Debugging
Xen Debugging
 
Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...
Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...
Red Bend Software: Separation Using Type-1 Virtualization in Vehicles and Aut...
 
What’s New in VMware vSphere 7?
What’s New in VMware vSphere 7?What’s New in VMware vSphere 7?
What’s New in VMware vSphere 7?
 
Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)Xen and the art of embedded virtualization (ELC 2017)
Xen and the art of embedded virtualization (ELC 2017)
 
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...
OSSJP/ALS19:  The Road to Safety Certification: Overcoming Community Challeng...OSSJP/ALS19:  The Road to Safety Certification: Overcoming Community Challeng...
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...
 
In-depth Troubleshooting on NetScaler using Command Line Tools
In-depth Troubleshooting on NetScaler using Command Line ToolsIn-depth Troubleshooting on NetScaler using Command Line Tools
In-depth Troubleshooting on NetScaler using Command Line Tools
 
Presentation citrix desktop virtualization
Presentation   citrix desktop virtualizationPresentation   citrix desktop virtualization
Presentation citrix desktop virtualization
 
Users and groups in Linux
Users and groups in LinuxUsers and groups in Linux
Users and groups in Linux
 
HCL Notes und Nomad Fehlerbehebung für Dummies
HCL Notes und Nomad Fehlerbehebung für DummiesHCL Notes und Nomad Fehlerbehebung für Dummies
HCL Notes und Nomad Fehlerbehebung für Dummies
 
Citrix XenApp and XenDesktop 7.X
Citrix XenApp and XenDesktop 7.XCitrix XenApp and XenDesktop 7.X
Citrix XenApp and XenDesktop 7.X
 
MemVerge: The Software Stack for CXL Environments
MemVerge: The Software Stack for CXL EnvironmentsMemVerge: The Software Stack for CXL Environments
MemVerge: The Software Stack for CXL Environments
 
Ibm spectrum scale fundamentals workshop for americas part 1 components archi...
Ibm spectrum scale fundamentals workshop for americas part 1 components archi...Ibm spectrum scale fundamentals workshop for americas part 1 components archi...
Ibm spectrum scale fundamentals workshop for americas part 1 components archi...
 
ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...
ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...
ALSF13: Xen on ARM - Virtualization for the Automotive Industry - Stefano Sta...
 
Emc vplex deep dive
Emc vplex deep diveEmc vplex deep dive
Emc vplex deep dive
 

Similar to Xen Cache Coloring & Real-Time Performance

“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...Edge AI and Vision Alliance
 
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...The Linux Foundation
 
Citrix Cloud Master Class June 2014
Citrix Cloud Master Class June 2014Citrix Cloud Master Class June 2014
Citrix Cloud Master Class June 2014Citrix
 
Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...
Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...
Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...Ceph Community
 
Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...Principled Technologies
 
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from QualcommEdge AI and Vision Alliance
 
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...KTN
 
Citrix Portfolio Updates
Citrix Portfolio UpdatesCitrix Portfolio Updates
Citrix Portfolio UpdatesPawel Serwan
 
Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...
Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...
Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...{code} by Dell EMC
 
Cit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_houseCit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_houseErica Price
 
Webinar: How to Put an End to Hyperconverged Silos
Webinar: How to Put an End to Hyperconverged SilosWebinar: How to Put an End to Hyperconverged Silos
Webinar: How to Put an End to Hyperconverged SilosStorage Switzerland
 
PLNOG 9: Adam Obszyński - DNS Caching
PLNOG 9: Adam Obszyński - DNS Caching PLNOG 9: Adam Obszyński - DNS Caching
PLNOG 9: Adam Obszyński - DNS Caching PROIDEA
 
“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...
“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...
“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...Edge AI and Vision Alliance
 
Affordable trustworthy-systems
Affordable trustworthy-systemsAffordable trustworthy-systems
Affordable trustworthy-systemsmicrokerneldude
 
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...Codemotion
 
Software based video, audio, web conferencing - can standard servers deliver?
Software based video, audio, web conferencing - can standard servers deliver?Software based video, audio, web conferencing - can standard servers deliver?
Software based video, audio, web conferencing - can standard servers deliver?Anders Løkke
 
Citrix xen desktop by Naeem ali saifi from Ricoh
Citrix xen desktop by Naeem ali saifi from RicohCitrix xen desktop by Naeem ali saifi from Ricoh
Citrix xen desktop by Naeem ali saifi from Ricohctxnaeem
 
2015 02-10 xen server master class
2015 02-10 xen server master class2015 02-10 xen server master class
2015 02-10 xen server master classCitrix
 

Similar to Xen Cache Coloring & Real-Time Performance (20)

Rendering in the Cloud
Rendering in the CloudRendering in the Cloud
Rendering in the Cloud
 
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
“Trends in Neural Network Topologies for Vision at the Edge,” a Presentation ...
 
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
XDF18: Heterogeneous Real-Time SoC Software Architecture - Stefano Stabellini...
 
Citrix Cloud Master Class June 2014
Citrix Cloud Master Class June 2014Citrix Cloud Master Class June 2014
Citrix Cloud Master Class June 2014
 
Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...
Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...
Ceph Day Chicago - Deploying flash storage for Ceph without compromising perf...
 
Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...Boost performance and provide strong end-user unified communications experien...
Boost performance and provide strong end-user unified communications experien...
 
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm“Efficient Video Perception Through AI,” a Presentation from Qualcomm
“Efficient Video Perception Through AI,” a Presentation from Qualcomm
 
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
Fast, Scalable Quantized Neural Network Inference on FPGAs with FINN and Logi...
 
Citrix Portfolio Updates
Citrix Portfolio UpdatesCitrix Portfolio Updates
Citrix Portfolio Updates
 
Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...
Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...
Deep-Dive on Container Networking Architectures - Frans van Rooyen - Dell EMC...
 
Cit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_houseCit bip v3_7.17_ally_house
Cit bip v3_7.17_ally_house
 
Webinar: How to Put an End to Hyperconverged Silos
Webinar: How to Put an End to Hyperconverged SilosWebinar: How to Put an End to Hyperconverged Silos
Webinar: How to Put an End to Hyperconverged Silos
 
PLNOG 9: Adam Obszyński - DNS Caching
PLNOG 9: Adam Obszyński - DNS Caching PLNOG 9: Adam Obszyński - DNS Caching
PLNOG 9: Adam Obszyński - DNS Caching
 
“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...
“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...
“Khronos Group Standards: Powering the Future of Embedded Vision,” a Presenta...
 
VDI-POC
VDI-POCVDI-POC
VDI-POC
 
Affordable trustworthy-systems
Affordable trustworthy-systemsAffordable trustworthy-systems
Affordable trustworthy-systems
 
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
Luca Relandini - Microservices and containers networking: Contiv, deep dive a...
 
Software based video, audio, web conferencing - can standard servers deliver?
Software based video, audio, web conferencing - can standard servers deliver?Software based video, audio, web conferencing - can standard servers deliver?
Software based video, audio, web conferencing - can standard servers deliver?
 
Citrix xen desktop by Naeem ali saifi from Ricoh
Citrix xen desktop by Naeem ali saifi from RicohCitrix xen desktop by Naeem ali saifi from Ricoh
Citrix xen desktop by Naeem ali saifi from Ricoh
 
2015 02-10 xen server master class
2015 02-10 xen server master class2015 02-10 xen server master class
2015 02-10 xen server master class
 

More from Stefano Stabellini

More from Stefano Stabellini (8)

RunX ELCE 2020
RunX ELCE 2020RunX ELCE 2020
RunX ELCE 2020
 
RunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edgeRunX: deploy real-time OSes as containers at the edge
RunX: deploy real-time OSes as containers at the edge
 
System Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and LopperSystem Device Tree update: Bus Firewalls and Lopper
System Device Tree update: Bus Firewalls and Lopper
 
Dom0less - Xen Developer Summit 2019
Dom0less  - Xen Developer Summit 2019Dom0less  - Xen Developer Summit 2019
Dom0less - Xen Developer Summit 2019
 
Xen Project for ARM Servers
Xen Project for ARM ServersXen Project for ARM Servers
Xen Project for ARM Servers
 
Xen and OpenStack
Xen and OpenStackXen and OpenStack
Xen and OpenStack
 
XDS15: Project Raisin
XDS15: Project RaisinXDS15: Project Raisin
XDS15: Project Raisin
 
OpenStack and Xen
OpenStack and XenOpenStack and Xen
OpenStack and Xen
 

Recently uploaded

Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxSasikiranMarri
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slidesvaideheekore1
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...Bert Jan Schrijver
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolsosttopstonverter
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdfSteve Caron
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfmaor17
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shardsChristopher Curtin
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonApplitools
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jNeo4j
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogueitservices996
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...kalichargn70th171
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITmanoharjgpsolutions
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecturerahul_net
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxRTS corp
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfRTS corp
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldRoberto Pérez Alcolea
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfkalichargn70th171
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...OnePlan Solutions
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdfAndrey Devyatkin
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesKrzysztofKkol1
 

Recently uploaded (20)

Understanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptxUnderstanding Plagiarism: Causes, Consequences and Prevention.pptx
Understanding Plagiarism: Causes, Consequences and Prevention.pptx
 
Introduction to Firebase Workshop Slides
Introduction to Firebase Workshop SlidesIntroduction to Firebase Workshop Slides
Introduction to Firebase Workshop Slides
 
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
JavaLand 2024 - Going serverless with Quarkus GraalVM native images and AWS L...
 
eSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration toolseSoftTools IMAP Backup Software and migration tools
eSoftTools IMAP Backup Software and migration tools
 
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
[ CNCF Q1 2024 ] Intro to Continuous Profiling and Grafana Pyroscope.pdf
 
Zer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdfZer0con 2024 final share short version.pdf
Zer0con 2024 final share short version.pdf
 
2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards2024 DevNexus Patterns for Resiliency: Shuffle shards
2024 DevNexus Patterns for Resiliency: Shuffle shards
 
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + KobitonLeveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
Leveraging AI for Mobile App Testing on Real Devices | Applitools + Kobiton
 
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4jGraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
GraphSummit Madrid - Product Vision and Roadmap - Luis Salvador Neo4j
 
Ronisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited CatalogueRonisha Informatics Private Limited Catalogue
Ronisha Informatics Private Limited Catalogue
 
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
The Ultimate Guide to Performance Testing in Low-Code, No-Code Environments (...
 
Best Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh ITBest Angular 17 Classroom & Online training - Naresh IT
Best Angular 17 Classroom & Online training - Naresh IT
 
Understanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM ArchitectureUnderstanding Flamingo - DeepMind's VLM Architecture
Understanding Flamingo - DeepMind's VLM Architecture
 
Advantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptxAdvantages of Cargo Cloud Solutions.pptx
Advantages of Cargo Cloud Solutions.pptx
 
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdfEnhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
Enhancing Supply Chain Visibility with Cargo Cloud Solutions.pdf
 
Keeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository worldKeeping your build tool updated in a multi repository world
Keeping your build tool updated in a multi repository world
 
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdfPros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
Pros and Cons of Selenium In Automation Testing_ A Comprehensive Assessment.pdf
 
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
Revolutionizing the Digital Transformation Office - Leveraging OnePlan’s AI a...
 
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
2024-04-09 - From Complexity to Clarity - AWS Summit AMS.pdf
 
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilitiesAmazon Bedrock in Action - presentation of the Bedrock's capabilities
Amazon Bedrock in Action - presentation of the Bedrock's capabilities
 

Xen Cache Coloring & Real-Time Performance

  • 1. © Copyright 2020 Xilinx Xen Cache Coloring & Real-Time Stefano Stabellini Marco Solieri Luca Miccio Giulio Corradi
  • 2. © Copyright 2020 Xilinx Xen in Embedded Consolidation and Componentization Isolation  Security  Reliability  Safety  Interference& Real-Time
  • 3. © Copyright 2020 Xilinx Reference Example Static Configuration – 2-4 VMs No multitenancy – All VMs owned by the same user Direct assignment of resources No PV drivers  Maybe except for PV Console  A shared page to exchange data between VMs Baremetal DomU driving a soft-logic block with latency requirements
  • 4. © Copyright 2020 Xilinx sched=null vwfi=native CPU CPU CPU CPU sched=null vCPU vCPU vCPU vCPU
  • 5. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 6. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 7. © Copyright 2020 Xilinx Shared L2 cache L2 Core 1 Core 2 Core 3 Core 4 DDR L1 L1 L1 L1
  • 8. © Copyright 2020 Xilinx Xen Cache Coloring CPUs clusters often share L2 cache Interference via L2 cache affects performance  App0 running on CPU0 can cause cache entries evictions, which affect App1 running on CPU1  App1 running on CPU1 could miss a deadline due to App0’s behavior  It can happen betweenApps running on the same OS & between VMs on the same hypervisor Hypervisor Solution: Cache Partitioning, AKA Cache Coloring  Each VM gets its own allocation of cache entries  No shared cache entries between VMs  Allows real-time apps to run with deterministic IRQ latency Core 1 Core 2 Core 3 Core 4 L2 D D R
  • 9. © Copyright 2020 Xilinx Colors Calculation Detect Way Size  via registers  via xen command line Calculate colors masks and total number of available colors  Xilinx ZynqMP: way size is 65536  Order 16 -> color bitmask 0xF000 -> 16 colors D D R
  • 10. © Copyright 2020 Xilinx Colored Allocation New colored memory allocator  In alternative to Buddy  Pages are stored by color in separate lists Can be used for Xen memory as well as Domain memory Xen relocates itself in colored memory
  • 11. © Copyright 2020 Xilinx Coloring Configuration Xen command line Dom0-less DomUs xl domU config files way_size=65536 xen_colors=0 dom0_colors=1-6 fdt set /chosen/domU0 colors <0x0 0x80> colors=["10-11"]
  • 12. © Copyright 2020 Xilinx Benchmarking  Hardware: Xilinx MPSoC  Workload: Bare Metal application  Stress: Bare Metal application  Test configuration:  Measured load: 1500 sums  Internal load: 8000 sums every 250us  Interrupt triggered every 8ms  Samples:1000  Colored configuration:  Xen: ["0,1"]  Motor Control DomU: ["4,7"]  Stress DomU: ["8,11"]  Stress2 DomU:["12,15"]  Linux stress:["2,3"]  Linux stress command: stress -m 1 --vm-bytes 8M --vm-stride 64 --vm-keep -t 3600 &
  • 13. © Copyright 2020 Xilinx L1 vs L2 cache L1 cache can hide L2 cache interference effects in simple test scenarios Make sure not use the L1 cache by accident during the measurements
  • 14. © Copyright 2020 Xilinx Benchmark #1: Motor Control Execution Time Measure the time it takes to run a typical motor control execution routine, with and without interference
  • 15. © Copyright 2020 Xilinx Results No interference: results for reference Lower is better
  • 16. © Copyright 2020 Xilinx Results Adding interference: Xen Coloring has the best execution time
  • 17. © Copyright 2020 Xilinx Results Increasing interference: • Xen Coloring is stable • Other results worsen
  • 18. © Copyright 2020 Xilinx Results Increasing interference again: • Xen Coloring is stable • Other results worsen still
  • 19. © Copyright 2020 Xilinx Results Comparing Xen vs. Xen Colored Lower is better
  • 20. © Copyright 2020 Xilinx Benchmark #2: interrupt response time Measure the difference between the time when the interrupt service routine runs and the time the interrupt is set to fire, with and without interference Timer in Programmable Logic
  • 21. © Copyright 2020 Xilinx Results No interference: results for reference Lower is better
  • 22. © Copyright 2020 Xilinx Results Adding interference: • Xen without Coloring worsen • Xen Coloring is stable
  • 23. © Copyright 2020 Xilinx Results Adding more interference: • Xen without Coloring worsen again • Xen Coloring is stable
  • 24. © Copyright 2020 Xilinx Results Adding even more interference: • Xen without Coloring worsen still • Xen Coloring is stable
  • 25. © Copyright 2020 Xilinx Results All results together Xen Coloring provides stable IRQ latency even under severe levels of interference
  • 26. © Copyright 2020 Xilinx Conclusions Xen Cache Coloring offers  Much lower execution times under stress  Much lower IRQ latency under stress Greatly improves Determinism -- measurements have far lower variance Xen Cache Coloring effectively reduces the effects of cache interference Enable deployments of real-time and non-real-time workloads on a single SoC
  • 27. © Copyright 2020 Xilinx Status All patches online at:  https://github.com/Xilinx/xen.git xilinx/release-2020.1 TODO  Upstreaming  Linux Dom0 not 1:1 mapped, i.e. PV drivers support
  • 28. © Copyright 2020 Xilinx Xen Cache Coloring: Demo Xen [0] Baremetal Latency Measurements [7] Baremetal Memory Stress [10 11] Dom0 [1 2 3 4 5 6]
  • 29. © Copyright 2020 Xilinx Baremetal Latency Measurements
  • 30. © Copyright 2020 Xilinx Baremetal Memory Stress
  • 31. © Copyright 2020 Xilinx Thank You