The needs for immediate responsiveness of VMs in the virtualized environments have been on the rise. Several services in SKT also require soft realtime support for virtual machines to substitute the physical machines to achieve high utilization and adaptability. However, consolidated multiple OSes and irregular external events might render the hypervisor infringe on a VM's promptitude. As a solution of this problem, we are improving Xen's credit scheduler by introducing the RT_PRIORITY that guarantees a VM's running at any given point in time as long as credits remains to be burn. It would increase the quality of service and make a VM's behavior predictable on the consolidated environment. In addition, we extend our suggestion to the multi-core environment and even a large number of physical machines by using live migrations.
Excessive interrupts can hurt I/O scalability in Xen. The proposals discuss software interrupt throttling and interrupt-less NAPI to reduce interrupt overhead. They also discuss exposing NUMA information to Xen to improve host I/O NUMA awareness and enabling guest I/O NUMA awareness by constructing _PXM methods and extending device assignment policies.
This document proposes a method for link virtualization on the Xen virtualization platform using Single Root I/O Virtualization (SR-IOV). It discusses using SR-IOV to minimize overhead by performing encapsulation/decapsulation and packet filtering in hardware. It also describes using MAC-in-UDP tunneling with a virtual network ID to isolate networks and a vARP protocol to map between virtual and physical MAC addresses. The document evaluates the proposed method's ability to guarantee bandwidth isolation and provides performance results for both weight-based and bandwidth-based bandwidth control approaches.
Xen has been very successful on servers, and yet there are substantial areas where Xen can evolve further. In this talk Jun will discuss a compelling area where the Xen technologies can be applied to -- Mobile virtualization. Using Android as an example, the talk will explore two types of usage models, 1) Android as a guest, 2) Android as the host, showing the benefits of using the Xen technologies.
Traditionally Linux has always run on Xen either as a pure PV guest or as a virtualization unaware guest in an HVM domain. Recently, under the name of "PV on HVM", a series of works has been done to make Linux aware that is running on Xen and enable as many PV interfaces as possible even when running in an HVM container. After enabling the basic PV network and disk drivers, some other more interesting optimizations were implemented: in particular remapping legacy interrupts and MSIs onto event channels. This talk will explain the idea behind the feature, the reason why avoiding interactions with the lapic is a good, and some implementation details.
This document discusses moving backend drivers from the Dom0 domain to a separate HVM driver domain in Xen. Testing showed the HVM driver domain provided better network performance than the PV backend domain, with lower CPU utilization. Issues were discussed around booting the system without physical device drivers in Dom0, requiring the HVM driver domain to run devices and provide networking/storage. Further analysis of EPT page flipping performance was suggested.
ARM Architecture-based System Virtualization: Xen ARM open source software pr...The Linux Foundation
Xen ARM provides virtualization capabilities for ARM-based systems. It allows multiple guest operating systems to run concurrently on a single system. Xen ARM uses a lightweight hypervisor that provides CPU, memory, and I/O virtualization. Performance tests show that Xen ARM incurs minimal overhead compared to native systems. Xen ARM also enables security and real-time applications by isolating guest operating systems and bounding interrupt latency.
This document summarizes Russell Pavlicek's presentation on the bare-metal hypervisor as a platform for innovation. Some key innovations enabled by the bare-metal hypervisor discussed include Xen Automotive for developing embedded automotive systems, real-time virtualization support, an ARM-based hypervisor for new applications on ARM architecture, and unikernel systems that create highly secure and efficient cloud applications. A bare-metal hypervisor provides advantages like density, scalability, security and custom scheduling that facilitate these innovations.
Advanced performance troubleshooting using esxtopAlan Renouf
This document discusses using esxtop and resxtop tools to troubleshoot performance issues on VMware ESXi hosts. It provides 10 key things to know about esxtop counters and how they work. It then gives examples of using esxtop to troubleshoot common problems like CPU contention, memory issues, network throughput problems, and disk I/O latency. It also lists some other diagnostic tools that can be used along with esxtop.
Excessive interrupts can hurt I/O scalability in Xen. The proposals discuss software interrupt throttling and interrupt-less NAPI to reduce interrupt overhead. They also discuss exposing NUMA information to Xen to improve host I/O NUMA awareness and enabling guest I/O NUMA awareness by constructing _PXM methods and extending device assignment policies.
This document proposes a method for link virtualization on the Xen virtualization platform using Single Root I/O Virtualization (SR-IOV). It discusses using SR-IOV to minimize overhead by performing encapsulation/decapsulation and packet filtering in hardware. It also describes using MAC-in-UDP tunneling with a virtual network ID to isolate networks and a vARP protocol to map between virtual and physical MAC addresses. The document evaluates the proposed method's ability to guarantee bandwidth isolation and provides performance results for both weight-based and bandwidth-based bandwidth control approaches.
Xen has been very successful on servers, and yet there are substantial areas where Xen can evolve further. In this talk Jun will discuss a compelling area where the Xen technologies can be applied to -- Mobile virtualization. Using Android as an example, the talk will explore two types of usage models, 1) Android as a guest, 2) Android as the host, showing the benefits of using the Xen technologies.
Traditionally Linux has always run on Xen either as a pure PV guest or as a virtualization unaware guest in an HVM domain. Recently, under the name of "PV on HVM", a series of works has been done to make Linux aware that is running on Xen and enable as many PV interfaces as possible even when running in an HVM container. After enabling the basic PV network and disk drivers, some other more interesting optimizations were implemented: in particular remapping legacy interrupts and MSIs onto event channels. This talk will explain the idea behind the feature, the reason why avoiding interactions with the lapic is a good, and some implementation details.
This document discusses moving backend drivers from the Dom0 domain to a separate HVM driver domain in Xen. Testing showed the HVM driver domain provided better network performance than the PV backend domain, with lower CPU utilization. Issues were discussed around booting the system without physical device drivers in Dom0, requiring the HVM driver domain to run devices and provide networking/storage. Further analysis of EPT page flipping performance was suggested.
ARM Architecture-based System Virtualization: Xen ARM open source software pr...The Linux Foundation
Xen ARM provides virtualization capabilities for ARM-based systems. It allows multiple guest operating systems to run concurrently on a single system. Xen ARM uses a lightweight hypervisor that provides CPU, memory, and I/O virtualization. Performance tests show that Xen ARM incurs minimal overhead compared to native systems. Xen ARM also enables security and real-time applications by isolating guest operating systems and bounding interrupt latency.
This document summarizes Russell Pavlicek's presentation on the bare-metal hypervisor as a platform for innovation. Some key innovations enabled by the bare-metal hypervisor discussed include Xen Automotive for developing embedded automotive systems, real-time virtualization support, an ARM-based hypervisor for new applications on ARM architecture, and unikernel systems that create highly secure and efficient cloud applications. A bare-metal hypervisor provides advantages like density, scalability, security and custom scheduling that facilitate these innovations.
Advanced performance troubleshooting using esxtopAlan Renouf
This document discusses using esxtop and resxtop tools to troubleshoot performance issues on VMware ESXi hosts. It provides 10 key things to know about esxtop counters and how they work. It then gives examples of using esxtop to troubleshoot common problems like CPU contention, memory issues, network throughput problems, and disk I/O latency. It also lists some other diagnostic tools that can be used along with esxtop.
The document summarizes Xen, an open source hypervisor, and its approach to virtualizing I/O. Xen uses a privileged "dom0" domain to control hardware access and export virtualized devices to other unprivileged domains. It implements I/O memory management through software techniques like grant tables and swiotlb, as well as emerging hardware support from AMD and Intel. Overall, Xen provides secure isolation of guest VMs while enabling high-performance shared access to physical hardware resources.
Xen can run on ARM hardware by taking advantage of hardware virtualization extensions. It uses a single guest type that leverages para-virtualized interfaces for I/O without QEMU. The hypervisor code size is small at around 200,000 lines of code. Xen and Linux are bootable on ARMv7 hardware, and work is ongoing to support 64-bit ARMv8 guests. Challenges include cache coherency and interrupt handling, but the project aims to have full ARMv7 and increasing ARMv8 support in upcoming Xen releases.
Hypervisors are becoming more and more widespread in embedded environments, from automotive to medical and avionics. Their use case is different from traditional server and desktop virtualization, and so are their requirements. This talk will explain why hypervisors are used in embedded, and the unique challenges posed by these environments to virtualization technologies.
Xen, a popular open source hypervisor, was born to virtualize x86 Linux systems for the data center. It is now the leading open source hypervisor for ARM embedded platforms. The presentation will show how the ARM port of Xen differs from its x86 counterpart. It will go through the fundamental design decisions that made Xen a good choice for ARM embedded virtualization. The talk will explain the implementation of key features such as device assignment and interrupt virtualization.
This document discusses using paravirtualized devices in Linux guests running on Xen in HVM (hardware virtual machine) mode. It offers the benefits of HVM like installing the same as native while also providing access to fast paravirtualized devices. Initial support added xenbus and grant table support while later enhancements added ballooning, spinlocks, and interrupt/MSI remapping. Benchmarks show PV on HVM performs close to PV guests for PV-favored workloads and far ahead for nested paging workloads. PV on HVM provides a spectrum of performance between HVM and PV guests.
- Xen can now run on ARM hardware thanks to its rearchitecting to exploit ARM hardware virtualization extensions and remove unnecessary code like QEMU and shadow pagetables.
- It supports booting Linux as the dom0 and domUs. PV interfaces are used for I/O and there is no need for multiple guest types.
- Current status supports booting on ARMv7 hardware and some features on ARMv8 64-bit. Future work includes more platform support, ACPI, and enabling full ARMv8 virtualization.
Sang-bum Suh will give a talk on the current status and the future direction of Xen ARM. Xen ARM is the first ARM virtualization S/W based on Xen Architecture.
This document discusses the history and development of the Xen hypervisor project. It provides an overview of how paravirtualization and hardware-assisted virtualization have improved performance. It also examines how virtualization benefits security through policy enforcement and workload isolation. Network and memory management virtualization techniques are described that improve performance for virtual machines.
XPDS13: HVM Dom0 - Any unmodified OS as Dom0 - Will Auld, IntelThe Linux Foundation
This document discusses using an unmodified operating system as a domain 0 (dom0) in a hypervisor-virtualized (HVM) environment rather than the traditional para-virtualized (PV) dom0. It outlines the limitations of PV dom0 and how new CPU virtualization technologies enable using HVM dom0 to support any unmodified guest OS, improve performance, and reduce maintenance efforts. The document calls for porting Xen userland tools to Windows and enabling PV drivers to support Windows as an HVM dom0. This would expand Xen's usage models to include new client and trusted execution scenarios.
This document discusses the status and progress of Xen RAS (reliability, availability, and serviceability). It provides an overview of Xen RAS and its goals of containing errors within virtual machines. It then details recent work on core error recovery, APEI support, and robustness enhancements. It calls for continued collaboration on supporting I/O errors through PCIe AER integration.
kexec / kdump implementation in Linux Kernel and Xen hypervisorThe Linux Foundation
kexec and kdump allow restarting the Linux kernel without rebooting into the BIOS. kexec loads a new kernel into memory, while kdump facilitates debugging of crashed systems by loading a dump capture kernel. These features were introduced in Linux 2.6.13 and are currently supported on several platforms. Development is ongoing to support kexec/kdump for paravirtualized Xen guests. Future plans include enabling crash recovery across Xen and guest domains.
Linux Foundation Collaboration Summit 13 :10 years of Xen and BeyondThe Linux Foundation
In 2013, the Xen Hypervisor will be 10 years old: when Xen was designed, we anticipated a world, which now is known as cloud computing. Today, Xen powers the largest clouds in production and is the basis for several commercial virtualization products. In this talk we will give on overview of Xen and related projects, cover hot developments in the Xen community and outline what comes next.
The talk is intended for users and developers that are familiar with virtualization: no deep knowledge is required. We will start with an architectural overview and cover topics such as: Xen and Linux, how to secure your cloud using disaggregation, SELinux and XSM/FLASK, the evolution of Paravirtualization, Xen on ARM and common challenges for open source hypervisors. We will explore the potential of Open Mirage for testing hypervisors. The talk will conclude with an outlook to the future of Xen.
Introduction to Virtualization, Virsh and Virt-Managerwalkerchang
Virtualization allows for the abstraction and sharing of computer hardware resources like CPU, memory, storage and network capacity. The document introduces virtualization concepts and the tools KVM, Virsh and Virt-manager. It provides documentation on Virsh commands to manage domains (VMs), interfaces and networks. These include commands to define, start, suspend, resume VMs and interfaces, as well as take and restore VM snapshots to revert states. Managing VMs, interfaces and networks with Virsh commands allows administrators to efficiently share hardware resources across VMs.
The 4.5 release no a minor "point" update: it is one of the most feature-rich releases in the project's history. It contains several important additions. Most notably, new Xen PVH virtualization mode now supports running as dom0, enhanced support for Remus, significant ARM architecture updates, security improvements, real-time scheduling, support for Intel Cache Monitoring Technology (CMT), as well as improvements for automotive and embedded use-cases. Other enhancements include additional support for FreeBSD, systemd support, additional libvirt support, the release of Mirage OS 2.0, and more.
Besides giving an overview of Xen 4.5, we will explain the project's roadmap process and share what's ahead for 2015: such as improved OpenStack integration and hotpatching (applying security fixes without the need to reboot).
This talk provides an overview of the Xen Project eco-system and its main use-cases in a number of important market segments: it covers server virtualization, cloud computing and embedded, automotive and related. Lars Kurth highlights why the Xen Project is relevant in these market segments: he provides an overview of the Xen Project's architecture, relevant existing functionality and ongoing and planned developments. To complement the picture, he covers open-source projects that are related to Xen and are of interest for these use-cases. Excellent Software security is key to all of these use-cases. Thus, Lars specifically covers the Xen Project's security features, track record and touches on the project's security practices. He concludes with a few resources that help you get started with the Xen Project and highlight Internship Programs which the project supports.
The talk was delivered at Root Linux Conference 2017. Learn more: http://linux.globallogic.com/materials. The video is available at https://www.youtube.com/watch?v=sjQnAIJji4k
ATG needed to virtualize over 2,000 remote betting terminals to allow for hardware independence and an evolutionary transition from older OS/2 systems to newer Linux systems. They implemented the open source Xen hypervisor which allowed them to run existing OS/2 applications virtually on new Linux machines, transition applications to a new Java platform gradually, and gain more flexibility and influence over the hypervisor development. This provided a cost effective solution to modernize systems while maintaining existing functionality during the transition.
This document discusses the challenges of graphics virtualization. It provides background on native device initialization, QEMU I/O virtualization, and PCI device pass-through. It then covers graphics pass-through for discrete and integrated graphics, including the current status and future work, such as supporting dual graphics devices and improving driver validation.
This is a level 200 - 300 presentation.
It assumes:
Good understanding of vCenter 4, ESX 4, ESXi 4.
Preferably hands-on
We will only cover the delta between 4.1 and 4.0
Overview understanding of related products like VUM, Data Recovery, SRM, View, Nexus, Chargeback, CapacityIQ, vShieldZones, etc
Good understanding of related storage, server, network technology
Target audience
VMware Specialist: SE + Delivery from partners
The document discusses a framework for creating virtual machine monitors (VMMs) using hardware virtualization on x86 processors. It reviews x86 virtualization methods and Intel VT/AMD SVM extensions. The framework abstracts the complexities of directly using virtualization instructions, providing an easier API to develop type-II VMMs as Windows device drivers. It supports features like SMP, error reporting, and a plugin architecture. The goal is to simplify the creation of hypervisors for research and application development.
Presentation power vm virtualization without limitssolarisyougood
This document discusses IBM PowerVM virtualization capabilities for IBM Power Systems. PowerVM allows for virtualization of workloads through logical partitions (LPARs) and virtual machines (VMs). It provides capabilities like rapid provisioning, scalability, recoverability, and workload consolidation to improve efficiency and reduce costs. PowerVM editions differ in features available like the number of concurrent VMs, types of virtual I/O supported, and advanced functions. The document also discusses the Virtual I/O Server (VIOS) appliance, virtual storage and networking options in PowerVM like virtual SCSI, NPIV, and shared Ethernet adapters.
In this talk, John will explore the technology and architecture introduced in the ARM Cortex-A15 processor in support of virtualization. This is the first of multiple processors from ARM that will support true virtualization, and the ability to host existing operating systems binaries without modification. The hardware extensions were defined following careful analysis to address the key virtualization performance limitations of today's solutions while bringing new technologies to the device to better support a virtualized system.
Skylark aims to provide a generic platform which enables cloud applications to be easily deployed and accessed.
In the front-end, Skylark ports Spice to Xen and customizes Spice for application window delivery and display. Skylark users can start any application subscribed from Skylark application store and does not need to know whether it runs in Linux or Windows.
In the back-end, by providing Xen Image File System (Xen-IFS), multiple VMs are allowed to start up simultaneously from the same snapshot or the same disk image. Xen-IFS will automatically keep and isolate the private data produced by each VM.
Skylark makes it possible for Multiple User Single Installation (MUSI) among VMs, and also makes it easy for application delivery through remote VM access.
The document summarizes Xen, an open source hypervisor, and its approach to virtualizing I/O. Xen uses a privileged "dom0" domain to control hardware access and export virtualized devices to other unprivileged domains. It implements I/O memory management through software techniques like grant tables and swiotlb, as well as emerging hardware support from AMD and Intel. Overall, Xen provides secure isolation of guest VMs while enabling high-performance shared access to physical hardware resources.
Xen can run on ARM hardware by taking advantage of hardware virtualization extensions. It uses a single guest type that leverages para-virtualized interfaces for I/O without QEMU. The hypervisor code size is small at around 200,000 lines of code. Xen and Linux are bootable on ARMv7 hardware, and work is ongoing to support 64-bit ARMv8 guests. Challenges include cache coherency and interrupt handling, but the project aims to have full ARMv7 and increasing ARMv8 support in upcoming Xen releases.
Hypervisors are becoming more and more widespread in embedded environments, from automotive to medical and avionics. Their use case is different from traditional server and desktop virtualization, and so are their requirements. This talk will explain why hypervisors are used in embedded, and the unique challenges posed by these environments to virtualization technologies.
Xen, a popular open source hypervisor, was born to virtualize x86 Linux systems for the data center. It is now the leading open source hypervisor for ARM embedded platforms. The presentation will show how the ARM port of Xen differs from its x86 counterpart. It will go through the fundamental design decisions that made Xen a good choice for ARM embedded virtualization. The talk will explain the implementation of key features such as device assignment and interrupt virtualization.
This document discusses using paravirtualized devices in Linux guests running on Xen in HVM (hardware virtual machine) mode. It offers the benefits of HVM like installing the same as native while also providing access to fast paravirtualized devices. Initial support added xenbus and grant table support while later enhancements added ballooning, spinlocks, and interrupt/MSI remapping. Benchmarks show PV on HVM performs close to PV guests for PV-favored workloads and far ahead for nested paging workloads. PV on HVM provides a spectrum of performance between HVM and PV guests.
- Xen can now run on ARM hardware thanks to its rearchitecting to exploit ARM hardware virtualization extensions and remove unnecessary code like QEMU and shadow pagetables.
- It supports booting Linux as the dom0 and domUs. PV interfaces are used for I/O and there is no need for multiple guest types.
- Current status supports booting on ARMv7 hardware and some features on ARMv8 64-bit. Future work includes more platform support, ACPI, and enabling full ARMv8 virtualization.
Sang-bum Suh will give a talk on the current status and the future direction of Xen ARM. Xen ARM is the first ARM virtualization S/W based on Xen Architecture.
This document discusses the history and development of the Xen hypervisor project. It provides an overview of how paravirtualization and hardware-assisted virtualization have improved performance. It also examines how virtualization benefits security through policy enforcement and workload isolation. Network and memory management virtualization techniques are described that improve performance for virtual machines.
XPDS13: HVM Dom0 - Any unmodified OS as Dom0 - Will Auld, IntelThe Linux Foundation
This document discusses using an unmodified operating system as a domain 0 (dom0) in a hypervisor-virtualized (HVM) environment rather than the traditional para-virtualized (PV) dom0. It outlines the limitations of PV dom0 and how new CPU virtualization technologies enable using HVM dom0 to support any unmodified guest OS, improve performance, and reduce maintenance efforts. The document calls for porting Xen userland tools to Windows and enabling PV drivers to support Windows as an HVM dom0. This would expand Xen's usage models to include new client and trusted execution scenarios.
This document discusses the status and progress of Xen RAS (reliability, availability, and serviceability). It provides an overview of Xen RAS and its goals of containing errors within virtual machines. It then details recent work on core error recovery, APEI support, and robustness enhancements. It calls for continued collaboration on supporting I/O errors through PCIe AER integration.
kexec / kdump implementation in Linux Kernel and Xen hypervisorThe Linux Foundation
kexec and kdump allow restarting the Linux kernel without rebooting into the BIOS. kexec loads a new kernel into memory, while kdump facilitates debugging of crashed systems by loading a dump capture kernel. These features were introduced in Linux 2.6.13 and are currently supported on several platforms. Development is ongoing to support kexec/kdump for paravirtualized Xen guests. Future plans include enabling crash recovery across Xen and guest domains.
Linux Foundation Collaboration Summit 13 :10 years of Xen and BeyondThe Linux Foundation
In 2013, the Xen Hypervisor will be 10 years old: when Xen was designed, we anticipated a world, which now is known as cloud computing. Today, Xen powers the largest clouds in production and is the basis for several commercial virtualization products. In this talk we will give on overview of Xen and related projects, cover hot developments in the Xen community and outline what comes next.
The talk is intended for users and developers that are familiar with virtualization: no deep knowledge is required. We will start with an architectural overview and cover topics such as: Xen and Linux, how to secure your cloud using disaggregation, SELinux and XSM/FLASK, the evolution of Paravirtualization, Xen on ARM and common challenges for open source hypervisors. We will explore the potential of Open Mirage for testing hypervisors. The talk will conclude with an outlook to the future of Xen.
Introduction to Virtualization, Virsh and Virt-Managerwalkerchang
Virtualization allows for the abstraction and sharing of computer hardware resources like CPU, memory, storage and network capacity. The document introduces virtualization concepts and the tools KVM, Virsh and Virt-manager. It provides documentation on Virsh commands to manage domains (VMs), interfaces and networks. These include commands to define, start, suspend, resume VMs and interfaces, as well as take and restore VM snapshots to revert states. Managing VMs, interfaces and networks with Virsh commands allows administrators to efficiently share hardware resources across VMs.
The 4.5 release no a minor "point" update: it is one of the most feature-rich releases in the project's history. It contains several important additions. Most notably, new Xen PVH virtualization mode now supports running as dom0, enhanced support for Remus, significant ARM architecture updates, security improvements, real-time scheduling, support for Intel Cache Monitoring Technology (CMT), as well as improvements for automotive and embedded use-cases. Other enhancements include additional support for FreeBSD, systemd support, additional libvirt support, the release of Mirage OS 2.0, and more.
Besides giving an overview of Xen 4.5, we will explain the project's roadmap process and share what's ahead for 2015: such as improved OpenStack integration and hotpatching (applying security fixes without the need to reboot).
This talk provides an overview of the Xen Project eco-system and its main use-cases in a number of important market segments: it covers server virtualization, cloud computing and embedded, automotive and related. Lars Kurth highlights why the Xen Project is relevant in these market segments: he provides an overview of the Xen Project's architecture, relevant existing functionality and ongoing and planned developments. To complement the picture, he covers open-source projects that are related to Xen and are of interest for these use-cases. Excellent Software security is key to all of these use-cases. Thus, Lars specifically covers the Xen Project's security features, track record and touches on the project's security practices. He concludes with a few resources that help you get started with the Xen Project and highlight Internship Programs which the project supports.
The talk was delivered at Root Linux Conference 2017. Learn more: http://linux.globallogic.com/materials. The video is available at https://www.youtube.com/watch?v=sjQnAIJji4k
ATG needed to virtualize over 2,000 remote betting terminals to allow for hardware independence and an evolutionary transition from older OS/2 systems to newer Linux systems. They implemented the open source Xen hypervisor which allowed them to run existing OS/2 applications virtually on new Linux machines, transition applications to a new Java platform gradually, and gain more flexibility and influence over the hypervisor development. This provided a cost effective solution to modernize systems while maintaining existing functionality during the transition.
This document discusses the challenges of graphics virtualization. It provides background on native device initialization, QEMU I/O virtualization, and PCI device pass-through. It then covers graphics pass-through for discrete and integrated graphics, including the current status and future work, such as supporting dual graphics devices and improving driver validation.
This is a level 200 - 300 presentation.
It assumes:
Good understanding of vCenter 4, ESX 4, ESXi 4.
Preferably hands-on
We will only cover the delta between 4.1 and 4.0
Overview understanding of related products like VUM, Data Recovery, SRM, View, Nexus, Chargeback, CapacityIQ, vShieldZones, etc
Good understanding of related storage, server, network technology
Target audience
VMware Specialist: SE + Delivery from partners
The document discusses a framework for creating virtual machine monitors (VMMs) using hardware virtualization on x86 processors. It reviews x86 virtualization methods and Intel VT/AMD SVM extensions. The framework abstracts the complexities of directly using virtualization instructions, providing an easier API to develop type-II VMMs as Windows device drivers. It supports features like SMP, error reporting, and a plugin architecture. The goal is to simplify the creation of hypervisors for research and application development.
Presentation power vm virtualization without limitssolarisyougood
This document discusses IBM PowerVM virtualization capabilities for IBM Power Systems. PowerVM allows for virtualization of workloads through logical partitions (LPARs) and virtual machines (VMs). It provides capabilities like rapid provisioning, scalability, recoverability, and workload consolidation to improve efficiency and reduce costs. PowerVM editions differ in features available like the number of concurrent VMs, types of virtual I/O supported, and advanced functions. The document also discusses the Virtual I/O Server (VIOS) appliance, virtual storage and networking options in PowerVM like virtual SCSI, NPIV, and shared Ethernet adapters.
In this talk, John will explore the technology and architecture introduced in the ARM Cortex-A15 processor in support of virtualization. This is the first of multiple processors from ARM that will support true virtualization, and the ability to host existing operating systems binaries without modification. The hardware extensions were defined following careful analysis to address the key virtualization performance limitations of today's solutions while bringing new technologies to the device to better support a virtualized system.
Skylark aims to provide a generic platform which enables cloud applications to be easily deployed and accessed.
In the front-end, Skylark ports Spice to Xen and customizes Spice for application window delivery and display. Skylark users can start any application subscribed from Skylark application store and does not need to know whether it runs in Linux or Windows.
In the back-end, by providing Xen Image File System (Xen-IFS), multiple VMs are allowed to start up simultaneously from the same snapshot or the same disk image. Xen-IFS will automatically keep and isolate the private data produced by each VM.
Skylark makes it possible for Multiple User Single Installation (MUSI) among VMs, and also makes it easy for application delivery through remote VM access.
This document discusses ways for hardware and software vendors to engage with the Xen developer community by contributing testing, reporting bugs, suggesting features, and submitting code patches. It outlines the benefits of contributing as well as practical guidelines for submitting high-quality bug reports, questions, and patches that are more likely to be accepted into the Xen codebase. The goal is to enable greater involvement from external organizations to help drive further development of the Xen hypervisor.
As part of the Google Summer of Code, we tried to add support for SeaBIOS in order to allow guest OSes to be booted directly from PV disk devices rather than from the emulated disk device. SeaBIOS is the BIOS implementation that upstream qemu uses. When the virtual machine is created, SeaBIOS upon initialization uses a generic Xenstore client to communicate with the back end and initialize the front-end block device that will connect to the back end. After the connection is established I/O requests are made via the BIOS int 0x13 interface, guest OSes use the int 0x13 without needing to be aware that PV drivers were used.
The Xen has a tracing infrastructure that can collect a large amount of very detailed data about what's happening at the hypervisor layer. But making sense out of that data isn't an easy thing to do. In this talk I give an overview of xenalyze, a powerful tool I've developed for finding meaning in the vast amounts of data generated by xentrace, enabling both debugging and performance analysis.
This talk explores what has gone in so far in the Linux kernel (version 3.0 and 3.1) and which Linux distributions are deliverinbg Xen again. The otalk explores outstanding challenges and the pieces that are missing and what we can do, and what we cannot do working with Linux.
Lars will give an update on the latest development from the XCP project, standing in for Mike McClurg. He will also talk about the latest developments in project Kronos, which is changing the delivery model for XCP from an appliance to distribution of all XCP components via Linux distributions, and explain what this means for you. He will share 2012 plans for the XCP projecft and explain how you can engage and influence the future direction of XCP.
Hardware accelerated Virtualization in the ARM Cortex™ ProcessorsThe Linux Foundation
The document discusses hardware accelerated virtualization capabilities in ARM Cortex processors including the Cortex-A15. It describes new features like large physical addressing, virtualization extensions, and a virtual interrupt controller that allow multiple operating system instances and work environments to run simultaneously in isolation on ARM devices.
Google uses virtualization for internal corporate infrastructure. As part of this, we have developed a number of tools, some open source, for managing the Xen deployment. The talk will describe the technical infrastructure used, the internal workflows and machine management processes, and the specific use-cases for virtualization.
XPDS16: Scope and Performance of Credit-2 Scheduler. - Anshul Makkar, Ctirix...The Linux Foundation
Credit 2 Scheduler was designed with the following goals in mind: to be functionally complete, have algorithmic improvements, be scalable, and have improved performance compared to Credit 1 Scheduler. Various tests were performed measuring boot times, login times, network throughput, and storage performance on systems with varying numbers of VMs, CPUs, RAM, and storage configurations. Credit 2 Scheduler demonstrated improvements over Credit 1 Scheduler across most of the metrics tested.
Migration of virtual machines without guest downtime is a key feature for hypervisors. Sadly, not all hardware is the same, and keeping guests running in a heterogeneous environment takes a lot of care. Normally, features are advertised via the CPUID instruction, but life is never as simple as we would like. Andrew will discuss what information needs to be controlled, what information can and can't be controlled, and how it applies to Xen guests.
Real-Time Event & Stream Processing on MS AzureKhalid Salama
These slides discuss the main concepts of event & stream processing, as well as the related technologies on Microsoft Azure. We start by giving and overview of what Event & Stream Processing is. Then we describe the canonical architecture of a Stream Processing solution. We will delve into Message Queuing part of the solution. After that, we Introduce Apache Storm on HDInsight, as well as Azure Stream Analytics. We compare Apache Storm to Azure Stream Analytics, and finally conclude with useful resources
Windows server 8 hyper v networking (aidan finn)hypervnu
This document discusses new networking features in Windows Server 8 Hyper-V including built-in NIC teaming, SMB 2.2 for storage access over the network, network virtualization to isolate tenant networks, and security features like port ACLs and private VLANs. It also covers performance optimizations for network traffic like dynamic VMQ, SR-IOV, and new Quality of Service capabilities for prioritizing applications in a multi-tenant hosting environment.
Presentation delivered at LinuxCon China 2017
Real-Time is used for deadline-oriented applications and time-sensitive workloads. Real-Time KVM is the extension of KVM(Linux Kernel-based Virtual Machine) to allow the virtual machines(VM) to be a truly Real-Time operating system.Users sometimes need to run low-latency applications(such as audio/video streaming, highly interactive systems, etc) to meet their requirements in clouds. NFV is a new network concept which uses virtualization and software instead of dedicated network appliances. For some use cases of telecommunications, network latency must be within a certain range of values. Real-Time KVM can help NFV meet this requirements.
In this presentation, Pei Zhang will talk about:
(1)Real-Time KVM introduction
(2)Real-Time cloud building
(3)Real-Time KVM in NFV: VM with openvswitch, dpdk and qemu’s vhostuser
(4)Performance testing results show
This document provides an overview of using esxtop and resxtop tools for advanced performance troubleshooting on VMware ESX hosts. It discusses esxtop counters, how they are derived from raw VMkernel counters, and how counter values are displayed. It also covers esxtop batch mode for generating perfmon compatible logs, using resxtop to connect to remote ESX hosts, and how esxtop can consume non-trivial CPU resources on large environments. Specific examples are given for troubleshooting CPU and memory problems using the tools.
This is the deck that I used at the January 2012 Hyper-V.nu event in Amsterdam, Netherlands. It focuses on the Build announced details on Windows Server 8 Hyper-V networking.
This document provides an overview and summary of key concepts around virtualization that will be covered in more depth at a technical deep dive session, including:
- Virtualization capabilities for desktops/laptops and servers including workstation virtualization and server consolidation.
- How virtual machines work and the overhead associated with virtualization.
- Properties of virtualization like partitioning, isolation, and encapsulation.
- Benefits of server virtualization like consolidation, simpler management, and automated resource pooling.
- Comparison of "hosted" and vSphere virtualization architectures.
- Technologies used in virtualization like binary translation, hardware assistance from Intel VT/AMD-V.
- Ability to virtualize CPU intensive applications with
Amazon EC2 provides a broad selection of instance types to accommodate a diverse mix of workloads. In this session, we provide an overview of the Amazon EC2 instance platform, key platform features, and the concept of instance generations. We dive into the current generation design choices of the different instance families, including General Purpose, Compute Optimized, Storage Optimized, Memory Optimized, and GPU instance. We also detail best practices and share performance tips for getting the most out of your Amazon EC2 instances.
Project ACRN CPU sharing BVT scheduler in ACRN hypervisorProject ACRN
This document describes the Borrowed Virtual Time (BVT) scheduler algorithm implemented in the ACRN hypervisor. BVT aims to provide weighted fair sharing of CPU resources across VMs while prioritizing latency-sensitive workloads. It tracks an effective virtual time for each VM and dispatches the VM with the earliest time. Latency threads can "warp" back in time. BVT is evaluated against the IORR scheduler in ACRN across CPU throughput, I/O throughput and latency tests, showing BVT provides more fair sharing and higher performance. The BVT implementation consists of 302 lines of code in the acrn-hypervisor.
Hardware support for virtualization originated in the 1970s with goals of running multiple virtual machines on a single physical machine. A key requirement was virtualization allowing equivalent execution of programs in a virtual environment as running natively. The x86 architecture posed challenges to virtualization due to sensitive instructions. Intel Virtualization Technology (VT-x) added hardware support for virtualization on x86 by introducing a new CPU operation mode called VMX non-root, and transitions between it and VMX root mode. This reduced the need for software emulation of sensitive instructions and improved virtualization performance.
Prairie DevCon-What's New in Hyper-V in Windows Server "8" Beta - Part 2Damir Bersinic
This is the second of a 2-part series delivered at Prairie DevCon in Calgry on March 15. 2012. The sessions provided a quick overview of the new features of Hyper-V in Windows Server "8" Beta and how these compare to VMware vSphere 5.
Cooperative VM Migration for a virtualized HPC Cluster with VMM-bypass I/O de...Ryousei Takano
1) Cooperative VM migration allows live migration of VMs with VMM-bypass I/O devices like InfiniBand adapters.
2) SymVirt enables coordination between the guest OS and VMM to safely detach and reattach devices during migration.
3) Experiments show SymVirt enables fault-tolerant live migration with minimal overhead for HPC workloads on an InfiniBand cluster. Postcopy migration further reduces downtime during migration.
The Next Generation of Microsoft Virtualization With Windows Server 2012Lai Yoong Seng
The document discusses new features in Windows Server 2012 that improve virtualization capabilities. Key features highlighted include increased scalability for Hyper-V hosts and virtual machines, live migration enhancements, storage migration capabilities, high availability options like Hyper-V Replica for disaster recovery, and flexibility in infrastructure deployment. The presentation aims to demonstrate how these features enable private cloud deployments with optimized performance, scalability, and availability.
Windows Server 2012 includes several new and improved networking features for Hyper-V. These features help improve performance and scalability by offloading more processing to the network interface card. New features include improved Receive Side Scaling, Receive Segment Coalescing, Dynamic Virtual Machine Queuing, Single Root I/O Virtualization, and NIC teaming. These features address challenges around availability, reliability, security and reducing complexity for virtualized workloads.
This document provides an overview of Amazon EC2 instance performance and tips for optimizing workload performance. It discusses how EC2 instances deliver virtualized performance while maintaining flexibility. It defines key performance factors like CPU, memory, networking and I/O. It also summarizes the performance characteristics of different EC2 instance types and how their features like credits, P-states or enhanced networking impact performance. The document aims to help users select the right instance type and tune their workloads for optimal performance on AWS.
• Semplifica l’ambiente di virtualizzazione per i sistemi IBM Power
• Aumenta la produttività dell’amministratore di Sistema
• Consente di replicare e creare rapidamente nuove VMs
• In fase di rilascio nuove funzioni e capacità di ripristino
The document discusses ACRN, an open-source lightweight hypervisor intended for consolidating heterogeneous workloads and streamlining IoT edge development. It provides an overview of ACRN's architecture and key modules, including boot process, CPU virtualization, memory management, interrupt handling, pass-through devices, and device model for handling I/O requests. The document also outlines enhancements in ACRN 2.0, such as supporting new operating systems and safety/real-time virtual machines.
Virtualization allows multiple operating systems to run on a single physical system by sharing underlying hardware resources. It provides flexibility for users, amortizes hardware costs, and isolates separate users. Early virtualization approaches required binary translation or modifying guest operating systems to address challenges posed by the x86 architecture. Modern virtualization leverages hardware extensions like Intel VT-x and AMD-V that introduce a new virtual machine mode to allow guest operating systems to run unmodified while providing hooks for the hypervisor to control privileged operations and resources. This improves performance over earlier software-only approaches.
Performance Profiling in a Virtualized EnvironmentJiaqing Du
The document discusses performance profiling challenges in virtualized environments and presents a solution for guest-wide profiling in the KVM virtual machine monitor. It exposes the CPU's performance monitoring unit interfaces to the guest to collect profiling information accurately accounting for both CPU scheduling and domain switches between guest virtual machines. An experiment profiling packet receive in a Linux guest shows the profiling results can attribute instructions to the correct functions despite virtualization.
webinar vmware v-sphere performance management Challenges and Best PracticesMetron
With the majority of businesses using internal Cloud Services, whether it be Software as a Service (SaaS), Platform as a Service (PaaS) or Infrastructure as a Service (IaaS) in a VMware vSphere environment, this presentation gives an insight into how to manage the gathering Storm Clouds. After an introduction to VMware's Virtual Infrastructure 4 (vSphere) environment andCloud Computing, we discuss how Capacity Management provides the means to spot potential Storm Clouds far in advance and more specifically, how you can avoid them.
Delving deeper we look at IaaS and how to identify potential capacity on demand issues. Discussion focuses on topics such as:
•identifying whether virtual machines are under or over provisioned
•the advantages/disadvantages of application sizing
•how to minimize SLA impact
•whether to scale the infrastructure out, up or in and ultimately how to get it right.
Typically organizations have adopted a "silo mentality" whereby they ring fence IT systems and don’t share resources through lack of trust and confidence. We look at the advantages virtualization brings in terms of flexibility, scalability, cost reduction (monetary and environmental) and how we can protect our 'loved ones' through resource pools, shares, reservations and limits.
With all this in mind, join us to find out what information and processes we recommend you need to have and implement to avoid an Internal Storm and ensure that Brighter Outlook!
Cloud os launch journey to cloud os with windows server 2012 r2 Susantha Silva
This document discusses new features in Windows Server 2012 R2 that enhance private cloud capabilities. It introduces Generation 2 virtual machines that improve performance and security. Features like live migration and replication now offer faster speeds. Storage can be resized online and virtual machines cloned live. The document also explores extending private clouds to Microsoft Azure for additional scalability and flexibility across hybrid cloud environments.
Similar to Realtime scheduling for virtual machines in SKT (20)
Static partitioning is used to split an embedded system into multiple domains, each of them having access only to a portion of the hardware on the SoC. It is key to enable mixed-criticality scenarios, where a critical application, often based on a small RTOS, runs alongside a larger non-critical app, typically based on Linux. The two domains cannot interfere with each other.
This talk will explain how to use Xen for static partitioning. It will introduce dom0-less, a new Xen feature written for the purpose. Dom0-less allows multiple VMs to start at boot time directly from the Xen hypervisor, decreasing boot times drastically. It makes it very easy to partition the system without virtualization overhead. Dom0 becomes unnecessary.
This presentation will go into details on how to setup a Xen dom0-less system. It will show configuration examples and explain device assignment. The talk will discuss its implications for latency-sensitive and safety-critical environments.
XPDDS19: How TrenchBoot is Enabling Measured Launch for Open-Source Platform ...The Linux Foundation
Daniel Smith discusses TrenchBoot, a project aiming to establish a unified approach to harnessing boot integrity technologies across open source platforms. TrenchBoot will enable establishing hardware-rooted integrity during platform boot (first launch inspection), runtime (runtime inspection), and other states. For runtime inspection, TrenchBoot will develop a way to securely re-establish the integrity of Xen at any time without rebooting by dynamically launching an integrity kernel to inspect and verify Xen. The talk outlines the initial and future work of TrenchBoot to integrate these capabilities with Linux, Xen and other open source projects.
XPDDS19 Keynote: Xen in Automotive - Artem Mygaiev, Director, Technology Solu...The Linux Foundation
Artem will briefly cover what has been done since the first talk on Xen in Automotive domain back in 2013, what is going on now and what is still missing for broad adaptation of Xen in vehicles. The following topics will be covered:
Embedded/automotive features of Xen
Collaboration with AGL and GENIVI organizations for standardization
Efforts on Functional Safety compliance
Artem will also go over typical automotive use scenarios for Xen which may not be the same as generic computing use of hypervisor.
XPDDS19 Keynote: Xen Project Weather Report 2019 - Lars Kurth, Director of Op...The Linux Foundation
In this keynote talk, we will give an overview of the state of the Xen Project, trends that impact the project, see whether challenges that surfaced last year have been addressed and how we did it, and highlight new challenges and solutions for the coming year.
In recent years unikernels have shown immense performance potential (e.g., boot times of only a few ms, image sizes of only hundreds of KBs).The fundamental drawback of unikernels is that they require that applications be manually ported to the underlying minimalistic OS, needing both expert work and often considerable amount of time.
The Unikraft project provides a unikernel code base and build system that significantly simplifies the building of unikernels. In addition to support for a number CPU architectures, languages and frameworks, Unikraft provides debugging and tracing features that are generally sorely missing from unikernel projects. In this talk we will talk about these features, show a set of preliminary performance numbers, and provide a roadmap for the project's future.
XPDDS19 Keynote: Secret-free Hypervisor: Now and Future - Wei Liu, Software E...The Linux Foundation
The idea of making Xen secret-free has been floating since Spectre and Meltdown came into light. In this talk we will discuss what is being done and what needs to be done next.
XPDDS19 Keynote: Xen Dom0-less - Stefano Stabellini, Principal Engineer, XilinxThe Linux Foundation
This talk will introduce Dom0-less: a new way of using Xen to build mixed-criticality solutions. Dom0-less is a Xen feature that adds a novel approach to static partitioning based on virtualization. It allows multiple domains to start at boot time directly from the Xen hypervisor, decreasing boot times dramatically. Xen userspace tools, such as xl and libvirt, become optional.
Dom0-less extends the existing device tree based Xen boot protocol to cover information required by additional domains. Binaries, such as kernels and ramdisks, are loaded by the bootloader (u-boot) and advertised to Xen via new device tree bindings.
The audience will learn how to use Dom0-less to partition the system. Uboot and device tree configuration details will be explained to enable the audience to get the most out of this feature. The talk will include a status update and details on future plans.
XPDDS19 Keynote: Patch Review for Non-maintainers - George Dunlap, Citrix Sys...The Linux Foundation
As the number of contributions grow, reviewer bandwidth becomes a bottleneck; and maintainers are always asking for more help. However, ultimately maintainers must at least Ack every patch that goes in; so if you're not a maintainer, how can you contribute? Why should anyone care about your opinion?
This talk will try to lay out some advice and guidelines for non-maintainers, for how they can do code review in a way which will effectively reduce the load on maintainers when they do come to review a patch.
This talk is a follow-up to our Summit 2017 presentation in which we covered our plans for Intel VMFUNC and #VE, as well as related use-cases. This year, we will provide a report on what we have accomplished in Xen 4.12, and what remains to be addressed. We will also give a brief status update of VMI on AMD hardware. The session will end with some real-world numbers of the Hypervisor Introspection solution running on Citrix Hypervisor 8.0 with #VE enabled.
OSSJP/ALS19: The Road to Safety Certification: Overcoming Community Challeng...The Linux Foundation
Safety certification is one of the essential requirements for software to be used in highly regulated industries. Besides technical and compliance issues (such as ISO 26262 vs IEC 611508) transitioning an existing project to become more easily safety certifiable requires significant changes to development practices within an open source project.
In this session, we will lay out some challenges of making safety certification achievable in open source and the Xen Project. We will outline the process the Xen Project has followed thus far and highlight lessons learned along the way. The talk will primarily focus on necessary process, tooling changes and community challenges that can prevent progress. We will be offering an in-depth review of how Xen Project is approaching this challenging goal and try to derive lessons for other projects and contributors.
OSSJP/ALS19: The Road to Safety Certification: How the Xen Project is Making...The Linux Foundation
This document summarizes a discussion around enabling functional safety certification for the Xen open source hypervisor project. Key points discussed include:
- Establishing a split development model with open and closed parts to balance community needs and safety requirements.
- Developing reference implementations and stacks supported by multiple vendors to demonstrate safety certification feasibility.
- Creating plans and processes around requirements, documentation, verification testing, and tooling integration to begin filling gaps for certification.
- Addressing challenges around funding, resources, expertise, and maintaining contributions to ensure any initial work is sustainable long-term.
- Taking an iterative, agile approach to make early progress while further securing necessary funding and support from interested parties.
XPDDS19: Speculative Sidechannels and Mitigations - Andrew Cooper, CitrixThe Linux Foundation
2018 saw fundamental shifts in security boundaries which were previously taken for granted. A lot of work has been done in the past 2 years, and largely in secret under embargo, but there is plenty more work to be done to strengthen the existing mitigations and to try to recover some performance without reopening security holes.
This talk will look at speculative execution sidechannels, the work which has already been done to mitigate the security holes, and future work which hopes to bring some improvements.
XPDDS19: Keeping Coherency on Arm: Reborn - Julien Grall, Arm ltdThe Linux Foundation
The Arm architecture provides a set of guidelines that any software should abide by when accessing the memory with MMU off and update page-tables. Failing to do so may result in getting TLB conflicts or breaking coherency.
In a previous talk ("Keeping coherency on Arm"), we focused on updating safely the stage-2 (aka P2M) page-tables. This talk will focus on the boot code and Xen memory management.
During this session, we will introduce some of the guidelines and when they should be used. We will also discuss how Xen boot sequence needs to be reworked to avoid breaking the guidelines.
XPDDS19: QEMU PV Backend 'qdevification'... What Does it Mean? - Paul Durrant...The Linux Foundation
For many years the QEMU codebase has contained PV backends for Xen guests, giving them paravirtual access to storage, network, keyboard, mouse, etc. however these backends have not been configurable as QEMU devices as their implementation did not fully adhere to the QEMU Object Model (QOM).
Particularly the PV storage backend not using proper QOM devices, or qdevs, meant that the QEMU block layer needed to maintain legacy code that was cluttering up the source. This was causing push-back from the maintainers who did not want to accept any patches relating to that Xen backend until it was 'qdevified'.
In this talk, I'll explain the modifications I made to QEMU to achieve 'qdevification' of the PV storage backend, how compatibility with the libxl toolstack was maintained, and what the next steps in both QEMU and libxl development should be.
XPDDS19: Status of PCI Emulation in Xen - Roger Pau Monné, Citrix Systems R&DThe Linux Foundation
PCI is a local computer bus for attaching hardware devices in a computer, and is the main peripheral bus on modern x86 systems. As such, having a proper way to emulate it is crucial for Xen to be able to expose both fully emulated devices or passthrough devices to guests.
This talk will focus on the current status of PCI emulation in Xen, how and where it is used, what are its main limitations and future plans to improve it in order to be more robust and modular.
XPDDS19: [ARM] OP-TEE Mediator in Xen - Volodymyr Babchuk, EPAM SystemsThe Linux Foundation
Volodymyr will speak about TEE mediators. This is a new feature in Xen which allows multiple virtual machines to interact with Trusted Execution Environment available on platform. He developed mediator for one of TEEs, namely OP-TEE.
He will give background information on why TEE is needed at all and share some implementation details.
XPDDS19: Bringing Xen to the Masses: The Story of Building a Community-driven...The Linux Foundation
Xen is a very powerful hypervisor with a talented and diverse developers community. Despite the fact it's almost everywhere (from the Cloud to the embedded world), it can be difficult to set up and manage as a system administrator. General purpose distros have Xen packages, but that's just a start in your Xen journey: you need some tooling and knowledge to have a working and scalable platform.
XCP-ng was built to overcome those issues: by bringing Xen to the masses with a fully turnkey distro with Xen as its core. It's the logical sequel to the XCP project, with a community focus from the start. We'll see how it happened, what we did, and what's next. Finally, we'll see the impact of XCP-ng on the Xen Project.
XPDDS19: Will Robots Automate Your Job Away? Streamlining Xen Project Contrib...The Linux Foundation
Doug has long advocated for more CI/CD (Continuous Integration / Continuous Delivery) processes to be adopted by the Xen Project from the use of Travis CI and now GitLab CI. This talk aims to propose ideas for building upon the existing process and transforming the development process to provide users a higher quality with each release by the Xen Project.
XPDDS19: Client Virtualization Toolstack in Go - Nick Rosbrook & Brendan Kerr...The Linux Foundation
High level toolstacks for server and cloud virtualization are very mature with large communities using and supporting them. Client virtualization is a much more niche community with unique requirements when compared to those found in the server space. In this talk, we’ll introduce a client virtualization toolstack for Xen (redctl) that we are using in Redfield, a new open-source client virtualization distribution that builds upon the work done by the greater virtualization and Linux communities. We will present a case for maturing libxl’s Go bindings and discuss what advantages Go has to offer for high level toolstacks, including in the server space.
Today Xen is scheduling guest virtual cpus on all available physical cpus independently from each other. Recent security issues on modern processors (e.g. L1TF) require to turn off hyperthreading for best security in order to avoid leaking information from one hyperthread to the other. One way to avoid having to turn off hyperthreading is to only ever schedule virtual cpus of the same guest on one physical core at the same time. This is called core scheduling.
This presentation shows results from the effort to implement core scheduling in the Xen hypervisor. The basic modifications in Xen are presented and performance numbers with core scheduling active are shown.
OpenID AuthZEN Interop Read Out - AuthorizationDavid Brossard
During Identiverse 2024 and EIC 2024, members of the OpenID AuthZEN WG got together and demoed their authorization endpoints conforming to the AuthZEN API
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
Your One-Stop Shop for Python Success: Top 10 US Python Development Providersakankshawande
Simplify your search for a reliable Python development partner! This list presents the top 10 trusted US providers offering comprehensive Python development services, ensuring your project's success from conception to completion.
Let's Integrate MuleSoft RPA, COMPOSER, APM with AWS IDP along with Slackshyamraj55
Discover the seamless integration of RPA (Robotic Process Automation), COMPOSER, and APM with AWS IDP enhanced with Slack notifications. Explore how these technologies converge to streamline workflows, optimize performance, and ensure secure access, all while leveraging the power of AWS IDP and real-time communication via Slack notifications.
Generating privacy-protected synthetic data using Secludy and MilvusZilliz
During this demo, the founders of Secludy will demonstrate how their system utilizes Milvus to store and manipulate embeddings for generating privacy-protected synthetic data. Their approach not only maintains the confidentiality of the original data but also enhances the utility and scalability of LLMs under privacy constraints. Attendees, including machine learning engineers, data scientists, and data managers, will witness first-hand how Secludy's integration with Milvus empowers organizations to harness the power of LLMs securely and efficiently.
Digital Marketing Trends in 2024 | Guide for Staying AheadWask
https://www.wask.co/ebooks/digital-marketing-trends-in-2024
Feeling lost in the digital marketing whirlwind of 2024? Technology is changing, consumer habits are evolving, and staying ahead of the curve feels like a never-ending pursuit. This e-book is your compass. Dive into actionable insights to handle the complexities of modern marketing. From hyper-personalization to the power of user-generated content, learn how to build long-term relationships with your audience and unlock the secrets to success in the ever-shifting digital landscape.
Building Production Ready Search Pipelines with Spark and MilvusZilliz
Spark is the widely used ETL tool for processing, indexing and ingesting data to serving stack for search. Milvus is the production-ready open-source vector database. In this talk we will show how to use Spark to process unstructured data to extract vector representations, and push the vectors to Milvus vector database for search serving.
5th LF Energy Power Grid Model Meet-up SlidesDanBrown980551
5th Power Grid Model Meet-up
It is with great pleasure that we extend to you an invitation to the 5th Power Grid Model Meet-up, scheduled for 6th June 2024. This event will adopt a hybrid format, allowing participants to join us either through an online Mircosoft Teams session or in person at TU/e located at Den Dolech 2, Eindhoven, Netherlands. The meet-up will be hosted by Eindhoven University of Technology (TU/e), a research university specializing in engineering science & technology.
Power Grid Model
The global energy transition is placing new and unprecedented demands on Distribution System Operators (DSOs). Alongside upgrades to grid capacity, processes such as digitization, capacity optimization, and congestion management are becoming vital for delivering reliable services.
Power Grid Model is an open source project from Linux Foundation Energy and provides a calculation engine that is increasingly essential for DSOs. It offers a standards-based foundation enabling real-time power systems analysis, simulations of electrical power grids, and sophisticated what-if analysis. In addition, it enables in-depth studies and analysis of the electrical power grid’s behavior and performance. This comprehensive model incorporates essential factors such as power generation capacity, electrical losses, voltage levels, power flows, and system stability.
Power Grid Model is currently being applied in a wide variety of use cases, including grid planning, expansion, reliability, and congestion studies. It can also help in analyzing the impact of renewable energy integration, assessing the effects of disturbances or faults, and developing strategies for grid control and optimization.
What to expect
For the upcoming meetup we are organizing, we have an exciting lineup of activities planned:
-Insightful presentations covering two practical applications of the Power Grid Model.
-An update on the latest advancements in Power Grid -Model technology during the first and second quarters of 2024.
-An interactive brainstorming session to discuss and propose new feature requests.
-An opportunity to connect with fellow Power Grid Model enthusiasts and users.
Webinar: Designing a schema for a Data WarehouseFederico Razzoli
Are you new to data warehouses (DWH)? Do you need to check whether your data warehouse follows the best practices for a good design? In both cases, this webinar is for you.
A data warehouse is a central relational database that contains all measurements about a business or an organisation. This data comes from a variety of heterogeneous data sources, which includes databases of any type that back the applications used by the company, data files exported by some applications, or APIs provided by internal or external services.
But designing a data warehouse correctly is a hard task, which requires gathering information about the business processes that need to be analysed in the first place. These processes must be translated into so-called star schemas, which means, denormalised databases where each table represents a dimension or facts.
We will discuss these topics:
- How to gather information about a business;
- Understanding dictionaries and how to identify business entities;
- Dimensions and facts;
- Setting a table granularity;
- Types of facts;
- Types of dimensions;
- Snowflakes and how to avoid them;
- Expanding existing dimensions and facts.
In the rapidly evolving landscape of technologies, XML continues to play a vital role in structuring, storing, and transporting data across diverse systems. The recent advancements in artificial intelligence (AI) present new methodologies for enhancing XML development workflows, introducing efficiency, automation, and intelligent capabilities. This presentation will outline the scope and perspective of utilizing AI in XML development. The potential benefits and the possible pitfalls will be highlighted, providing a balanced view of the subject.
We will explore the capabilities of AI in understanding XML markup languages and autonomously creating structured XML content. Additionally, we will examine the capacity of AI to enrich plain text with appropriate XML markup. Practical examples and methodological guidelines will be provided to elucidate how AI can be effectively prompted to interpret and generate accurate XML markup.
Further emphasis will be placed on the role of AI in developing XSLT, or schemas such as XSD and Schematron. We will address the techniques and strategies adopted to create prompts for generating code, explaining code, or refactoring the code, and the results achieved.
The discussion will extend to how AI can be used to transform XML content. In particular, the focus will be on the use of AI XPath extension functions in XSLT, Schematron, Schematron Quick Fixes, or for XML content refactoring.
The presentation aims to deliver a comprehensive overview of AI usage in XML development, providing attendees with the necessary knowledge to make informed decisions. Whether you’re at the early stages of adopting AI or considering integrating it in advanced XML development, this presentation will cover all levels of expertise.
By highlighting the potential advantages and challenges of integrating AI with XML development tools and languages, the presentation seeks to inspire thoughtful conversation around the future of XML development. We’ll not only delve into the technical aspects of AI-powered XML development but also discuss practical implications and possible future directions.
Fueling AI with Great Data with Airbyte WebinarZilliz
This talk will focus on how to collect data from a variety of sources, leveraging this data for RAG and other GenAI use cases, and finally charting your course to productionalization.
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
2. Cloud by Virtualization in SKT
•
• Provide virtualized ICT infra to customers like Amazon EC2 from
SKT’s cloud resource pool exploiting server virtualization
• Resources : Servers/PC, Network, Storage, …
• Functionalities : load balancing, security solution, back-up, …
• Private cloud inside SKT – virtualized servers, virtualized I/O
• Migrate IT services on legacy servers to virtualized servers
• Provide employees with PaaS for software development
• Virtual desktop infrastructure for employees
3. Cloud as Telecommunication Operator
• SK Telecom is a Telecommunication operator as well as a
Cloud service provider
One Common Cloud Computing Infrastructure
General-purpose Server farms
Legacy Network/Telecom Services
for Cloud Hosting based on
on dedicated/reliable equipments
virtualization technique
Virtual Telco
Advantage Requirements
- Scale dynamically with demand 1. Guaranteeing time readiness
- High utilization 2. Scalability of services
- Easy start-up of new services 3. Cost-effective secure storage
4. Case Study – Virtual Telco.
• IMS (IP multimedia subsystems) on Cloud
• Delivering IP multimedia services (VoIP, VOD, Instance Message, …) requiring
session initiation between participants on Internet to users connected to
wireless telecom networks
• Launch easily Internet services on wireless network/telecom infra
IP network
Application Service Media Processing
Session Management User Info. Database
SIP
• Migrate servers for components into Cloud – require high reliability
5. Challenges
1. Guaranteeing time readiness
2. Scalability of services
3. Cost-effective secure storage
• Which virtualization technique, i.e., hypervisor, is most
suitable for supporting real-time VM?
• We choose….
• Xen Hypervisor – best in responsiveness benchmark, open source
• Credit scheduler – default in Xen 4.1, known to be stable
• Second option : Credit 2 scheduler
6. Limitation of credit scheduler
(sec) CPU usage(sec) – Media Player VM
60
Contention between 6 CPU-intensive
VMs(weight = 256) and 1 Media player VM
50
Real-time VM can not occupy the
40
proper amount of CPU even though
with very high weight
30
CPU intensive VM makes use of the
20 residual credits of non CPU-intensive
(i.e. media player) VM’s credits
10
0
Need improvement!!
(weight)
7. Research Goal
• Find improved soft real-time schedulers based on stable
credit scheduler
• Fair CPU sharing – each VM occupies CPU (almost) exactly
proportional to its weight + work-conserving
• Real-time support – fast responsiveness of real-time VMs
• Modify credit scheduler to distinguish realtime VM and non-realtime VM
• Realtime VMs are marked externally and treated specially to provide fast
responsiveness
• Co-work with
8. Preempt based scheduling
BOOST > RT > UNDER > OVER
Idea - Realtime VM’s VCPU is inserted to the runQ of a physical cpu
at right after BOOST priority
Non-realtime VMs can run when RT VMs consume all given credits
or are blocked
Run Queue
CPU 0 VCPU 4 VCPU 2 VCPU 7 VCPU 1 VCPU 0 VCPU 5
: Under Priority
: Over Priority
: Boost Priority VCPU 3 VCPU 6 VCPU 8
: RT Priority
New Job to Schedule
9. BOOST based scheduling (Min Lee, VEE’10 )
In the credit scheduler, VMs can get the highest priority (BOOST)
when they receives events if they were blocked
However, VMs in runQ is not boosted
BOOST realtime VMs always they receives external event even they
are in already in runQ
Run Queue
CPU 0 VCPU 4 VCPU 2 VCPU 7 VCPU 1 VCPU 0 VCPU 5
: Under Priority : Boost Priority External Event
: Over Priority : RT Priority
10. Multi BOOST (by Korea Univ. at XenSummit, Aug, 2011)
Multiple BOOSTs at the same time
Driver domain and realtime VM cannot always get the highest
priority
DRIVER_BOOST > RT_BOOST > BOOST > RT > UNDER > OVER
Run Queue
CPU 0 VCPU 2 VCPU 7 VCPU 1 VCPU 0 VCPU 5
: Under Priority
VCPU 3 VCPU 6 VCPU 8
: Over Priority
: Boost Priority
: RT Priority External Event !
11. Performance Evaluation
VM5 VM6 RT VM
(micro bench) (micro bench) (media player) Physical server spec.
V1 V2 V1 V2
V1 AMD Phenom™ II X6
V3 V4 V3 V4 CPU
1055T (6 cores)
VM1 VM2 VM3 VM4 Memory 16GB
(micro bench) (micro bench) (micro bench) (micro bench)
V1 V2 V1 V2 V1 V2 V1 V2 NET Gigabit Ethernet
V3 V4 V3 V4 V3 V4 V3 V4 Xen 4.1.1
VM VCPU:4, MEM:1GB
PCPU PCPU PCPU PCPU PCPU PCPU
1 2 3 4 5 6
XenHypervisor
Micro bench - Not set as RT priority
- repeat CPU-intensive computing during random time and sleep for random time
- above 98% CPU usage
16. Responsiveness (Ping RTT, Credit)
• The cumulative distribution of ping RTT as the number of simultaneous CPU-
intensive VMs increases
no_contention contention_VM1 contention_VM2 contention_VM3
(cumulative
contention_VM4 contention_VM5 contention_VM6
distribution)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
(ping RTT, ms)
0 1 2 3 4 5 6 7 8 9 10
+
All pings take only 0.5ms 20% of ping takes longer than 10 ms
without contention when 7 VMs run simultaneously
17. Responsiveness (Ping RTT, modified)
no_contention contention_VM6 contention_VM6
(cumulative +preempt
distribution) contention_VM6 contention_VM6 contention_VM6
+rtboost +multiboost preempt+multiboost
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0 1 2 3 4 5 6 7 8 9 10
(ping RTT, ms)
After applying our modification, all pings take only 0.5ms even with contention
weight of realtime VM = 256
weight of non-realtime VM = 256
18. What about Credit2 scheduler?
(sec) CPU usage(sec) – Media Player VM
60
VM burn credits based on their weight
50 Higher weight means credits burn
more slowly
40 VCPUs are inserted into the runQ by
credit order
30 VM with more credits runs first
Credits are “reset” when the next vcpu
20
in the runqueue is less than or equal to
zero
10
0 Achieve both fairness
and work-conserving
(weight)
19. Responsiveness of Credit 2 scheduler
(cumulative weight(256) weight(512) weight(768) weight(1024) weight(1536) weight(2048)
distribution)
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
(ping RTT, ms)
0
0 1 2 3 4 5 6 7 8 9 10
For fast responsiveness, VM needs higher weight.
If we want to divide CPU cycle equally between VMs?
Using special policy for realtime VM is necessary.
20. Ongoing Research
• What if there are several realtime VMs competing
limited physical server/core ?
• Prediction based scheduling between real-time VMs
• Load balancing between physical cores
• Efficient placement policy of RTVMs between physical
servers
• Load balancing between physical servers using live
migration of VMs
21. Summary
• SK telecom is trying to operate Telco services on cloud
resources
• Realtime support in hypervisor is essential
• Analyzed the performance of modifications of Credit
scheduler of Xen hypervisor
• For one realtime VM per physical core, fair sharing and fast
responsiveness
• Plan to improve for more complex and practical cases
23. Comparison of hypervisers
• Evaluation environment
• Physical server : Dell R410 (Xeon 8 cores, 16GB Memory)
• Virtual Machine : 1 Core, 1 GB Memory, 20GB HDD
• Increase the number of VMs running benchmarks
• Benchmarks : PCMARK 2005, kernel compile, SPEC-CPU 2006
• A real-time application measure the delay of the timer interrupt
handling in OS of VM
• Measure every 5 sec. for ten minutes
26. Xen 4.0
•Xen is the best one, but not sufficient
•Contention of non real-time VMs affects
the responsiveness of real-time VM
27. Approach
• VM scheduler in the hypervisor is important
• Credit scheduler
• Stable (default scheduler in Xen hypervisor 4.0) , SMP support
• Need improvement for latency-sensitive VM
• Credit 2 scheduler
• Proportional sharing according to weight of each VM
• Provide responsiveness to VMs with larger weights
• Not so stable yet, need more analysis