The document discusses a use-after-free vulnerability, CVE-2011-1751, that was found in the emulation of the PIIX4 southbridge chip in QEMU KVM. By hot unplugging the emulated ISA bridge, it was possible to exploit a use-after-free bug in the emulated RTC device and achieve arbitrary code execution on the host. The talk outlines an exploit, virtunoid.c, that leverages this bug to inject shellcode and achieve a root shell on the KVM host.
This document provides an introduction to KVM (Kernel-based Virtual Machine), an open source hypervisor that allows running multiple operating systems on a single computer. KVM is integrated into the Linux kernel since version 2.6.20 and provides full virtualization capabilities. It supports hardware virtualization features, sound, snapshots, PCI passthrough, live migration, and more through a layered model. KVM can be managed through tools like Libvirt, Virsh, and Ovirt and is well-suited for cloud computing due to its security, speed, scalability, and reliability.
KVM provides virtualization capabilities using the Linux kernel. It supports full virtualization of x86, PowerPC, s390 and IA-64 architectures using hardware extensions like Intel-VTx and AMD-V. KVM leverages existing Linux components like the scheduler and uses the Linux security model. Guests are scheduled as regular processes. Paravirtualization is used to improve performance through virtio drivers and paravirt_ops. KVM development is ongoing with goals of supporting more hardware features, improving scalability and integrating with management tools like libvirt.
This document provides an overview of virtualization using KVM and oVirt. It discusses the architecture of KVM and Xen, how Qemu works with KVM, and the Libvirt architecture. It also covers installing KVM on CentOS, checking for hardware virtualization support, and installing required packages. Finally, it briefly introduces oVirt and provides some reference documentation links.
LXC – NextGen Virtualization for Cloud benefit realization (cloudexpo)Boden Russell
This document summarizes a presentation on Linux containers (LXC) as an alternative to virtual machines for cloud computing and containerization. It finds that LXC provides significantly better performance than virtual machines in terms of provisioning time, CPU and memory usage, and load on the compute node. Specifically, when packing 15 active VMs/containers on a node, LXC uses 0.54% CPU on average compared to 7.64% for KVM, and 734 MB of memory total compared to 4,387 MB for KVM. When booting VMs/containers serially, the average boot time is 3.5 seconds for LXC versus 5.8 seconds for KVM, and CPU usage is lower overall for
QEMU Disk IO Which performs Better: Native or threads?Pradeep Kumar
Pradeep Kumar Surisetty from Red Hat presented a comparison of native and threaded I/O performance in QEMU disk I/O. He outlined KVM I/O architecture, storage transport options in KVM including virtio-blk configurations, and benchmark tools used. Performance testing was done with various disk types, file systems, images and configurations. Native generally outperformed threads for random I/O workloads, while threads sometimes showed better performance for sequential reads, especially with multiple VMs.
Performance characteristics of traditional v ms vs docker containers (dockerc...Boden Russell
Docker containers provide significantly lower resource usage and higher density than traditional virtual machines when running multiple workloads concurrently on a server.
When booting 15 Ubuntu VMs with MySQL sequentially, Docker containers boot on average 3.5 seconds compared to 5.8 seconds for KVMs. During steady state operation of 15 active VMs, Docker uses on average 0.2% CPU and 49MB RAM per container, while KVMs use 1.9% CPU and 292MB RAM each. Docker maintains low 1-minute load averages of 0.15, while KVMs average 35.9% under load.
Kdump is a long existing method for acquiring dump of crashed kernel, however very few literatures are available to understand it's usage and internals. We receive a lot of queries on kexec mailing list about different issues related to the kexec/kdump environment.
In this presentation, we talk about basics of kdump usage and some internals about kdump/kexec kernel implementation. It includes end to end flow from kdump kernel configuration to crash analysis. We discuss some of the problem which is frequently faced by kdump users. It also includes related information about ELF structure, so that one can debug if vmcore itself gets corrupted because of any architecture related issue.
CIF16/Scale14x: The latest from the Xen Project (Lars Kurth, Chairman of Xen ...The Linux Foundation
An important facilitator of Unikernel development, Xen Project continues to develop new and interesting technologies to support the needs of the next generation datacenter. Potentially game-changing technologies like Unikernels will never reach their full potential unless the hypervisor they rely on can handle a large number of potentially tiny VMs effectively and efficiently.
In this talk, Xen Project Advisory Board Chairman Lars Kurth will discuss some of the major advances in the hypervisor produced in last year's releases (4.5 and 4.6). He will also discuss some of the work in development which could appear in upcoming releases.
This document provides an introduction to KVM (Kernel-based Virtual Machine), an open source hypervisor that allows running multiple operating systems on a single computer. KVM is integrated into the Linux kernel since version 2.6.20 and provides full virtualization capabilities. It supports hardware virtualization features, sound, snapshots, PCI passthrough, live migration, and more through a layered model. KVM can be managed through tools like Libvirt, Virsh, and Ovirt and is well-suited for cloud computing due to its security, speed, scalability, and reliability.
KVM provides virtualization capabilities using the Linux kernel. It supports full virtualization of x86, PowerPC, s390 and IA-64 architectures using hardware extensions like Intel-VTx and AMD-V. KVM leverages existing Linux components like the scheduler and uses the Linux security model. Guests are scheduled as regular processes. Paravirtualization is used to improve performance through virtio drivers and paravirt_ops. KVM development is ongoing with goals of supporting more hardware features, improving scalability and integrating with management tools like libvirt.
This document provides an overview of virtualization using KVM and oVirt. It discusses the architecture of KVM and Xen, how Qemu works with KVM, and the Libvirt architecture. It also covers installing KVM on CentOS, checking for hardware virtualization support, and installing required packages. Finally, it briefly introduces oVirt and provides some reference documentation links.
LXC – NextGen Virtualization for Cloud benefit realization (cloudexpo)Boden Russell
This document summarizes a presentation on Linux containers (LXC) as an alternative to virtual machines for cloud computing and containerization. It finds that LXC provides significantly better performance than virtual machines in terms of provisioning time, CPU and memory usage, and load on the compute node. Specifically, when packing 15 active VMs/containers on a node, LXC uses 0.54% CPU on average compared to 7.64% for KVM, and 734 MB of memory total compared to 4,387 MB for KVM. When booting VMs/containers serially, the average boot time is 3.5 seconds for LXC versus 5.8 seconds for KVM, and CPU usage is lower overall for
QEMU Disk IO Which performs Better: Native or threads?Pradeep Kumar
Pradeep Kumar Surisetty from Red Hat presented a comparison of native and threaded I/O performance in QEMU disk I/O. He outlined KVM I/O architecture, storage transport options in KVM including virtio-blk configurations, and benchmark tools used. Performance testing was done with various disk types, file systems, images and configurations. Native generally outperformed threads for random I/O workloads, while threads sometimes showed better performance for sequential reads, especially with multiple VMs.
Performance characteristics of traditional v ms vs docker containers (dockerc...Boden Russell
Docker containers provide significantly lower resource usage and higher density than traditional virtual machines when running multiple workloads concurrently on a server.
When booting 15 Ubuntu VMs with MySQL sequentially, Docker containers boot on average 3.5 seconds compared to 5.8 seconds for KVMs. During steady state operation of 15 active VMs, Docker uses on average 0.2% CPU and 49MB RAM per container, while KVMs use 1.9% CPU and 292MB RAM each. Docker maintains low 1-minute load averages of 0.15, while KVMs average 35.9% under load.
Kdump is a long existing method for acquiring dump of crashed kernel, however very few literatures are available to understand it's usage and internals. We receive a lot of queries on kexec mailing list about different issues related to the kexec/kdump environment.
In this presentation, we talk about basics of kdump usage and some internals about kdump/kexec kernel implementation. It includes end to end flow from kdump kernel configuration to crash analysis. We discuss some of the problem which is frequently faced by kdump users. It also includes related information about ELF structure, so that one can debug if vmcore itself gets corrupted because of any architecture related issue.
CIF16/Scale14x: The latest from the Xen Project (Lars Kurth, Chairman of Xen ...The Linux Foundation
An important facilitator of Unikernel development, Xen Project continues to develop new and interesting technologies to support the needs of the next generation datacenter. Potentially game-changing technologies like Unikernels will never reach their full potential unless the hypervisor they rely on can handle a large number of potentially tiny VMs effectively and efficiently.
In this talk, Xen Project Advisory Board Chairman Lars Kurth will discuss some of the major advances in the hypervisor produced in last year's releases (4.5 and 4.6). He will also discuss some of the work in development which could appear in upcoming releases.
Virtualization with KVM (Kernel-based Virtual Machine)Novell
As a technical preview, SUSE Linux Enterprise Server 11 contains KVM, which is the next-generation virtualization software delivered with the Linux kernel. In this technical session we will demonstrate how to set up SUSE Linux Enterprise Server 11 for KVM, install some virtual machines and deal with different storage and networking setups.
To demonstrate live migration we will also show a distributed replicated block device (DRBD) setup and a setup based on iSCSI and OCFS2, which are included in SUSE Linux Enterprise Server 11 and SUSE Linux Enterprise 11 High Availability Extension.
XPDS14: Removing the Xen Linux Upstream Delta of Various Linux Distros - Luis...The Linux Foundation
Xen is being used in production by many folks, but are they really using the upstream code? If not what are they using? At least SUSE's supported delta for the Linux kernel consists of 116 patches totaling 353,770 lines of code. Debian has 43 patches for a delta of about 1693 lines of code. What is this delta and how do we shrink it? I will give an overview of the supported Linux kernel delta for Xen at SUSE and Debian with upstream but also layout a proposed roadmap of addressing the delta in collaboration with different teams in the Xen community.
XPDS14 - Xen on ARM: Status and Performance - Stefano Stabellini, CitrixThe Linux Foundation
As the first ARM servers and microservers hit the market, Xen on ARM is becoming more mature, stable and reaching feature parity with x86. This talk will present the current status of the project, will describe the latest improvements, the gaps that still need to be filled and the roadmap going forward. ARMv8 silicon is now available for purchase: we can measure how well Xen on ARM 64-bit is performing on real hardware and compare the performance figures with other hypervisors. The presentation will show these results, it will measure the overhead introduced by Xen on ARM and will compare it with the overhead introduced by Xen and KVM on x86. The talk will explain the reasons behind performance shortfalls and present ideas on how to address them in the future. The performance results will be used to determine when it makes sense to use Xen on ARM and what are the best use cases for it.
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE SystemsThe Linux Foundation
The OpenXT Project is an Open Source community producing a Xen-based platform for client devices with a focus on providing strong security properties. The different primary use cases of this project versus server-based Xen systems have motivated notable technical differences and consequently OpenXT should be of interest to anyone seeking to understand the full set of capabilities on offer within the Xen ecosystem.
In this presentation, Christopher Clark will describe the technical architecture of OpenXT, its current status and development activity within the project and its engagement with the upstream OpenEmbedded and Xen projects. This will include an overview of OpenXT's differentiating features such as Measured Launch, Virtual TPMs, Linux-based stubdoms, a specialized input layer and a distinct PV USB stack for Windows and Linux.
XPDS14: OpenXT - Security and the Properties of a Xen Virtualisation Platform...The Linux Foundation
Released as Open Source Software (OSS) in June 2014, OpenXT is a collection of hardened Linux VMs configured to provide a user facing Xen platform for client devices. This default configuration was mostly static, applying some disaggregation techniques to segregate system components based on a general threat analysis. The goals embodied in
this code base up to its release produced a one-size-fits-most configuration with extensibility in specific areas to encapsulate 3rd party value-add.
With a community now forming around OpenXT we must come to terms with the limitations of the this approach. In this talk Philip will define what OpenXT is and in this definition, show that OpenXT can meet the varied needs of the security and virtualization community through the
construction of a toolkit for the configurable disaggregation of a Xen platform.
The document provides information about an IT professional who manages Insan Solutions and provides various IT services including software development, virtualization using KVM, and IT support. It then discusses KVM virtualization in more detail, explaining that KVM allows using the Linux kernel as a hypervisor for virtual machines, providing benefits like leveraging the Linux scheduler and memory management, free cost, and stable I/O performance. The document concludes with a demonstration of KVM virtualization.
This document discusses unikernels and uKVM running on ARM64 architecture. It provides an introduction to unikernels and how they address issues with traditional workloads on the cloud like slow initialization and excessive resource usage. It describes how uKVM provides virtualization for unikernels and the work being done to port uKVM to ARM64. A demo shows the small footprint and fast boot time of an Nginx unikernel running on uKVM for ARM64, and how many instances can run simultaneously on a single host. Future work is outlined to improve compatibility and performance.
The talk is a status report for the latest release and development projects. It will cover the new features and important bug fixes (if any) in 4.7. It will also provide insight on what’s in the queue for the next major release. Retrospective on the release process will also be part of talk.
KVM and docker LXC Benchmarking with OpenStackBoden Russell
Passive benchmarking with docker LXC and KVM using OpenStack hosted in SoftLayer. These results provide initial incite as to why LXC as a technology choice offers benefits over traditional VMs and seek to provide answers as to the typical initial LXC question -- "why would I consider Linux Containers over VMs" from a performance perspective.
Results here provide insight as to:
- Cloudy ops times (start, stop, reboot) using OpenStack.
- Guest micro benchmark performance (I/O, network, memory, CPU).
- Guest micro benchmark performance of MySQL; OLTP read, read / write complex and indexed insertion.
- Compute node resource consumption; VM / Container density factors.
- Lessons learned during benchmarking.
The tests here were performed using OpenStack Rally to drive the OpenStack cloudy tests and various other linux tools to test the guest performance on a "micro level". The nova docker virt driver was used in the Cloud scenario to realize VMs as docker LXC containers and compared to the nova virt driver for libvirt KVM.
Please read the disclaimers in the presentation as this is only intended to be the "chip of the ice burg".
Containers are becoming increasingly popular as the future of cloud computing. The document discusses and compares several open source container virtualization platforms - KVM, Xen, OpenVZ, and LXC. It provides details on each platform such as the main developer, status, hardware support, virtualization type, and supported operating systems. OpenVZ is highlighted as being production ready since 2006, having extremely low overhead compared to Xen and KVM, and being widely used in projects like Docker.
This document summarizes OpenStack Compute features related to the Libvirt/KVM driver, including updates in Kilo and predictions for Liberty. Key Kilo features discussed include CPU pinning for performance, huge page support, and I/O-based NUMA scheduling. Predictions for Liberty include improved hardware policy configuration, post-plug networking scripts, further SR-IOV support, and hot resize capability. The document provides examples of how these features can be configured and their impact on guest virtual machine configuration and performance.
XPDS14 - Xen as High-Performance NFV Platform - Jun Nakajima, IntelThe Linux Foundation
We are building a high-performance NFV (Network Functions Virtualization) platform using Xen. Unlike the conventional use cases of Xen-based systems, such as data center or public cloud, NFV platforms require low-latency and high-bandwidth I/O for virtual middleboxes to handle small packets efficiently. It is challenging to meet such requirements in virtualization environments because of additional virtualization overhead.
We'll discuss a couple of solutions for high-performance packet forwarding, including SR-IOV VF (Virtual Function), shared-memory channels for VM-to-VM communication, resolving the major bottlenecks and latency/realtime issues, taking advantage of the Xen's architecture. We also discuss how the modern hardware-based virtualization features, such as Posted Interrupts, are helpful. Finally we share the best practices when achieving such high-performance systems.
"We are building a high-performance NFV (Network Functions Virtualization) platform using Xen. Unlike the conventional use cases of Xen-based systems, such as data center or public cloud, NFV platforms require low-latency and high-bandwidth I/O for virtual middleboxes to handle small packets efficiently. It is challenging to meet such requirements in virtualization environments because of additional virtualization overhead.
We'll discuss a couple of solutions for high-performance packet forwarding, including SR-IOV VF (Virtual Function), shared-memory channels for VM-to-VM communication, resolving the major bottlenecks and latency/realtime issues, taking advantage of the Xen's architecture. We also discuss how the modern hardware-based virtualization features, such as Posted Interrupts, are helpful. Finally we share the best practices when achieving such high-performance systems.
Shoot4U: Using VMM Assists to Optimize TLB Operations on Preempted vCPUsJiannan Ouyang, PhD
This slides were presented at the 12th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE’16).
Virtual Machine based approaches to workload consolidation, as seen in IaaS cloud as well as datacenter platforms, have long had to contend with performance degradation caused by synchronization primitives inside the guest environments. These primitives can be affected by virtual CPU preemptions by the host scheduler that can introduce delays that are orders of magnitude longer than those primitives were designed for. While a significant amount of work has focused on the behavior of spinlock primitives as a source of these performance issues, spinlocks do not represent the entirety of synchronization mechanisms that are susceptible to scheduling issues when running in a virtualized environment. In this paper we address the virtualized performance issues introduced by TLB shootdown operations. Our profiling study, based on the PARSEC benchmark suite, has shown that up to 64% of a VM's CPU time can be spent on TLB shootdown operations under certain workloads. In order to address this problem, we present a paravirtual TLB shootdown scheme named Shoot4U. Shoot4U completely eliminates TLB shootdown preemptions by invalidating guest TLB entries from the VMM and allowing guest TLB shootdown operations to complete without waiting for remote virtual CPUs to be scheduled. Our performance evaluation using the PARSEC benchmark suite demonstrates that Shoot4U can reduce benchmark runtime by up to 85% compared an unmodified Linux kernel, and up to 44% over a state-of-the-art paravirtual TLB shootdown scheme.
This document discusses obstacles and solutions for supporting live kernel patching (livepatching) on ARM64 architecture. Livepatching allows patching a running kernel without reboots. It is currently only supported on x86, s390 and powerpc. For ARM64, the main obstacle is that the ARM64 calling convention stores return addresses in a link register rather than the stack. The document proposes a solution that extends the profile instrumentation to save and restore the link register, allowing ftrace to be adapted for ARM64 livepatching. It provides an overview of upstream work on gcc and kernel support to enable livepatching for ARM64.
The document discusses paravirtualized operations (pv-ops) in the Linux kernel which allows it to detect if it is running under virtualization like Xen or KVM and use optimized operations. Pv-ops was successful and mainly involved bug fixes recently. Most Xen kernel development is in subsystems and amounts to over 50,000 lines of code added per release. Future work includes optimizations to blkback and netback drivers to improve I/O performance, as well as adding perf support and developing a hybrid PV/HVM domain 0 capability. The presenter takes questions at the end.
LibVirt and KVM provide virtualization capabilities on Linux systems. LibVirt uses a standardized API to manage different hypervisors like KVM. KVM allows running virtual machines at native speeds by using hardware virtualization extensions. It provides high density and portability compared to running systems directly on hardware. LibVirt handles tasks like networking, storage, and interfaces through tools like Virsh and Virt-Manager, and advanced topics include security with SELinux, bridged networking, and remote access.
Presentation delivered at LinuxCon China 2016
UEFI HTTP/HTTPS Boot is a new feature of UEFI 2.5+. In the meantime, this feature is not yet implemented in any Linux bootloader. This Birds of a Feather session will give an introduction to UEFI HTTP/HTTPS Boot, and share a proof-of-concept implementation based on grub2 that works on both the emulator (QEMU/OVMF) and HPE ProLiant Gen10 servers.
For HTTPS, the experience and comparison will be shared between the purely software-based and UEFI-based implementations in the aspects of ease of implementation, security strength, and limitation.
This document compares the performance of KVM and Docker instances in OpenStack. It finds that for boot and delete operations, KVM is around 20% faster than Docker at high concurrency levels. For CPU and memory tests at low concurrency, KVM is 2-10% slower than bare metal, while Docker is 2-5% slower. For sequential and random read/write file I/O tests, KVM is similar or slightly slower than bare metal, while Docker is slightly slower than KVM. Overall performance decreases for both KVM and Docker as concurrency increases.
VMware ESXi - Intel and Qlogic NIC throughput difference v0.6David Pasek
We are observing different network throughputs on Intel X710 NICs and QLogic FastLinQ QL41xxx NIC. ESXi hardware supports NIC hardware offloading and queueing on 10Gb, 25Gb, 40Gb and 100Gb NIC adapters. Multiple hardware queues per NIC interface (vmnic) and multiple software threads on ESXi VMkernel is depicted and documented in this paper which may or may not be the root cause of the observed problem. The key objective of this document is to clearly document and collect NIC information on two specific Network Adapters and do a comparison to find the difference or at least root cause hypothesis for further troubleshooting.
Virtualization with KVM (Kernel-based Virtual Machine)Novell
As a technical preview, SUSE Linux Enterprise Server 11 contains KVM, which is the next-generation virtualization software delivered with the Linux kernel. In this technical session we will demonstrate how to set up SUSE Linux Enterprise Server 11 for KVM, install some virtual machines and deal with different storage and networking setups.
To demonstrate live migration we will also show a distributed replicated block device (DRBD) setup and a setup based on iSCSI and OCFS2, which are included in SUSE Linux Enterprise Server 11 and SUSE Linux Enterprise 11 High Availability Extension.
XPDS14: Removing the Xen Linux Upstream Delta of Various Linux Distros - Luis...The Linux Foundation
Xen is being used in production by many folks, but are they really using the upstream code? If not what are they using? At least SUSE's supported delta for the Linux kernel consists of 116 patches totaling 353,770 lines of code. Debian has 43 patches for a delta of about 1693 lines of code. What is this delta and how do we shrink it? I will give an overview of the supported Linux kernel delta for Xen at SUSE and Debian with upstream but also layout a proposed roadmap of addressing the delta in collaboration with different teams in the Xen community.
XPDS14 - Xen on ARM: Status and Performance - Stefano Stabellini, CitrixThe Linux Foundation
As the first ARM servers and microservers hit the market, Xen on ARM is becoming more mature, stable and reaching feature parity with x86. This talk will present the current status of the project, will describe the latest improvements, the gaps that still need to be filled and the roadmap going forward. ARMv8 silicon is now available for purchase: we can measure how well Xen on ARM 64-bit is performing on real hardware and compare the performance figures with other hypervisors. The presentation will show these results, it will measure the overhead introduced by Xen on ARM and will compare it with the overhead introduced by Xen and KVM on x86. The talk will explain the reasons behind performance shortfalls and present ideas on how to address them in the future. The performance results will be used to determine when it makes sense to use Xen on ARM and what are the best use cases for it.
XPDS16: The OpenXT Project in 2016 - Christopher Clark, BAE SystemsThe Linux Foundation
The OpenXT Project is an Open Source community producing a Xen-based platform for client devices with a focus on providing strong security properties. The different primary use cases of this project versus server-based Xen systems have motivated notable technical differences and consequently OpenXT should be of interest to anyone seeking to understand the full set of capabilities on offer within the Xen ecosystem.
In this presentation, Christopher Clark will describe the technical architecture of OpenXT, its current status and development activity within the project and its engagement with the upstream OpenEmbedded and Xen projects. This will include an overview of OpenXT's differentiating features such as Measured Launch, Virtual TPMs, Linux-based stubdoms, a specialized input layer and a distinct PV USB stack for Windows and Linux.
XPDS14: OpenXT - Security and the Properties of a Xen Virtualisation Platform...The Linux Foundation
Released as Open Source Software (OSS) in June 2014, OpenXT is a collection of hardened Linux VMs configured to provide a user facing Xen platform for client devices. This default configuration was mostly static, applying some disaggregation techniques to segregate system components based on a general threat analysis. The goals embodied in
this code base up to its release produced a one-size-fits-most configuration with extensibility in specific areas to encapsulate 3rd party value-add.
With a community now forming around OpenXT we must come to terms with the limitations of the this approach. In this talk Philip will define what OpenXT is and in this definition, show that OpenXT can meet the varied needs of the security and virtualization community through the
construction of a toolkit for the configurable disaggregation of a Xen platform.
The document provides information about an IT professional who manages Insan Solutions and provides various IT services including software development, virtualization using KVM, and IT support. It then discusses KVM virtualization in more detail, explaining that KVM allows using the Linux kernel as a hypervisor for virtual machines, providing benefits like leveraging the Linux scheduler and memory management, free cost, and stable I/O performance. The document concludes with a demonstration of KVM virtualization.
This document discusses unikernels and uKVM running on ARM64 architecture. It provides an introduction to unikernels and how they address issues with traditional workloads on the cloud like slow initialization and excessive resource usage. It describes how uKVM provides virtualization for unikernels and the work being done to port uKVM to ARM64. A demo shows the small footprint and fast boot time of an Nginx unikernel running on uKVM for ARM64, and how many instances can run simultaneously on a single host. Future work is outlined to improve compatibility and performance.
The talk is a status report for the latest release and development projects. It will cover the new features and important bug fixes (if any) in 4.7. It will also provide insight on what’s in the queue for the next major release. Retrospective on the release process will also be part of talk.
KVM and docker LXC Benchmarking with OpenStackBoden Russell
Passive benchmarking with docker LXC and KVM using OpenStack hosted in SoftLayer. These results provide initial incite as to why LXC as a technology choice offers benefits over traditional VMs and seek to provide answers as to the typical initial LXC question -- "why would I consider Linux Containers over VMs" from a performance perspective.
Results here provide insight as to:
- Cloudy ops times (start, stop, reboot) using OpenStack.
- Guest micro benchmark performance (I/O, network, memory, CPU).
- Guest micro benchmark performance of MySQL; OLTP read, read / write complex and indexed insertion.
- Compute node resource consumption; VM / Container density factors.
- Lessons learned during benchmarking.
The tests here were performed using OpenStack Rally to drive the OpenStack cloudy tests and various other linux tools to test the guest performance on a "micro level". The nova docker virt driver was used in the Cloud scenario to realize VMs as docker LXC containers and compared to the nova virt driver for libvirt KVM.
Please read the disclaimers in the presentation as this is only intended to be the "chip of the ice burg".
Containers are becoming increasingly popular as the future of cloud computing. The document discusses and compares several open source container virtualization platforms - KVM, Xen, OpenVZ, and LXC. It provides details on each platform such as the main developer, status, hardware support, virtualization type, and supported operating systems. OpenVZ is highlighted as being production ready since 2006, having extremely low overhead compared to Xen and KVM, and being widely used in projects like Docker.
This document summarizes OpenStack Compute features related to the Libvirt/KVM driver, including updates in Kilo and predictions for Liberty. Key Kilo features discussed include CPU pinning for performance, huge page support, and I/O-based NUMA scheduling. Predictions for Liberty include improved hardware policy configuration, post-plug networking scripts, further SR-IOV support, and hot resize capability. The document provides examples of how these features can be configured and their impact on guest virtual machine configuration and performance.
XPDS14 - Xen as High-Performance NFV Platform - Jun Nakajima, IntelThe Linux Foundation
We are building a high-performance NFV (Network Functions Virtualization) platform using Xen. Unlike the conventional use cases of Xen-based systems, such as data center or public cloud, NFV platforms require low-latency and high-bandwidth I/O for virtual middleboxes to handle small packets efficiently. It is challenging to meet such requirements in virtualization environments because of additional virtualization overhead.
We'll discuss a couple of solutions for high-performance packet forwarding, including SR-IOV VF (Virtual Function), shared-memory channels for VM-to-VM communication, resolving the major bottlenecks and latency/realtime issues, taking advantage of the Xen's architecture. We also discuss how the modern hardware-based virtualization features, such as Posted Interrupts, are helpful. Finally we share the best practices when achieving such high-performance systems.
"We are building a high-performance NFV (Network Functions Virtualization) platform using Xen. Unlike the conventional use cases of Xen-based systems, such as data center or public cloud, NFV platforms require low-latency and high-bandwidth I/O for virtual middleboxes to handle small packets efficiently. It is challenging to meet such requirements in virtualization environments because of additional virtualization overhead.
We'll discuss a couple of solutions for high-performance packet forwarding, including SR-IOV VF (Virtual Function), shared-memory channels for VM-to-VM communication, resolving the major bottlenecks and latency/realtime issues, taking advantage of the Xen's architecture. We also discuss how the modern hardware-based virtualization features, such as Posted Interrupts, are helpful. Finally we share the best practices when achieving such high-performance systems.
Shoot4U: Using VMM Assists to Optimize TLB Operations on Preempted vCPUsJiannan Ouyang, PhD
This slides were presented at the 12th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments (VEE’16).
Virtual Machine based approaches to workload consolidation, as seen in IaaS cloud as well as datacenter platforms, have long had to contend with performance degradation caused by synchronization primitives inside the guest environments. These primitives can be affected by virtual CPU preemptions by the host scheduler that can introduce delays that are orders of magnitude longer than those primitives were designed for. While a significant amount of work has focused on the behavior of spinlock primitives as a source of these performance issues, spinlocks do not represent the entirety of synchronization mechanisms that are susceptible to scheduling issues when running in a virtualized environment. In this paper we address the virtualized performance issues introduced by TLB shootdown operations. Our profiling study, based on the PARSEC benchmark suite, has shown that up to 64% of a VM's CPU time can be spent on TLB shootdown operations under certain workloads. In order to address this problem, we present a paravirtual TLB shootdown scheme named Shoot4U. Shoot4U completely eliminates TLB shootdown preemptions by invalidating guest TLB entries from the VMM and allowing guest TLB shootdown operations to complete without waiting for remote virtual CPUs to be scheduled. Our performance evaluation using the PARSEC benchmark suite demonstrates that Shoot4U can reduce benchmark runtime by up to 85% compared an unmodified Linux kernel, and up to 44% over a state-of-the-art paravirtual TLB shootdown scheme.
This document discusses obstacles and solutions for supporting live kernel patching (livepatching) on ARM64 architecture. Livepatching allows patching a running kernel without reboots. It is currently only supported on x86, s390 and powerpc. For ARM64, the main obstacle is that the ARM64 calling convention stores return addresses in a link register rather than the stack. The document proposes a solution that extends the profile instrumentation to save and restore the link register, allowing ftrace to be adapted for ARM64 livepatching. It provides an overview of upstream work on gcc and kernel support to enable livepatching for ARM64.
The document discusses paravirtualized operations (pv-ops) in the Linux kernel which allows it to detect if it is running under virtualization like Xen or KVM and use optimized operations. Pv-ops was successful and mainly involved bug fixes recently. Most Xen kernel development is in subsystems and amounts to over 50,000 lines of code added per release. Future work includes optimizations to blkback and netback drivers to improve I/O performance, as well as adding perf support and developing a hybrid PV/HVM domain 0 capability. The presenter takes questions at the end.
LibVirt and KVM provide virtualization capabilities on Linux systems. LibVirt uses a standardized API to manage different hypervisors like KVM. KVM allows running virtual machines at native speeds by using hardware virtualization extensions. It provides high density and portability compared to running systems directly on hardware. LibVirt handles tasks like networking, storage, and interfaces through tools like Virsh and Virt-Manager, and advanced topics include security with SELinux, bridged networking, and remote access.
Presentation delivered at LinuxCon China 2016
UEFI HTTP/HTTPS Boot is a new feature of UEFI 2.5+. In the meantime, this feature is not yet implemented in any Linux bootloader. This Birds of a Feather session will give an introduction to UEFI HTTP/HTTPS Boot, and share a proof-of-concept implementation based on grub2 that works on both the emulator (QEMU/OVMF) and HPE ProLiant Gen10 servers.
For HTTPS, the experience and comparison will be shared between the purely software-based and UEFI-based implementations in the aspects of ease of implementation, security strength, and limitation.
This document compares the performance of KVM and Docker instances in OpenStack. It finds that for boot and delete operations, KVM is around 20% faster than Docker at high concurrency levels. For CPU and memory tests at low concurrency, KVM is 2-10% slower than bare metal, while Docker is 2-5% slower. For sequential and random read/write file I/O tests, KVM is similar or slightly slower than bare metal, while Docker is slightly slower than KVM. Overall performance decreases for both KVM and Docker as concurrency increases.
VMware ESXi - Intel and Qlogic NIC throughput difference v0.6David Pasek
We are observing different network throughputs on Intel X710 NICs and QLogic FastLinQ QL41xxx NIC. ESXi hardware supports NIC hardware offloading and queueing on 10Gb, 25Gb, 40Gb and 100Gb NIC adapters. Multiple hardware queues per NIC interface (vmnic) and multiple software threads on ESXi VMkernel is depicted and documented in this paper which may or may not be the root cause of the observed problem. The key objective of this document is to clearly document and collect NIC information on two specific Network Adapters and do a comparison to find the difference or at least root cause hypothesis for further troubleshooting.
KubeCon EU 2016: Secure, Cloud-Native Networking with Project CalicoKubeAcademy
Why does the network matter and why does it need to be simple (the 3am test)? Why should we build networks that scale to the extremes and how can we do that with proven technologies? Finally, how can we secure microservices, why should we bother, and what does this mean for developers and operators?
Sched Link: http://sched.co/6BUR
Secure Container solution is to enhance container security by isolating memory between Docker containers inside one VM with Intel VT-x EPT HW, which is highly effective to protect container’s memory and at the meantime defends ret2user privilege escalation attack that exploits kernel vulnerabilities (eg. CVE-2017-6074 UAF (use-after-free) vulnerability). It extends KVM interfaces which the guest OS can leverage to isolate container memory from other containers, and the interfaces rely on Intel VT-x EPT hardware extension and provide memory access protection for the container which sits in an isolated memory region. Each secure container has a dedicated EPT table rather than sharing one EPT table with guest OS, which enforces the cross-EPT memory access protection. The whole solution is user-friendly to fit in the existing cloud server infrastructure with very limited changes.
Xen can run on ARM hardware by taking advantage of hardware virtualization extensions. It uses a single guest type that leverages para-virtualized interfaces for I/O without QEMU. The hypervisor code size is small at around 200,000 lines of code. Xen and Linux are bootable on ARMv7 hardware, and work is ongoing to support 64-bit ARMv8 guests. Challenges include cache coherency and interrupt handling, but the project aims to have full ARMv7 and increasing ARMv8 support in upcoming Xen releases.
This talk will focus on a brief overview of Kubernetes, with a brief demo, and then more of an in-depth focus on issues we've faced moving PHP projects into Docker and Kubernetes like signal propagation, init systems, and logging.
Talk from Cape Town PHP meetup on Feb. 7, 2016:
https://www.meetup.com/Cape-Town-PHP-Group/events/237226310/
Code: https://github.com/zoidbergwill/kubernetes-php-examples
Slides as markdown: http://www.zoidbergwill.com/presentations/2017/kubernetes-php/index.md
This document compares different open source cloud computing platforms and their features. It lists several hypervisor and cloud management platforms including Xen, KVM, Eucalyptus, OpenStack, OpenQRM, OpenNebula, XenServer, Oracle VM, and CloudStack. For each it indicates whether they support various hypervisor technologies like Xen, KVM, LXC, and VMware. It also provides information on the licensing, plugins, and APIs supported by each platform. The document then details the architecture and components of OpenStack including Nova, Swift, and Glance. It lists URLs and credentials for accessing demo OpenStack and OpenQRM installations. Finally, it outlines additional features of the commercial OpenQRM Enterprise edition over the open source version
MOVED: Optimizing the Design and Implementation of KVM/ARM - SFO17-403Linaro
This presentation has been moved to this address:
https://www.slideshare.net/linaroorg/optimizing-the-design-and-implementation-of-kvmarm-sfo17403-81026985
SR-IOV ixgbe Driver Limitations and ImprovementLF Events
SR-IOV is a device virtualization technology, it’s mainly used for improving network performance of virtual machines. However, SR-IOV has some limitations which come from hardware and/or driver implementation. For certain use case, such as Network Function Virtualization(NFV), those limitations are critical to provide services. Intel 10Gb NIC, Niantic(82599), has such limitations(e.g. VLAN filtering, multicast promiscuous) for NFV use case.
This presentation will show the limitations and issues and how they are being addressed, then explain how implements VF multicast promiscuous mode support in ixgbe driver and VF trust, iproute2 functionality enhancement.
This presentation was delivered at LinuxCon Japan 2016 by Hiroshi Shimamoto
LCC17 - Securing Embedded Systems with the Hypervisor - Lars Kurth, CitrixThe Linux Foundation
Lars Kurth is the Community Manager of the Xen Project and Chairman of the Xen Project Advisory Board. He is also the Director of Open Source at Citrix.
The document discusses how Xen can provide security and safety benefits for embedded systems through isolation of guest operating systems and applications using different privilege levels. It also describes how Xen supports real-time workloads and minimizing interrupt latency.
The document shares examples of how Xen is used in various products and projects for aerospace & defense systems, security research, and embedded/industrial applications. It highlights ongoing work to improve standardization and certification.
The document provides an overview of the FreeBSD/VPC virtual private cloud solution. Key points include:
- VPC uses the bhyve hypervisor for virtualization with good CPU and memory isolation between guests.
- Network isolation between guests is challenging with existing FreeBSD networking approaches like tap/bridge/vxlan due to performance issues.
- A new VPC subsystem is proposed to provide dedicated virtual network interfaces (vmnic, vpcp) for guests with improved performance.
- The VPC solution uses VXLAN encapsulation and unique VXLAN network identifiers (VNIs) to provide overlay network isolation between guests on different hosts in a multi-host deployment.
This document discusses advanced microservices techniques with .NET, including Linux containers, Kubernetes, OpenShift, zero-downtime deployments, blue/green and canary releases, and the circuit breaker pattern. It provides demos of building and running a Docker container with a .NET core app on OpenShift, as well as blue-green and canary deployments. Resources are listed for additional information.
Hacktivity 2016: Stealthy, hypervisor based malware analysisTamas K Lengyel
This document discusses techniques for stealthy malware analysis using hypervisor-based monitoring. It describes how debuggers can be detected by malware and introduces using a hypervisor like Xen to monitor guest VMs in a more stealthy way. It covers using features like alternate page tables (altp2m) to improve stealth when single-stepping or handling events from multiple VCPUs. Challenges of porting these techniques to ARM and hiding from techniques malware uses to detect debugging and virtualization are also discussed.
Atf 3 q15-6 - solutions for scaling the cloud computing network infrastructureMason Mei
This document discusses using Arista networking solutions to scale cloud computing networks. It presents two use cases: 1) using OpenStack with the ML2 plugin for network virtualization, and 2) using an SDN controller with OVSDB for network abstraction. For use case 1, it demonstrates how CloudVision Exchange integrates with OpenStack Neutron to provide network automation, visibility, and improved scaling through VXLAN hardware VTEPs and an L3 ECMP fabric. For use case 2, it shows how CloudVision Exchange abstracts the network topology to allow adding bare metal servers and VXLAN gateways through an SDN controller using OVSDB.
- Xen can now run on ARM hardware thanks to its rearchitecting to exploit ARM hardware virtualization extensions and remove unnecessary code like QEMU and shadow pagetables.
- It supports booting Linux as the dom0 and domUs. PV interfaces are used for I/O and there is no need for multiple guest types.
- Current status supports booting on ARMv7 hardware and some features on ARMv8 64-bit. Future work includes more platform support, ACPI, and enabling full ARMv8 virtualization.
Kernel Based virtual machine setup easily on Linux operation systems. It is basically done on Ubuntu operating system.
Anyone with small knowledge of operating Ubuntu can set up it.
LAS16-500: The Rise and Fall of Assembler and the VGIC from HellLinaro
LAS16-500: The Rise and Fall of Assembler and the VGIC from Hell
Speakers: Marc Zyngier, Christoffer Dall
Date: September 30, 2016
★ Session Description ★
KVM/ARM has grown up. While the initial implementation of virtualization support for ARM processors in Linux was a quality upstream software project, there were initial design decisions simply not suitable for a long-term maintained hypervisor code base. For example, the way KVM/ARM utilized the hardware support for virtualization, was by running a ‘switching’ layer of code in EL2, purely written in assembly. This was a reasonable design decision in the initial implementation, as the switching layer only had to do one thing: Switch between a VM and the host. But as we began to optimize the implementation, add support for ARMv8.1 and VHE, and added features such as debugging support, we had to move to a more integrated approach, writing the switching logic in C code as well. As another example, the support for virtual interrupts, famously known as the VGIC, was designed with a focus on optimizing MMIO operations. As it turns out, MMIO operations is a less important and infrequent operation on the GIC, and the design had some serious negative consequences for supporting other state transitions for virtual interrupts and had negative performance implications. Therefore, we completely redesigned the VGIC support, and implemented the whole thing from scratch as a team effort, with a very promising result, upstream since Linux v4.7. In this talk we will cover the evolution of this software project and give an overview of the state of the project as it is today.
★ Resources ★
Etherpad: pad.linaro.org/p/las16-500
Presentations & Videos: http://connect.linaro.org/resource/las16/las16-500/
★ Event Details ★
Linaro Connect Las Vegas 2016 – #LAS16
September 26-30, 2016
http://www.linaro.org
http://connect.linaro.org
Rmll Virtualization As Is Tool 20090707 V1.0guest72e8c1
Virtualization can be used as a tool for consolidating information systems. There are several common issues that come up with virtualization including ensuring sufficient processor architecture support, network capacity, and dealing with legacy physical hardware. It is important to analyze legacy systems and map application relationships before starting virtualization. Popular hypervisors include KVM, Xen, and OpenVZ. KVM is recommended due to its integration with Linux. Libvirt provides an abstraction layer for different hypervisors. Orchestrators like Enomalism can help manage large virtualized environments through a web interface. Tools were also discussed for snapshotting images, configuring networks, and preventing out-of-memory issues.
Virtualization can be used as a tool for consolidating information systems. There are several common issues to address when starting virtualization including ensuring sufficient processor architecture support, network capacity, and dealing with physical legacy systems. Key steps include analyzing legacy systems and mapping application relationships. Popular hypervisors include KVM, Xen, and OpenVZ. KVM is recommended due to its integration with Linux. Libvirt provides an abstraction layer for different hypervisors. Orchestrators like Enomalism can help manage virtual machines. Useful related tools include those for snapshots, configuration, and dealing with out of memory situations.
AI 101: An Introduction to the Basics and Impact of Artificial IntelligenceIndexBug
Imagine a world where machines not only perform tasks but also learn, adapt, and make decisions. This is the promise of Artificial Intelligence (AI), a technology that's not just enhancing our lives but revolutionizing entire industries.
Threats to mobile devices are more prevalent and increasing in scope and complexity. Users of mobile devices desire to take full advantage of the features
available on those devices, but many of the features provide convenience and capability but sacrifice security. This best practices guide outlines steps the users can take to better protect personal devices and information.
Programming Foundation Models with DSPy - Meetup SlidesZilliz
Prompting language models is hard, while programming language models is easy. In this talk, I will discuss the state-of-the-art framework DSPy for programming foundation models with its powerful optimizers and runtime constraint system.
HCL Notes and Domino License Cost Reduction in the World of DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-and-domino-license-cost-reduction-in-the-world-of-dlau/
The introduction of DLAU and the CCB & CCX licensing model caused quite a stir in the HCL community. As a Notes and Domino customer, you may have faced challenges with unexpected user counts and license costs. You probably have questions on how this new licensing approach works and how to benefit from it. Most importantly, you likely have budget constraints and want to save money where possible. Don’t worry, we can help with all of this!
We’ll show you how to fix common misconfigurations that cause higher-than-expected user counts, and how to identify accounts which you can deactivate to save money. There are also frequent patterns that can cause unnecessary cost, like using a person document instead of a mail-in for shared mailboxes. We’ll provide examples and solutions for those as well. And naturally we’ll explain the new licensing model.
Join HCL Ambassador Marc Thomas in this webinar with a special guest appearance from Franz Walder. It will give you the tools and know-how to stay on top of what is going on with Domino licensing. You will be able lower your cost through an optimized configuration and keep it low going forward.
These topics will be covered
- Reducing license cost by finding and fixing misconfigurations and superfluous accounts
- How do CCB and CCX licenses really work?
- Understanding the DLAU tool and how to best utilize it
- Tips for common problem areas, like team mailboxes, functional/test users, etc
- Practical examples and best practices to implement right away
TrustArc Webinar - 2024 Global Privacy SurveyTrustArc
How does your privacy program stack up against your peers? What challenges are privacy teams tackling and prioritizing in 2024?
In the fifth annual Global Privacy Benchmarks Survey, we asked over 1,800 global privacy professionals and business executives to share their perspectives on the current state of privacy inside and outside of their organizations. This year’s report focused on emerging areas of importance for privacy and compliance professionals, including considerations and implications of Artificial Intelligence (AI) technologies, building brand trust, and different approaches for achieving higher privacy competence scores.
See how organizational priorities and strategic approaches to data security and privacy are evolving around the globe.
This webinar will review:
- The top 10 privacy insights from the fifth annual Global Privacy Benchmarks Survey
- The top challenges for privacy leaders, practitioners, and organizations in 2024
- Key themes to consider in developing and maintaining your privacy program
Communications Mining Series - Zero to Hero - Session 1DianaGray10
This session provides introduction to UiPath Communication Mining, importance and platform overview. You will acquire a good understand of the phases in Communication Mining as we go over the platform with you. Topics covered:
• Communication Mining Overview
• Why is it important?
• How can it help today’s business and the benefits
• Phases in Communication Mining
• Demo on Platform overview
• Q/A
How to Get CNIC Information System with Paksim Ga.pptxdanishmna97
Pakdata Cf is a groundbreaking system designed to streamline and facilitate access to CNIC information. This innovative platform leverages advanced technology to provide users with efficient and secure access to their CNIC details.
Unlock the Future of Search with MongoDB Atlas_ Vector Search Unleashed.pdfMalak Abu Hammad
Discover how MongoDB Atlas and vector search technology can revolutionize your application's search capabilities. This comprehensive presentation covers:
* What is Vector Search?
* Importance and benefits of vector search
* Practical use cases across various industries
* Step-by-step implementation guide
* Live demos with code snippets
* Enhancing LLM capabilities with vector search
* Best practices and optimization strategies
Perfect for developers, AI enthusiasts, and tech leaders. Learn how to leverage MongoDB Atlas to deliver highly relevant, context-aware search results, transforming your data retrieval process. Stay ahead in tech innovation and maximize the potential of your applications.
#MongoDB #VectorSearch #AI #SemanticSearch #TechInnovation #DataScience #LLM #MachineLearning #SearchTechnology
HCL Notes und Domino Lizenzkostenreduzierung in der Welt von DLAUpanagenda
Webinar Recording: https://www.panagenda.com/webinars/hcl-notes-und-domino-lizenzkostenreduzierung-in-der-welt-von-dlau/
DLAU und die Lizenzen nach dem CCB- und CCX-Modell sind für viele in der HCL-Community seit letztem Jahr ein heißes Thema. Als Notes- oder Domino-Kunde haben Sie vielleicht mit unerwartet hohen Benutzerzahlen und Lizenzgebühren zu kämpfen. Sie fragen sich vielleicht, wie diese neue Art der Lizenzierung funktioniert und welchen Nutzen sie Ihnen bringt. Vor allem wollen Sie sicherlich Ihr Budget einhalten und Kosten sparen, wo immer möglich. Das verstehen wir und wir möchten Ihnen dabei helfen!
Wir erklären Ihnen, wie Sie häufige Konfigurationsprobleme lösen können, die dazu führen können, dass mehr Benutzer gezählt werden als nötig, und wie Sie überflüssige oder ungenutzte Konten identifizieren und entfernen können, um Geld zu sparen. Es gibt auch einige Ansätze, die zu unnötigen Ausgaben führen können, z. B. wenn ein Personendokument anstelle eines Mail-Ins für geteilte Mailboxen verwendet wird. Wir zeigen Ihnen solche Fälle und deren Lösungen. Und natürlich erklären wir Ihnen das neue Lizenzmodell.
Nehmen Sie an diesem Webinar teil, bei dem HCL-Ambassador Marc Thomas und Gastredner Franz Walder Ihnen diese neue Welt näherbringen. Es vermittelt Ihnen die Tools und das Know-how, um den Überblick zu bewahren. Sie werden in der Lage sein, Ihre Kosten durch eine optimierte Domino-Konfiguration zu reduzieren und auch in Zukunft gering zu halten.
Diese Themen werden behandelt
- Reduzierung der Lizenzkosten durch Auffinden und Beheben von Fehlkonfigurationen und überflüssigen Konten
- Wie funktionieren CCB- und CCX-Lizenzen wirklich?
- Verstehen des DLAU-Tools und wie man es am besten nutzt
- Tipps für häufige Problembereiche, wie z. B. Team-Postfächer, Funktions-/Testbenutzer usw.
- Praxisbeispiele und Best Practices zum sofortigen Umsetzen
Essentials of Automations: The Art of Triggers and Actions in FMESafe Software
In this second installment of our Essentials of Automations webinar series, we’ll explore the landscape of triggers and actions, guiding you through the nuances of authoring and adapting workspaces for seamless automations. Gain an understanding of the full spectrum of triggers and actions available in FME, empowering you to enhance your workspaces for efficient automation.
We’ll kick things off by showcasing the most commonly used event-based triggers, introducing you to various automation workflows like manual triggers, schedules, directory watchers, and more. Plus, see how these elements play out in real scenarios.
Whether you’re tweaking your current setup or building from the ground up, this session will arm you with the tools and insights needed to transform your FME usage into a powerhouse of productivity. Join us to discover effective strategies that simplify complex processes, enhancing your productivity and transforming your data management practices with FME. Let’s turn complexity into clarity and make your workspaces work wonders!
“An Outlook of the Ongoing and Future Relationship between Blockchain Technologies and Process-aware Information Systems.” Invited talk at the joint workshop on Blockchain for Information Systems (BC4IS) and Blockchain for Trusted Data Sharing (B4TDS), co-located with with the 36th International Conference on Advanced Information Systems Engineering (CAiSE), 3 June 2024, Limassol, Cyprus.
Best 20 SEO Techniques To Improve Website Visibility In SERPPixlogix Infotech
Boost your website's visibility with proven SEO techniques! Our latest blog dives into essential strategies to enhance your online presence, increase traffic, and rank higher on search engines. From keyword optimization to quality content creation, learn how to make your site stand out in the crowded digital landscape. Discover actionable tips and expert insights to elevate your SEO game.
UiPath Test Automation using UiPath Test Suite series, part 5DianaGray10
Welcome to UiPath Test Automation using UiPath Test Suite series part 5. In this session, we will cover CI/CD with devops.
Topics covered:
CI/CD with in UiPath
End-to-end overview of CI/CD pipeline with Azure devops
Speaker:
Lyndsey Byblow, Test Suite Sales Engineer @ UiPath, Inc.
GraphRAG for Life Science to increase LLM accuracyTomaz Bratanic
GraphRAG for life science domain, where you retriever information from biomedical knowledge graphs using LLMs to increase the accuracy and performance of generated answers
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/building-and-scaling-ai-applications-with-the-nx-ai-manager-a-presentation-from-network-optix/
Robin van Emden, Senior Director of Data Science at Network Optix, presents the “Building and Scaling AI Applications with the Nx AI Manager,” tutorial at the May 2024 Embedded Vision Summit.
In this presentation, van Emden covers the basics of scaling edge AI solutions using the Nx tool kit. He emphasizes the process of developing AI models and deploying them globally. He also showcases the conversion of AI models and the creation of effective edge AI pipelines, with a focus on pre-processing, model conversion, selecting the appropriate inference engine for the target hardware and post-processing.
van Emden shows how Nx can simplify the developer’s life and facilitate a rapid transition from concept to production-ready applications.He provides valuable insights into developing scalable and efficient edge AI solutions, with a strong focus on practical implementation.
“Building and Scaling AI Applications with the Nx AI Manager,” a Presentation...
Virtunoid: Breaking out of KVM
1. Virtunoid: Breaking out of KVM
Nelson Elhage
DEFCON 19
August 8, 2011
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 1 / 50
2. KVM
The new hotness for Virtualization on Linux
Official virtualization platform for Ubuntu and RHEL.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 2 / 50
3. Who am I?
Kernel engineer at Ksplice (now Oracle).
Open-source security hacker in my spare time.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 3 / 50
4. Outline
1 KVM: Architecture overview
Attack Surface
2 CVE-2011-1751: The bug
3 virtunoid.c: The exploit
%rip control
Getting to shellcode
Bypassing ASLR
4 Conclusions and further research
5 Demo
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 4 / 50
5. KVM: Architecture overview
1 KVM: Architecture overview
Attack Surface
2 CVE-2011-1751: The bug
3 virtunoid.c: The exploit
%rip control
Getting to shellcode
Bypassing ASLR
4 Conclusions and further research
5 Demo
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 5 / 50
6. KVM: Architecture overview
KVM: The components
kvm.ko
kvm-intel.ko / kvm-amd.ko
qemu-kvm
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 6 / 50
7. KVM: Architecture overview
kvm.ko
The core KVM kernel module
Implements the virtual CPU and MMU (with the hardware’s help).
Emulates a few devices in-kernel for efficiency.
Provides ioctls for communicating with the kernel module.
Contains an emulator for a subset of x86 used in handling certain
traps (!)
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 7 / 50
8. KVM: Architecture overview
kvm-intel.ko / kvm-amd.ko
Provides support for Intel’s VMX and AMD’s SVM virtualization
extensions.
Relatively small compared to the rest of KVM (one .c file each)
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 8 / 50
9. KVM: Architecture overview
qemu-kvm
Provides the most direct user interface to KVM.
Based on the classic qemu emulator.
Implements the bulk of the virtual devices a VM uses.
Implements a wide variety of types of devices.
An order of magnitude more code than the kernel module.
There is work in progress to replace this component, but it’s a ways
out, if ever.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 9 / 50
10. KVM: Architecture overview Attack Surface
kvm.ko
A tempting target – successful exploitation gets ring0 on the host
without further escalation.
Much less code than qemu-kvm, and much of that is dedicated to
interfacing with qemu-kvm, not the guest directly.
The x86 emulator is an interesting target.
A number of bugs have been discovered allowing privesc within the
guest.
A lot of tricky code that is not often exercised.
Not the target of this talk, but I have some ideas for future work.
Also, be on the lookout for privesc within either the host or guest.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 10 / 50
11. KVM: Architecture overview Attack Surface
kvm-intel.ko / kvm-amd.ko
Not much direct attack surface.
Largely straight-line code doing lots of low-level bit twiddling with the
hardware structures.
Lots of subtlety, possibly some more complex attacks.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 11 / 50
12. KVM: Architecture overview Attack Surface
qemu-kvm
A veritable goldmine of targets.
Hundreds of thousands of lines of device emulation code.
Emulated devices communicate directly with the guest via MMIO or
IO ports, lots of attack surface.
Much of the code comes straight from qemu and is ancient.
qemu-kvm is often sandboxed using SELinux or similar, meaning that
successful exploitation will often require a second privesc within the
host.
(Fortunately, Linux never has any of those)
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 12 / 50
13. CVE-2011-1751: The bug
1 KVM: Architecture overview
Attack Surface
2 CVE-2011-1751: The bug
3 virtunoid.c: The exploit
%rip control
Getting to shellcode
Bypassing ASLR
4 Conclusions and further research
5 Demo
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 13 / 50
14. CVE-2011-1751: The bug
RHSA-2011:0534-1
“It was found that the PIIX4 Power Management emulation layer in
qemu-kvm did not properly check for hot plug eligibility during device
removals. A privileged guest user could use this flaw to crash the guest or,
possibly, execute arbitrary code on the host. (CVE-2011-1751)”
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 14 / 50
15. CVE-2011-1751: The bug
PIIX4
The PIIX4 was a southbridge chip used in many circa-2000 Intel
chipsets.
The default southbridge emulated by qemu-kvm
Includes ACPI support, a PCI-ISA bridge, an embedded MC146818
RTC, and much more.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 15 / 50
16. CVE-2011-1751: The bug
Device Hotplug
The PIIX4 supports PCI hotplug, implemented by writing values to IO
port 0xae08.
qemu-kvm emulates this by calling qdev_free(qdev);, which calls a
device’s cleanup function and free()s it.
Many devices weren’t implemented with hotplug in mind!
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 16 / 50
17. CVE-2011-1751: The bug
The PCI-ISA bridge
In particular, it should not be possible to unplug the ISA bridge.
Among other things, the emulated MC146818 RTC hangs off the ISA
bridge.
KVM’s emulated RTC is not designed to be unplugged; In particular,
it leaves around dangling QEMUTimer objects when unplugged.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 17 / 50
18. CVE-2011-1751: The bug
QEMUTimer
t y p e d e f v o i d QEMUTimerCB( v o i d * opaque ) ;
s t r u c t QEMUTimer {
...
i n t 6 4 t e x p i r e t i m e ; /* i n n a n o s e c o n d s */
QEMUTimerCB * cb ;
v o i d * opaque ;
s t r u c t QEMUTimer * n e x t ;
};
t y p e d e f s t r u c t RTCState {
...
QEMUTimer * s e c o n d t i m e r ;
...
} RTCState ;
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 18 / 50
19. CVE-2011-1751: The bug
Use-after-free
Unplugging the virtual RTC free()s the RTCState
It doesn’t free() or unregister either of the timers.
So we’re left with dangling pointers from the QEMUTimers
On the next second, we’ll call rtc_update_second(<freed
RTCState> )
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 19 / 50
20. CVE-2011-1751: The bug
Reproducer
#i n c l u d e <s y s / i o . h>
i n t main ( v o i d ) {
iopl (3);
o u t l (2 , 0 xae08 ) ;
return 0;
}
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 20 / 50
21. virtunoid.c: The exploit
1 KVM: Architecture overview
Attack Surface
2 CVE-2011-1751: The bug
3 virtunoid.c: The exploit
%rip control
Getting to shellcode
Bypassing ASLR
4 Conclusions and further research
5 Demo
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 21 / 50
22. virtunoid.c: The exploit %rip control
High-level TODO
1 Inject a controlled QEMUTimer into qemu-kvm at a known address
2 Eject the emulated ISA bridge
3 Force an allocation into the freed RTCState, with second_timer
pointing at our dummy timer.
When rtc_update_second next runs, our timer will get scheduled.
One second later, boom.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 22 / 50
23. virtunoid.c: The exploit %rip control
1. Injecting data
The guest’s RAM is backed by a simple mmap()ed region inside the
qemu-kvm process.
So we allocate an object in the guest, and compute
hva = physmem_base + gpa
gpa = (gva_to_gfn(gva) << PAGE_SHIFT)
+ page_offset(gva)
For now, assume we can guess physmem_base (e.g. no ASLR)
hva host virtual address
gva guest virtual address
gpa guest physical address
gfn guest frame (physical page) number
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 23 / 50
24. virtunoid.c: The exploit %rip control
qemu-kvm userspace network stack
qemu-kvm contains a user-mode networking stack.
Implements a DHCP server, DNS server, and a gateway NAT.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 24 / 50
25. virtunoid.c: The exploit %rip control
Userspace network stack packet delivery
The user-mode stack normally handles packets synchronously.
To prevent recursion, if a second packet is emitted while handling a
first packet, the second packet is queued, using malloc().
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 25 / 50
26. virtunoid.c: The exploit %rip control
The virtual network gateway responds synchronously to ICMP ping.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 26 / 50
27. virtunoid.c: The exploit %rip control
Putting it together
1 Allocate a fake QEMUTimer
Point ->cb at the desired %rip.
2 Calculate its address in the host.
3 Write 2 to IO port 0xae08 to eject the ISA bridge.
4 ping the emulated gateway with ICMP packets containing pointers to
your allocated timer in the host.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 27 / 50
28. virtunoid.c: The exploit Getting to shellcode
We’ve got %rip, now what?
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
29. virtunoid.c: The exploit Getting to shellcode
We’ve got %rip, now what?
Get EIP = 0x41414141 and declare victory.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
30. virtunoid.c: The exploit Getting to shellcode
We’ve got %rip, now what?
Get EIP = 0x41414141 and declare victory.
Disable NX in my BIOS and call it good enough for a demo.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
31. virtunoid.c: The exploit Getting to shellcode
We’ve got %rip, now what?
Get EIP = 0x41414141 and declare victory.
Disable NX in my BIOS and call it good enough for a demo.
Do a ROP pivot, ROP to victory.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
32. virtunoid.c: The exploit Getting to shellcode
We’ve got %rip, now what?
Get EIP = 0x41414141 and declare victory.
Disable NX in my BIOS and call it good enough for a demo.
Do a ROP pivot, ROP to victory.
Do something else clever.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 28 / 50
33. virtunoid.c: The exploit Getting to shellcode
Another look at QEMUTimer
s t r u c t QEMUTimer {
...
i n t 6 4 t e x p i r e t i m e ; /* i n n a n o s e c o n d s */
...
s t r u c t QEMUTimer * n e x t ;
};
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 29 / 50
34. virtunoid.c: The exploit Getting to shellcode
qemu_run_timers
s t a t i c v o i d q e m u r u n t i m e r s ( QEMUClock * c l o c k )
{
QEMUTimer ** p t i m e r h e a d , * t s ;
int64 t current time ;
current time = qemu get clock ns ( clock );
p t i m e r h e a d = &a c t i v e t i m e r s [ c l o c k −>t y p e ] ;
for ( ; ; ) {
ts = * ptimer head ;
i f ( ! qemu timer expired ns ( ts , c u r r e n t t i m e ))
break ;
* p t i m e r h e a d = t s −>n e x t ;
t s −>n e x t = NULL ;
t s −>cb ( t s −>opaque ) ;
}
}
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 30 / 50
35. virtunoid.c: The exploit Getting to shellcode
Timer chains
second_timer
f1
−>cb
−>opaque X
−>next
f2
−>cb
−>opaque Y
−>next
f3
−>cb
−>opaque Z
−>next
NULL
⇒ f1(X); f2(Y); f3(Z);
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 31 / 50
36. virtunoid.c: The exploit Getting to shellcode
More arguments
amd64 calling convention: %rdi, %rsi, %rdx, . . .
Every version of qemu_run_timers I’ve checked leaves %rsi
untouched.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 32 / 50
37. virtunoid.c: The exploit Getting to shellcode
More arguments
set rsi :
movl %r d i , %r s i
ret
Let f1 = set_rsi
f2(Y, X)
Same trick doesn’t work with %rdx.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 33 / 50
38. virtunoid.c: The exploit Getting to shellcode
set_rsi
v o i d c p u o u t l ( p i o a d d r t addr , u i n t 3 2 t v a l )
{
i o p o r t w r i t e ( 2 , addr , v a l ) ;
}
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 34 / 50
39. virtunoid.c: The exploit Getting to shellcode
Getting to mprotect
int mprotect(const void *addr, size_t len, int prot);
#define PROT_EXEC 0x4
s t a t i c u i n t 3 2 t i o p o r t r e a d l t h u n k ( v o i d * opaque , u i n t 3 2 t a d d r )
{
IORange * i o p o r t = opaque ;
u i n t 6 4 t data ;
i o p o r t −>ops−>r e a d ( i o p o r t , a d d r − i o p o r t −>b a s e , 4 , &d a t a ) ;
r e t u r n data ;
}
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 35 / 50
40. virtunoid.c: The exploit Getting to shellcode
Putting it together
Allocate a fake IORangeOps, with fake_ops->read = mprotect.
Allocate a page-aligned IORange, with ->ops = fake_ops and
->base = -PAGE_SIZE.
Copy shellcode immediately following the IORange.
Construct a timer chain that calls
cpu_outl(0, *)
ioport_readl_thunk(fake_ioport, 0)
fake_ioport + 1
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 36 / 50
41. virtunoid.c: The exploit Getting to shellcode
Why not ROP?
Continued execution is dead simple.
Reduced dependence on details of compiled code.
I’m not that good at ROP :)
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 37 / 50
42. virtunoid.c: The exploit Bypassing ASLR
Addresses
For a known qemu-kvm binary, we need two addresses.
The base address of the qemu-kvm binary, to find code addresses.
physmem_base, the address of the physical memory mapping inside
qemu-kvm.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 38 / 50
43. virtunoid.c: The exploit Bypassing ASLR
Option A
Find an information leak.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 39 / 50
44. virtunoid.c: The exploit Bypassing ASLR
Option B
Assume non-PIE, and be clever.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 40 / 50
45. virtunoid.c: The exploit Bypassing ASLR
fw_cfg
Emulated IO ports 0x510 (address) and 0x511 (data)
Used to communicate various tables to the qemu BIOS (e820 map,
ACPI tables, etc)
Also provides support for exporting writable tables to the BIOS.
However, fw_cfg_write doesn’t check if the target table is supposed
to be writable!
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 41 / 50
46. virtunoid.c: The exploit Bypassing ASLR
Static data
Several fw_cfg areas are backed by statically-allocated buffers.
Net result: nearly 500 writable bytes inside static variables.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 42 / 50
47. virtunoid.c: The exploit Bypassing ASLR
read4 your way to victory
mprotect needs a page-aligned address, so these aren’t suitable for
our shellcode.
But, we can construct fake timer chains in this space to build a
read4() primitive.
Follow pointers from static variables to find physmem_base
Proceed as before.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 43 / 50
48. virtunoid.c: The exploit Bypassing ASLR
Repeated timer chaining
Previously, we ended timer chains with ->next = NULL.
Instead, end them with a timer that calls rtc_update_second.
The timer we control will be scheduled once a second, and we can
change ->cb at any time.
Now we can execute a read4, update structures based on the result,
and then hijack the list again.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 44 / 50
49. Conclusions and further research
Conclusions
VM breakouts aren’t magic.
Hypervisors are just as vulnerable as anything else.
Device drivers are the weak spot.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 45 / 50
50. Conclusions and further research
Comparing with some past breakouts
2008 “Adventures with a certain Xen vulnerability”, Xen, Invisible Things
Lab
2009 “Cloudburst”, Immunity, VMware
2011 “Software attacks against Intel VT-d technology”, Invisible Things
Lab, Xen
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 46 / 50
51. Conclusions and further research
Possible hardening directions
Sandbox qemu-kvm (work underway well before this talk).
Build qemu-kvm as PIE.
Lazily mmap/mprotect guest RAM?
XOR-encode key function pointers?
More auditing and fuzzing of qemu-kvm.
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 47 / 50
52. Conclusions and further research
Future research directions
Fuzzing/auditing kvm.ko (That x86 emulator sketches me)
Fingerprinting qemu-kvm versions
Searching for infoleaks (Rosenbugs?)
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 48 / 50
53. Demo
It’s demo time
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 49 / 50
54. Demo
Questions?
nelhage@nelhage.com
http://blog.nelhage.com
@nelhage
Nelson Elhage (DEFCON 19) Virtunoid: Breaking out of KVM August 8, 2011 50 / 50