Disco: Running Commodity Operating Systems on Scalable Multiprocessors
VMware started as a grad school project out of Stansford university.
Many servers were under utilized so Project Disco was born.
Yes, VMware was originally called Project Disco.
In fact both GOOGLE and Project Disco started out together as grad projects.
Server virtualization allows multiple virtual machines to run on the same physical server hardware. It increases hardware utilization and enables server consolidation. The benefits of virtualization include higher utilization, decreased provisioning times, load balancing, improved security, and easier disaster recovery. However, virtualization also increases management complexity and physical hardware failures can affect multiple virtual machines.
This slides focuses on Virtualization concepts, types of virtualization, Hypervisors, Evolution of virtualization towards cloud and QEMU-KVM architecture.
Communication takes place between user modules using message passing
Benefits:
Easier to extend a microkernel
Easier to port the operating system to new architectures
More reliable (less code is running in kernel mode)
More secure
Detriments:
Performance overhead of user space to kernel space communication
Virtualization allows multiple operating systems and applications to run on the same physical server at the same time. This increases hardware utilization and flexibility while reducing IT costs. VMware virtualization solutions can reduce energy costs by 80% through server consolidation and powering down unused servers without affecting applications or users. Virtualization makes hardware resources independent of operating systems and applications, treating them as single unified units that can be more easily deployed, maintained, and supported.
Virtualization is a technology that allows multiple operating systems and applications to run on a single physical machine simultaneously. It provides a layer of abstraction between the physical hardware and the applications running on top of it. The document discusses concepts of virtualization like partitioning, full virtualization, paravirtualization, and VMware's product portfolio for data center, desktop, and mobile virtualization.
The document provides an overview of VMware vSphere Replication 5.5. It describes vSphere Replication as a feature of VMware vSphere that continuously replicates running virtual machines to another location for disaster recovery purposes. It replicates only changed blocks in near real-time and ensures application consistency. Administrators can configure replication for individual VMs and retain multiple historical copies for recovery to previous points in time if needed.
This document discusses RAID (Redundant Array of Inexpensive Disks) levels including RAID 0, 1, 5, 6, and 10. It describes the characteristics of each RAID level such as striping, mirroring, parity protection and performance. The advantages and disadvantages of different RAID levels are provided. Additionally, the key differences between software RAID and hardware RAID are outlined. The document concludes that RAID protects against single drive failures except RAID 0 and states the importance of using RAID to ensure data retrieval.
Server virtualization allows multiple virtual machines to run on the same physical server hardware. It increases hardware utilization and enables server consolidation. The benefits of virtualization include higher utilization, decreased provisioning times, load balancing, improved security, and easier disaster recovery. However, virtualization also increases management complexity and physical hardware failures can affect multiple virtual machines.
This slides focuses on Virtualization concepts, types of virtualization, Hypervisors, Evolution of virtualization towards cloud and QEMU-KVM architecture.
Communication takes place between user modules using message passing
Benefits:
Easier to extend a microkernel
Easier to port the operating system to new architectures
More reliable (less code is running in kernel mode)
More secure
Detriments:
Performance overhead of user space to kernel space communication
Virtualization allows multiple operating systems and applications to run on the same physical server at the same time. This increases hardware utilization and flexibility while reducing IT costs. VMware virtualization solutions can reduce energy costs by 80% through server consolidation and powering down unused servers without affecting applications or users. Virtualization makes hardware resources independent of operating systems and applications, treating them as single unified units that can be more easily deployed, maintained, and supported.
Virtualization is a technology that allows multiple operating systems and applications to run on a single physical machine simultaneously. It provides a layer of abstraction between the physical hardware and the applications running on top of it. The document discusses concepts of virtualization like partitioning, full virtualization, paravirtualization, and VMware's product portfolio for data center, desktop, and mobile virtualization.
The document provides an overview of VMware vSphere Replication 5.5. It describes vSphere Replication as a feature of VMware vSphere that continuously replicates running virtual machines to another location for disaster recovery purposes. It replicates only changed blocks in near real-time and ensures application consistency. Administrators can configure replication for individual VMs and retain multiple historical copies for recovery to previous points in time if needed.
This document discusses RAID (Redundant Array of Inexpensive Disks) levels including RAID 0, 1, 5, 6, and 10. It describes the characteristics of each RAID level such as striping, mirroring, parity protection and performance. The advantages and disadvantages of different RAID levels are provided. Additionally, the key differences between software RAID and hardware RAID are outlined. The document concludes that RAID protects against single drive failures except RAID 0 and states the importance of using RAID to ensure data retrieval.
Hypervisors and Virtualization - VMware, Hyper-V, XenServer, and KVMvwchu
With co-presenter Maninder Singh, delivered a presentation about hypervisors and virtualization technology for an independent topic study project for the Operating System Design (EECS 4221) course at York University, Canada in October 2014.
Virtualization, briefly, is the separation of resources or requests for a service from the underlying physical delivery of that service. It is a concept in which access to a single underlying piece of hardware is coordinated so that multiple guest operating systems can share a single piece of hardware, with no guest operating system being aware that it is actually sharing anything at all.
Virtual machines and their architectureMrinmoy Dalal
Virtual machines allow multiple operating systems to run simultaneously on the same physical hardware through virtualization. A hypervisor manages access to physical resources and allocates what each virtual machine needs, providing isolation. This allows different operating systems, applications, and workloads to be securely isolated while sharing the same underlying hardware resources.
VMware vSphere 6.0 - Troubleshooting Training - Day 1Sanjeev Kumar
This document provides an introduction and overview of VMware vSphere: Install, Configure, Manage training course. It discusses how the course aligns with the VCP-Core certification exam blueprint and objectives. It also provides definitions of key data center concepts like tiers and an overview of the evolution of data centers. Finally, it discusses the history and benefits of data center virtualization using VMware technologies like ESXi, virtual machines, and vCenter Server.
This document provides an overview of GPU virtualization including:
1. Defining GPU virtualization and classifying different approaches like passthrough and full virtualization.
2. Describing use cases for GPU virtualization in providing GPU as a service and for applications like machine learning.
3. Highlighting critical techniques used in GPU virtualization like SR-IOV, mediated devices, and managing resources like memory.
This document summarizes a talk on redesigning Xen's memory sharing (grant) mechanism. It proposes moving grant-related hypercalls to guest domains to allow unilateral revocation of grants by domains and enable better reuse of grants. An evaluation shows the redesigned mechanism with grant reuse reduces overhead and improves I/O performance compared to the traditional approach.
Xen is a virtual machine monitor that allows multiple guest operating systems to run simultaneously on the same computer hardware. It uses paravirtualization, where the guest operating systems are modified to interface with the hypervisor rather than directly with hardware. This allows Xen to provide isolation between guest virtual machines while maintaining high performance. Xen introduces a new privileged level, where the hypervisor runs at a higher privilege than the guest operating systems. This allows Xen to maintain control over CPU, memory, and I/O access between virtual machines.
The document discusses a mid-evaluation of a major project comparing several hypervisors. It will compare Xen, KVM, VMware, and VirtualBox based on their technical differences and performance benchmarks. The benchmarks will test CPU speed, network speed, I/O speed, and performance running various server workloads. This comparison will help determine the best hypervisor for a given virtualization situation. Key factors that will be compared include OS support, security, CPU speed, network speed, I/O speed, and response times.
The document provides details about installation, upgrade, hardware requirements, supported operating systems and databases for VMware ESX Server 3.0.1 and Virtual Center 2.0.1. It discusses the major components, minimum hardware requirements for VirtualCenter Server and Virtual Infrastructure Client. It also lists the supported databases, file extensions, differences between ESX and GSX, current ESX hardware version and various virtualization products.
This document discusses virtual machines. It defines a virtual machine as software that creates a virtualized environment between the computer hardware and the end user, allowing the user to operate software. Virtual machines implement full virtual computer hardware and are created through a combination of real hardware and virtualizing software. There are two types of virtual machines: process virtual machines and system virtual machines. Virtual machines provide benefits like partitioning and isolation, allowing multiple operating systems and applications to run securely side-by-side on the same physical machine.
Virtualization Concepts
This document discusses various types of virtualization including server, storage, network, and application virtualization. It begins with defining virtualization as creating virtual versions of hardware platforms, operating systems, storage devices, and network resources. Server virtualization partitions physical servers into multiple virtual servers. Storage virtualization pools physical storage to appear as a single device. Network virtualization combines network resources into software-defined logical networks. Application virtualization encapsulates programs from the underlying OS. The document then covers the history of virtualization in mainframes and personal computers and dives deeper into specific virtualization types.
The document discusses Virtio, an interface for virtualized I/O devices. It introduces Virtio's architecture, which involves Virtqueues and Vrings to facilitate communication between guest drivers and hypervisor-level device emulators. It outlines the five Virtio APIs for adding, kicking, getting buffers and enabling/disabling callbacks. It also provides an overview of steps for adding new Virtio devices and drivers.
VMware provides server virtualization software that allows multiple virtual machines to run on a single physical server. The document discusses VMware's history and products, outlines the benefits of server virtualization such as increased hardware utilization and reduced costs, and describes various VMware solutions like VMotion, HA, and DRS that provide capabilities like live migration of VMs and high availability of workloads. It also presents statistics on VMware's business and customer base and shares examples of how organizations have benefited from virtualization.
Virtualization - Kernel Virtual Machine (KVM)Wan Leung Wong
KVM is a virtualization solution that leverages hardware virtualization extensions like Intel VT or AMD-V for full virtualization. It uses kernel modules, QEMU, and libvirt to manage virtual machines. KVM is widely used in Linux distributions and offers benefits like isolation, emulation, and easy migration. It allows hosting multiple virtual machines with their images stored on a shared LVM storage that is connected via iSCSI. Management tools like virsh and virt-manager can be used to control the virtual machines from the command line or GUI.
Virtualization involves dividing the resources of a computer into multiple execution environments. It has been used since the 1960s and there are several types including hardware, desktop, and language virtualization. The key components of a virtualization architecture are the hypervisor and guest/host machines. Hypervisors allow multiple operating systems to run on a single system and can be type 1 (runs directly on hardware) or type 2 (runs within an operating system). Virtualization provides benefits but also has limitations related to resource allocation and compatibility that vendors continue working to address.
A virtual machine (VM) is a software program or operating system that not only exhibits the behavior of a separate computer but is also capable of performing tasks such as running applications and programs like a separate computer.
This presentation introduces VMware vRealize Log Insight, a log management platform for collecting and analyzing logs from VMware environments and beyond. It discusses use cases for log analysis including troubleshooting, monitoring, and compliance. It provides examples of queries such as identifying privileged user activity, VM configuration changes, and performance issues. Finally, it outlines architectural considerations for deploying Log Insight at scale within an enterprise.
This document discusses various types of computer storage media and storage devices. It describes internal hard disks, external storage, solid state drives, optical discs, flash memory, tape storage, and cloud storage. The document provides details on the characteristics, uses, and care of different storage options and how they are suited for different types of computer users from home to enterprise levels.
This document discusses virtualization and cloud computing. It begins by explaining the motivations for virtualization, such as managing resources more efficiently and improving performance, security, and reliability. It then covers key concepts in virtualization like virtual machines, hypervisors, paravirtualization, and hardware assisted virtualization. A major virtualization technology discussed is Xen, which uses paravirtualization. The document also briefly mentions Linux containers as a form of lightweight virtualization.
Virtualization allows resources like processors, memory, storage and machines to be divided into multiple isolated virtual instances that can each run their own operating systems and applications. This is done through virtualization software that manages the underlying physical resources and presents a separate virtual environment to each virtual instance. Popular virtualization platforms include Xen, VMWare, and Microsoft Virtual Server which allow multiple virtual machines to run simultaneously on a single physical server.
Hypervisors and Virtualization - VMware, Hyper-V, XenServer, and KVMvwchu
With co-presenter Maninder Singh, delivered a presentation about hypervisors and virtualization technology for an independent topic study project for the Operating System Design (EECS 4221) course at York University, Canada in October 2014.
Virtualization, briefly, is the separation of resources or requests for a service from the underlying physical delivery of that service. It is a concept in which access to a single underlying piece of hardware is coordinated so that multiple guest operating systems can share a single piece of hardware, with no guest operating system being aware that it is actually sharing anything at all.
Virtual machines and their architectureMrinmoy Dalal
Virtual machines allow multiple operating systems to run simultaneously on the same physical hardware through virtualization. A hypervisor manages access to physical resources and allocates what each virtual machine needs, providing isolation. This allows different operating systems, applications, and workloads to be securely isolated while sharing the same underlying hardware resources.
VMware vSphere 6.0 - Troubleshooting Training - Day 1Sanjeev Kumar
This document provides an introduction and overview of VMware vSphere: Install, Configure, Manage training course. It discusses how the course aligns with the VCP-Core certification exam blueprint and objectives. It also provides definitions of key data center concepts like tiers and an overview of the evolution of data centers. Finally, it discusses the history and benefits of data center virtualization using VMware technologies like ESXi, virtual machines, and vCenter Server.
This document provides an overview of GPU virtualization including:
1. Defining GPU virtualization and classifying different approaches like passthrough and full virtualization.
2. Describing use cases for GPU virtualization in providing GPU as a service and for applications like machine learning.
3. Highlighting critical techniques used in GPU virtualization like SR-IOV, mediated devices, and managing resources like memory.
This document summarizes a talk on redesigning Xen's memory sharing (grant) mechanism. It proposes moving grant-related hypercalls to guest domains to allow unilateral revocation of grants by domains and enable better reuse of grants. An evaluation shows the redesigned mechanism with grant reuse reduces overhead and improves I/O performance compared to the traditional approach.
Xen is a virtual machine monitor that allows multiple guest operating systems to run simultaneously on the same computer hardware. It uses paravirtualization, where the guest operating systems are modified to interface with the hypervisor rather than directly with hardware. This allows Xen to provide isolation between guest virtual machines while maintaining high performance. Xen introduces a new privileged level, where the hypervisor runs at a higher privilege than the guest operating systems. This allows Xen to maintain control over CPU, memory, and I/O access between virtual machines.
The document discusses a mid-evaluation of a major project comparing several hypervisors. It will compare Xen, KVM, VMware, and VirtualBox based on their technical differences and performance benchmarks. The benchmarks will test CPU speed, network speed, I/O speed, and performance running various server workloads. This comparison will help determine the best hypervisor for a given virtualization situation. Key factors that will be compared include OS support, security, CPU speed, network speed, I/O speed, and response times.
The document provides details about installation, upgrade, hardware requirements, supported operating systems and databases for VMware ESX Server 3.0.1 and Virtual Center 2.0.1. It discusses the major components, minimum hardware requirements for VirtualCenter Server and Virtual Infrastructure Client. It also lists the supported databases, file extensions, differences between ESX and GSX, current ESX hardware version and various virtualization products.
This document discusses virtual machines. It defines a virtual machine as software that creates a virtualized environment between the computer hardware and the end user, allowing the user to operate software. Virtual machines implement full virtual computer hardware and are created through a combination of real hardware and virtualizing software. There are two types of virtual machines: process virtual machines and system virtual machines. Virtual machines provide benefits like partitioning and isolation, allowing multiple operating systems and applications to run securely side-by-side on the same physical machine.
Virtualization Concepts
This document discusses various types of virtualization including server, storage, network, and application virtualization. It begins with defining virtualization as creating virtual versions of hardware platforms, operating systems, storage devices, and network resources. Server virtualization partitions physical servers into multiple virtual servers. Storage virtualization pools physical storage to appear as a single device. Network virtualization combines network resources into software-defined logical networks. Application virtualization encapsulates programs from the underlying OS. The document then covers the history of virtualization in mainframes and personal computers and dives deeper into specific virtualization types.
The document discusses Virtio, an interface for virtualized I/O devices. It introduces Virtio's architecture, which involves Virtqueues and Vrings to facilitate communication between guest drivers and hypervisor-level device emulators. It outlines the five Virtio APIs for adding, kicking, getting buffers and enabling/disabling callbacks. It also provides an overview of steps for adding new Virtio devices and drivers.
VMware provides server virtualization software that allows multiple virtual machines to run on a single physical server. The document discusses VMware's history and products, outlines the benefits of server virtualization such as increased hardware utilization and reduced costs, and describes various VMware solutions like VMotion, HA, and DRS that provide capabilities like live migration of VMs and high availability of workloads. It also presents statistics on VMware's business and customer base and shares examples of how organizations have benefited from virtualization.
Virtualization - Kernel Virtual Machine (KVM)Wan Leung Wong
KVM is a virtualization solution that leverages hardware virtualization extensions like Intel VT or AMD-V for full virtualization. It uses kernel modules, QEMU, and libvirt to manage virtual machines. KVM is widely used in Linux distributions and offers benefits like isolation, emulation, and easy migration. It allows hosting multiple virtual machines with their images stored on a shared LVM storage that is connected via iSCSI. Management tools like virsh and virt-manager can be used to control the virtual machines from the command line or GUI.
Virtualization involves dividing the resources of a computer into multiple execution environments. It has been used since the 1960s and there are several types including hardware, desktop, and language virtualization. The key components of a virtualization architecture are the hypervisor and guest/host machines. Hypervisors allow multiple operating systems to run on a single system and can be type 1 (runs directly on hardware) or type 2 (runs within an operating system). Virtualization provides benefits but also has limitations related to resource allocation and compatibility that vendors continue working to address.
A virtual machine (VM) is a software program or operating system that not only exhibits the behavior of a separate computer but is also capable of performing tasks such as running applications and programs like a separate computer.
This presentation introduces VMware vRealize Log Insight, a log management platform for collecting and analyzing logs from VMware environments and beyond. It discusses use cases for log analysis including troubleshooting, monitoring, and compliance. It provides examples of queries such as identifying privileged user activity, VM configuration changes, and performance issues. Finally, it outlines architectural considerations for deploying Log Insight at scale within an enterprise.
This document discusses various types of computer storage media and storage devices. It describes internal hard disks, external storage, solid state drives, optical discs, flash memory, tape storage, and cloud storage. The document provides details on the characteristics, uses, and care of different storage options and how they are suited for different types of computer users from home to enterprise levels.
This document discusses virtualization and cloud computing. It begins by explaining the motivations for virtualization, such as managing resources more efficiently and improving performance, security, and reliability. It then covers key concepts in virtualization like virtual machines, hypervisors, paravirtualization, and hardware assisted virtualization. A major virtualization technology discussed is Xen, which uses paravirtualization. The document also briefly mentions Linux containers as a form of lightweight virtualization.
Virtualization allows resources like processors, memory, storage and machines to be divided into multiple isolated virtual instances that can each run their own operating systems and applications. This is done through virtualization software that manages the underlying physical resources and presents a separate virtual environment to each virtual instance. Popular virtualization platforms include Xen, VMWare, and Microsoft Virtual Server which allow multiple virtual machines to run simultaneously on a single physical server.
The virtualization can be described in a generic way as a separation of the service request from the underlying physical delivery of that service. In computer virtualization, an additional layer called hypervisor is typically added between the hardware and the operating system. The hypervisor layer is responsible for both sharing of hardware resource and the enforcement of mandatory access control rules based on the available hardware resources.
There are three types of virtualization: full virtualization, para-virtualization and operating system level (OS-level) virtualization.
This document discusses virtualization and cloud computing. It begins by explaining the motivations for virtualization, such as managing resources more efficiently and allowing live migration of virtual machines. It then covers key concepts in virtualization like virtual machine monitors, full virtualization versus paravirtualization, and hardware assisted virtualization. Specific virtualization technologies are discussed like Xen, a popular open source hypervisor based on paravirtualization. The document also covers challenges in virtualizing the x86 CPU architecture.
Virtual PC allows multiple operating systems to run simultaneously on a single computer by creating separate virtual machines (VMs). Each VM acts like a fully independent computer with its own processors, memory, storage, and network connections. This enables testing and running incompatible operating systems side by side. Virtual PC provides features like shared clipboard, drag and drop between VMs and the host system. It is useful for rapid testing and reconfiguration by developers, support professionals, and architects. A new product, Microsoft Virtual Server, will provide similar virtualization capabilities for servers.
The document discusses cloud middleware and various cloud platforms. It provides details about Eucalyptus, OpenStack, Ubuntu Enterprise Cloud, Amazon EC2, Google App Engine, and their components. Eucalyptus is an open-source software for building private and hybrid clouds. It implements Amazon Web Services APIs and interfaces. OpenStack is an open-source cloud platform consisting of modules like Nova, Swift, Glance, Keystone and Horizon. Ubuntu Enterprise Cloud is a commercial version of Eucalyptus that provides tools for managing infrastructure and users. Amazon EC2 and Google App Engine are commercial cloud platforms that allow deploying and scaling web applications.
What is an operating System Structure?
We want a clear structure to let us apply an operating system to our particular needs because operating systems have complex structures. It is easier to create an operating system in pieces, much as we break down larger issues into smaller, more manageable subproblems. Every segment is also a part of the operating system. Operating system structure can be thought of as the strategy for connecting and incorporating various operating system components within the kernel. Operating systems are implemented using many types of structures, as will be discussed below:
SIMPLE STRUCTURE
It is the most straightforward operating system structure, but it lacks definition and is only appropriate for usage with tiny and restricted systems. Since the interfaces and degrees of functionality in this structure are clearly defined, programs are able to access I/O routines, which may result in unauthorized access to I/O procedures.
This document discusses virtualization concepts and provides definitions and background information. It begins with references and sources on virtualization. It then defines virtualization and key terms like virtual machine monitor and guest/host. It covers the origins and principles of virtualization, how virtualization was rediscovered, and discusses various virtualization architectures, interfaces, types of virtual machine monitors, and strategies for virtualizing systems like the IA-32 architecture. It concludes by discussing memory management challenges in virtualization.
This document discusses virtualization concepts and provides definitions and background information. It begins with references and sources on virtualization. It then defines virtualization and key terms like virtual machine monitor and guest/host. It covers the origins and principles of virtualization, how virtualization was rediscovered, and discusses various virtualization architectures, interfaces, types of virtual machine monitors, and strategies for virtualizing systems like the IA-32 architecture. It concludes by discussing memory management challenges in virtualization.
Virtualization allows multiple virtual machines to run on a single physical machine. It relies on hardware advances like multi-core CPUs and networking improvements. Virtualization works by either emulating hardware, trapping privileged instructions and emulating them, dynamic binary translation, or paravirtualization where the guest OS is aware it is virtualized. I/O virtualization can emulate devices, use paravirtualized drivers, or directly assign devices to VMs. This enables server consolidation and efficient utilization of resources in cloud computing.
Virtual Server 2005 Overview Rich McBrine, CISSPwebhostingguy
Virtual Server 2005 provides virtualization software that allows multiple virtual machines to run simultaneously on a single physical server. This improves hardware utilization and reduces costs. Key benefits include increased consolidation to reduce hardware needs, single point of management, and a test environment without needing additional physical hardware. The document discusses Virtual Server 2005 architecture, capabilities like live migration between physical servers, and competitive pricing compared to other virtualization platforms.
This document discusses different levels and approaches to virtualization including instruction set architecture level, hardware abstraction level, operating system level, library support level, and user-application level virtualization. It also covers virtualization of CPU, memory, I/O devices, and virtual clusters. Key points include hardware-assisted virtualization using features like VT-x, two-stage memory mapping using EPT, different approaches to I/O virtualization, and live VM migration involving transferring memory and synchronizing state changes.
Inroduction to Virtualization and Video Playback during a Live Migrated Virtual Machine hosting the server with its time analysis.
OS- Ubuntu
Hypervisor- KVM
The document discusses virtualization and provides an overview of key concepts and types of virtualization. It defines virtualization as using software to create virtual versions of hardware resources like servers, storage, and networks. The summary discusses the main types of virtualization covered in the document:
1. Server virtualization allows consolidating multiple physical servers onto one server by virtualizing hardware resources.
2. OS virtualization works at the OS layer, virtualizing a physical server and OS into isolated partitions that function like separate servers.
3. Hardware emulation uses a hypervisor to emulate hardware for guest operating systems, allowing different OS types on one server.
4. Paravirtualization coordinates hardware access without fully emulating
The document discusses virtual machine systems and how they allow multiple operating systems to run simultaneously on the same hardware. A virtual machine provides an interface identical to the underlying hardware, creating the illusion of multiple processors. This allows different operating systems to run independently in virtual environments without directly sharing resources. Early examples include CP/67 which virtualized an IBM 360/67 computer. Modern virtualization software like VMware replicates hardware resources to support multiple guest operating systems on a single physical machine.
Virtual Server 2005 provides virtualization capabilities that allow organizations to consolidate servers, improve hardware utilization, and reduce costs. Key benefits include using underutilized processors in a shared environment, single point of management, reducing hardware sprawl and aging equipment costs, and testing environments. Early customers report savings of up to $7.5 million over 5 years. Virtual Server 2005 supports Windows guest operating systems and allows for workload management and resource allocation policies. Migration of physical servers to a virtual environment can be automated using the Virtual Server Migration Toolkit.
Virtual Server 2005 provides virtualization capabilities that allow consolidating servers to reduce costs. It offers key benefits like increased hardware utilization, single point of management, and test environments. Customers are seeing savings of millions of dollars by deploying hundreds to thousands of virtual machines. The architecture isolates guest operating systems and applications from physical hardware through virtual devices and resource management. This enables easy migration of workloads between physical and virtual servers with minimal downtime.
Virtualization is a framework that divides computer resources into execution environments. It abstracts physical hardware from the operating system to improve resource utilization and flexibility. Virtualization technologies enable multiple operating systems and applications to run simultaneously on a single physical server by simulating virtual machines. While not a new concept dating back to IBM in the 1960s, modern virtualization platforms like VMware vSphere transform hardware resources into fully functional virtual machines running their own operating systems. This improves efficiency, fault tolerance, and simplifies management and provisioning of computer systems.
Chapter 5 – Cloud Resource
Virtualization
Contents
Virtualization.
Layering and virtualization.
Virtual machine monitor.
Virtual machine.
Performance and security isolation.
Architectural support for virtualization.
x86 support for virtualization.
Full and paravirtualization.
Xen 1.0 and Xen 2.0.
Performance comparison of virtual machine monitors.
The darker side of virtualization.
Software fault isolation.
Cloud Computing: Theory and Practice. Chapter 5 2 Dan C. Marinescu
Motivation
There are many physical realizations of the fundamental
abstractions necessary to describe the operation of a computing
systems.
Interpreters.
Memory.
Communications links.
Virtualization is a basic tenet of cloud computing, it simplifies the
management of physical resources for the three abstractions.
The state of a virtual machine (VM) running under a virtual machine
monitor (VMM) can de saved and migrated to another server to
balance the load.
Virtualization allows users to operate in environments they are
familiar with, rather than forcing them to idiosyncratic ones.
Cloud Computing: Theory and Practice.
Chapter 5 3 Dan C. Marinescu
Motivation (cont’d)
Cloud resource virtualization is important for:
System security, as it allows isolation of services running on
the same hardware.
Performance and reliability, as it allows applications to migrate
from one platform to another.
The development and management of services offered by a
provider.
Performance isolation.
Cloud Computing: Theory and Practice.
Chapter 5 4 Dan C. Marinescu
Virtualization
Simulates the interface to a physical object by:
Multiplexing: creates multiple virtual objects from one instance
of a physical object. Example - a processor is multiplexed
among a number of processes or threads.
Aggregation: creates one virtual object from multiple physical
objects. Example - a number of physical disks are aggregated
into a RAID disk.
Emulation: constructs a virtual object from a different type of a
physical object. Example - a physical disk emulates a Random
Access Memory (RAM).
Multiplexing and emulation. Examples - virtual memory with
paging multiplexes real memory and disk; a virtual address
emulates a real address.
Cloud Computing: Theory and Practice.
Chapter 5 5 Dan C. Marinescu
Layering
Layering – a common approach to manage system complexity.
Minimizes the interactions among the subsystems of a complex
system.
Simplifies the description of the subsystems; each subsystem is
abstracted through its interfaces with the other subsystems.
We are able to design, implement, and modify the individual
subsystems independently.
Layering in a computer system.
Hardware.
Software.
Operating system.
Libraries.
Applications.
.
Chapter 5 – Cloud Resource
Virtualization
Contents
Virtualization.
Layering and virtualization.
Virtual machine monitor.
Virtual machine.
Performance and security isolation.
Architectural support for virtualization.
x86 support for virtualization.
Full and paravirtualization.
Xen 1.0 and Xen 2.0.
Performance comparison of virtual machine monitors.
The darker side of virtualization.
Software fault isolation.
Cloud Computing: Theory and Practice. Chapter 5 2 Dan C. Marinescu
Motivation
There are many physical realizations of the fundamental
abstractions necessary to describe the operation of a computing
systems.
Interpreters.
Memory.
Communications links.
Virtualization is a basic tenet of cloud computing, it simplifies the
management of physical resources for the three abstractions.
The state of a virtual machine (VM) running under a virtual machine
monitor (VMM) can de saved and migrated to another server to
balance the load.
Virtualization allows users to operate in environments they are
familiar with, rather than forcing them to idiosyncratic ones.
Cloud Computing: Theory and Practice.
Chapter 5 3 Dan C. Marinescu
Motivation (cont’d)
Cloud resource virtualization is important for:
System security, as it allows isolation of services running on
the same hardware.
Performance and reliability, as it allows applications to migrate
from one platform to another.
The development and management of services offered by a
provider.
Performance isolation.
Cloud Computing: Theory and Practice.
Chapter 5 4 Dan C. Marinescu
Virtualization
Simulates the interface to a physical object by:
Multiplexing: creates multiple virtual objects from one instance
of a physical object. Example - a processor is multiplexed
among a number of processes or threads.
Aggregation: creates one virtual object from multiple physical
objects. Example - a number of physical disks are aggregated
into a RAID disk.
Emulation: constructs a virtual object from a different type of a
physical object. Example - a physical disk emulates a Random
Access Memory (RAM).
Multiplexing and emulation. Examples - virtual memory with
paging multiplexes real memory and disk; a virtual address
emulates a real address.
Cloud Computing: Theory and Practice.
Chapter 5 5 Dan C. Marinescu
Layering
Layering – a common approach to manage system complexity.
Minimizes the interactions among the subsystems of a complex
system.
Simplifies the description of the subsystems; each subsystem is
abstracted through its interfaces with the other subsystems.
We are able to design, implement, and modify the individual
subsystems independently.
Layering in a computer system.
Hardware.
Software.
Operating system.
Libraries.
Applications.
.
Similar to Disco: Running Commodity Operating Systems on Scalable Multiprocessors Disco (20)
Chad Sakac, VMworld 2012
The latest in IT transformation at EMC (and also VMware).
Topics discussed:
What is the “Software Defined Datacenter” doing to the primary storage and backup landscape?
What is EMC doing about it?
What are the latest updates in integration (with everything VMware does) today and tomorrow?
VBLOCK produces, delivers, and manages IT infrastructure as a service. It provides pre-engineered and integrated compute, storage, and networking solutions from Cisco and EMC. VBLOCK platforms range from small entry-level configurations to large-scale solutions. It aims to accelerate customers' virtualization journeys and help them produce and consume IT as a service through standardized, automated, and policy-driven infrastructure. VBLOCK solutions are supported globally through dedicated joint support labs and follow-the-sun support.
The document discusses the opportunities and requirements of modern IT from an EMC perspective. It focuses on guiding customers towards cloud computing and IT as a service. EMC sees opportunities in transforming traditional IT infrastructure and applications to private, public and hybrid cloud models. The document provides examples of how EMC can help optimize data center execution and migrate applications and data to more advanced infrastructure like flash storage. It also discusses new roles and skills needed for IT as a service environments and how EMC training programs can help with the transition.
Information Wars - Starring Symmetrix VMAXMagnus Backman
The document is a marketing presentation that highlights the Symmetrix VMAX storage solution. It depicts Symmetrix as the original and dominant storage solution, with imagery suggesting it can consolidate and rule all other storage. The presentation lists several TV shows and movies where Symmetrix has appeared, positioning it as a leading solution seen in major entertainment franchises. It concludes by announcing Symmetrix VMAX will soon be in a datacenter near the viewer.
The document discusses EMC's perspective on the future of IT and the transition to cloud computing. It describes how EMC is helping customers transform their infrastructure through strategies like consolidating and virtualizing systems, building private and hybrid clouds, and offering IT as a service. The document also previews some of EMC's upcoming storage platforms that will be able to run as virtual machines in cloud environments.
The Department of Veteran Affairs (VA) invited Taylor Paschal, Knowledge & Information Management Consultant at Enterprise Knowledge, to speak at a Knowledge Management Lunch and Learn hosted on June 12, 2024. All Office of Administration staff were invited to attend and received professional development credit for participating in the voluntary event.
The objectives of the Lunch and Learn presentation were to:
- Review what KM ‘is’ and ‘isn’t’
- Understand the value of KM and the benefits of engaging
- Define and reflect on your “what’s in it for me?”
- Share actionable ways you can participate in Knowledge - - Capture & Transfer
Freshworks Rethinks NoSQL for Rapid Scaling & Cost-EfficiencyScyllaDB
Freshworks creates AI-boosted business software that helps employees work more efficiently and effectively. Managing data across multiple RDBMS and NoSQL databases was already a challenge at their current scale. To prepare for 10X growth, they knew it was time to rethink their database strategy. Learn how they architected a solution that would simplify scaling while keeping costs under control.
Taking AI to the Next Level in Manufacturing.pdfssuserfac0301
Read Taking AI to the Next Level in Manufacturing to gain insights on AI adoption in the manufacturing industry, such as:
1. How quickly AI is being implemented in manufacturing.
2. Which barriers stand in the way of AI adoption.
3. How data quality and governance form the backbone of AI.
4. Organizational processes and structures that may inhibit effective AI adoption.
6. Ideas and approaches to help build your organization's AI strategy.
"Choosing proper type of scaling", Olena SyrotaFwdays
Imagine an IoT processing system that is already quite mature and production-ready and for which client coverage is growing and scaling and performance aspects are life and death questions. The system has Redis, MongoDB, and stream processing based on ksqldb. In this talk, firstly, we will analyze scaling approaches and then select the proper ones for our system.
For the full video of this presentation, please visit: https://www.edge-ai-vision.com/2024/06/temporal-event-neural-networks-a-more-efficient-alternative-to-the-transformer-a-presentation-from-brainchip/
Chris Jones, Director of Product Management at BrainChip , presents the “Temporal Event Neural Networks: A More Efficient Alternative to the Transformer” tutorial at the May 2024 Embedded Vision Summit.
The expansion of AI services necessitates enhanced computational capabilities on edge devices. Temporal Event Neural Networks (TENNs), developed by BrainChip, represent a novel and highly efficient state-space network. TENNs demonstrate exceptional proficiency in handling multi-dimensional streaming data, facilitating advancements in object detection, action recognition, speech enhancement and language model/sequence generation. Through the utilization of polynomial-based continuous convolutions, TENNs streamline models, expedite training processes and significantly diminish memory requirements, achieving notable reductions of up to 50x in parameters and 5,000x in energy consumption compared to prevailing methodologies like transformers.
Integration with BrainChip’s Akida neuromorphic hardware IP further enhances TENNs’ capabilities, enabling the realization of highly capable, portable and passively cooled edge devices. This presentation delves into the technical innovations underlying TENNs, presents real-world benchmarks, and elucidates how this cutting-edge approach is positioned to revolutionize edge AI across diverse applications.
Must Know Postgres Extension for DBA and Developer during MigrationMydbops
Mydbops Opensource Database Meetup 16
Topic: Must-Know PostgreSQL Extensions for Developers and DBAs During Migration
Speaker: Deepak Mahto, Founder of DataCloudGaze Consulting
Date & Time: 8th June | 10 AM - 1 PM IST
Venue: Bangalore International Centre, Bangalore
Abstract: Discover how PostgreSQL extensions can be your secret weapon! This talk explores how key extensions enhance database capabilities and streamline the migration process for users moving from other relational databases like Oracle.
Key Takeaways:
* Learn about crucial extensions like oracle_fdw, pgtt, and pg_audit that ease migration complexities.
* Gain valuable strategies for implementing these extensions in PostgreSQL to achieve license freedom.
* Discover how these key extensions can empower both developers and DBAs during the migration process.
* Don't miss this chance to gain practical knowledge from an industry expert and stay updated on the latest open-source database trends.
Mydbops Managed Services specializes in taking the pain out of database management while optimizing performance. Since 2015, we have been providing top-notch support and assistance for the top three open-source databases: MySQL, MongoDB, and PostgreSQL.
Our team offers a wide range of services, including assistance, support, consulting, 24/7 operations, and expertise in all relevant technologies. We help organizations improve their database's performance, scalability, efficiency, and availability.
Contact us: info@mydbops.com
Visit: https://www.mydbops.com/
Follow us on LinkedIn: https://in.linkedin.com/company/mydbops
For more details and updates, please follow up the below links.
Meetup Page : https://www.meetup.com/mydbops-databa...
Twitter: https://twitter.com/mydbopsofficial
Blogs: https://www.mydbops.com/blog/
Facebook(Meta): https://www.facebook.com/mydbops/
How information systems are built or acquired puts information, which is what they should be about, in a secondary place. Our language adapted accordingly, and we no longer talk about information systems but applications. Applications evolved in a way to break data into diverse fragments, tightly coupled with applications and expensive to integrate. The result is technical debt, which is re-paid by taking even bigger "loans", resulting in an ever-increasing technical debt. Software engineering and procurement practices work in sync with market forces to maintain this trend. This talk demonstrates how natural this situation is. The question is: can something be done to reverse the trend?
"Frontline Battles with DDoS: Best practices and Lessons Learned", Igor IvaniukFwdays
At this talk we will discuss DDoS protection tools and best practices, discuss network architectures and what AWS has to offer. Also, we will look into one of the largest DDoS attacks on Ukrainian infrastructure that happened in February 2022. We'll see, what techniques helped to keep the web resources available for Ukrainians and how AWS improved DDoS protection for all customers based on Ukraine experience
High performance Serverless Java on AWS- GoTo Amsterdam 2024Vadym Kazulkin
Java is for many years one of the most popular programming languages, but it used to have hard times in the Serverless community. Java is known for its high cold start times and high memory footprint, comparing to other programming languages like Node.js and Python. In this talk I'll look at the general best practices and techniques we can use to decrease memory consumption, cold start times for Java Serverless development on AWS including GraalVM (Native Image) and AWS own offering SnapStart based on Firecracker microVM snapshot and restore and CRaC (Coordinated Restore at Checkpoint) runtime hooks. I'll also provide a lot of benchmarking on Lambda functions trying out various deployment package sizes, Lambda memory settings, Java compilation options and HTTP (a)synchronous clients and measure their impact on cold and warm start times.
AppSec PNW: Android and iOS Application Security with MobSFAjin Abraham
Mobile Security Framework - MobSF is a free and open source automated mobile application security testing environment designed to help security engineers, researchers, developers, and penetration testers to identify security vulnerabilities, malicious behaviours and privacy concerns in mobile applications using static and dynamic analysis. It supports all the popular mobile application binaries and source code formats built for Android and iOS devices. In addition to automated security assessment, it also offers an interactive testing environment to build and execute scenario based test/fuzz cases against the application.
This talk covers:
Using MobSF for static analysis of mobile applications.
Interactive dynamic security assessment of Android and iOS applications.
Solving Mobile app CTF challenges.
Reverse engineering and runtime analysis of Mobile malware.
How to shift left and integrate MobSF/mobsfscan SAST and DAST in your build pipeline.
Northern Engraving | Modern Metal Trim, Nameplates and Appliance PanelsNorthern Engraving
What began over 115 years ago as a supplier of precision gauges to the automotive industry has evolved into being an industry leader in the manufacture of product branding, automotive cockpit trim and decorative appliance trim. Value-added services include in-house Design, Engineering, Program Management, Test Lab and Tool Shops.
What is an RPA CoE? Session 1 – CoE VisionDianaGray10
In the first session, we will review the organization's vision and how this has an impact on the COE Structure.
Topics covered:
• The role of a steering committee
• How do the organization’s priorities determine CoE Structure?
Speaker:
Chris Bolin, Senior Intelligent Automation Architect Anika Systems
zkStudyClub - LatticeFold: A Lattice-based Folding Scheme and its Application...Alex Pruden
Folding is a recent technique for building efficient recursive SNARKs. Several elegant folding protocols have been proposed, such as Nova, Supernova, Hypernova, Protostar, and others. However, all of them rely on an additively homomorphic commitment scheme based on discrete log, and are therefore not post-quantum secure. In this work we present LatticeFold, the first lattice-based folding protocol based on the Module SIS problem. This folding protocol naturally leads to an efficient recursive lattice-based SNARK and an efficient PCD scheme. LatticeFold supports folding low-degree relations, such as R1CS, as well as high-degree relations, such as CCS. The key challenge is to construct a secure folding protocol that works with the Ajtai commitment scheme. The difficulty, is ensuring that extracted witnesses are low norm through many rounds of folding. We present a novel technique using the sumcheck protocol to ensure that extracted witnesses are always low norm no matter how many rounds of folding are used. Our evaluation of the final proof system suggests that it is as performant as Hypernova, while providing post-quantum security.
Paper Link: https://eprint.iacr.org/2024/257
Northern Engraving | Nameplate Manufacturing Process - 2024Northern Engraving
Manufacturing custom quality metal nameplates and badges involves several standard operations. Processes include sheet prep, lithography, screening, coating, punch press and inspection. All decoration is completed in the flat sheet with adhesive and tooling operations following. The possibilities for creating unique durable nameplates are endless. How will you create your brand identity? We can help!
Session 1 - Intro to Robotic Process Automation.pdfUiPathCommunity
👉 Check out our full 'Africa Series - Automation Student Developers (EN)' page to register for the full program:
https://bit.ly/Automation_Student_Kickstart
In this session, we shall introduce you to the world of automation, the UiPath Platform, and guide you on how to install and setup UiPath Studio on your Windows PC.
📕 Detailed agenda:
What is RPA? Benefits of RPA?
RPA Applications
The UiPath End-to-End Automation Platform
UiPath Studio CE Installation and Setup
💻 Extra training through UiPath Academy:
Introduction to Automation
UiPath Business Automation Platform
Explore automation development with UiPath Studio
👉 Register here for our upcoming Session 2 on June 20: Introduction to UiPath Studio Fundamentals: https://community.uipath.com/events/details/uipath-lagos-presents-session-2-introduction-to-uipath-studio-fundamentals/
Monitoring and Managing Anomaly Detection on OpenShift.pdfTosin Akinosho
Monitoring and Managing Anomaly Detection on OpenShift
Overview
Dive into the world of anomaly detection on edge devices with our comprehensive hands-on tutorial. This SlideShare presentation will guide you through the entire process, from data collection and model training to edge deployment and real-time monitoring. Perfect for those looking to implement robust anomaly detection systems on resource-constrained IoT/edge devices.
Key Topics Covered
1. Introduction to Anomaly Detection
- Understand the fundamentals of anomaly detection and its importance in identifying unusual behavior or failures in systems.
2. Understanding Edge (IoT)
- Learn about edge computing and IoT, and how they enable real-time data processing and decision-making at the source.
3. What is ArgoCD?
- Discover ArgoCD, a declarative, GitOps continuous delivery tool for Kubernetes, and its role in deploying applications on edge devices.
4. Deployment Using ArgoCD for Edge Devices
- Step-by-step guide on deploying anomaly detection models on edge devices using ArgoCD.
5. Introduction to Apache Kafka and S3
- Explore Apache Kafka for real-time data streaming and Amazon S3 for scalable storage solutions.
6. Viewing Kafka Messages in the Data Lake
- Learn how to view and analyze Kafka messages stored in a data lake for better insights.
7. What is Prometheus?
- Get to know Prometheus, an open-source monitoring and alerting toolkit, and its application in monitoring edge devices.
8. Monitoring Application Metrics with Prometheus
- Detailed instructions on setting up Prometheus to monitor the performance and health of your anomaly detection system.
9. What is Camel K?
- Introduction to Camel K, a lightweight integration framework built on Apache Camel, designed for Kubernetes.
10. Configuring Camel K Integrations for Data Pipelines
- Learn how to configure Camel K for seamless data pipeline integrations in your anomaly detection workflow.
11. What is a Jupyter Notebook?
- Overview of Jupyter Notebooks, an open-source web application for creating and sharing documents with live code, equations, visualizations, and narrative text.
12. Jupyter Notebooks with Code Examples
- Hands-on examples and code snippets in Jupyter Notebooks to help you implement and test anomaly detection models.
"Scaling RAG Applications to serve millions of users", Kevin GoedeckeFwdays
How we managed to grow and scale a RAG application from zero to thousands of users in 7 months. Lessons from technical challenges around managing high load for LLMs, RAGs and Vector databases.
"Scaling RAG Applications to serve millions of users", Kevin Goedecke
Disco: Running Commodity Operating Systems on Scalable Multiprocessors Disco
1. Disco: Running Commodity
Operating Systems on Scalable
Multiprocessors
Edouard Bugnion, Scott Devine, Mendel Rosenblum,
Stanford University, 1997
Presented by Divya Parekh
1
2. Outline
Virtualization
Disco description
Disco performance
Discussion
2
3. Virtualization
“a technique for hiding the physical
characteristics of computing resources from the
way in which other systems, applications, or end
users interact with those resources. This includes
making a single physical resource appear to
function as multiple logical resources; or it can
include making multiple physical resources
appear as a single logical resource”
3
4. Old idea from the 1960s
IBM VM/370 – A VMM for IBM mainframe
Multiple OS environments on expensive hardware
Desirable when few machine around
Popular research idea in 1960s and 1970s
Entire conferences on virtual machine monitors
Hardware/VMM/OS designed together
Interest died out in the 1980s and 1990s
Hardware got more cheaper
Operating systems got more powerful (e.g. multi-user)
4
5. A Return to Virtual Machines
Disco: Stanford research project (SOSP ’97)
Run commodity OSes on scalable multiprocessors
Focus on high-end: NUMA, MIPS, IRIX
Commercial virtual machines for x86 architecture
VMware Workstation (now EMC) (1999-)
Connectix VirtualPC (now Microsoft)
Research virtual machines for x86 architecture
Xen (SOSP ’03)
plex86
OS-level virtualization
FreeBSD Jails, User-mode-linux, UMLinux
5
6. Overview
Virtual Machine
A fully protected and isolated copy of the underlying
physical machine’s hardware. (definition by IBM)”
Virtual Machine Monitor
A thin layer of software that's between the hardware
and the Operating system, virtualizing and managing
all hardware resources.
Also known as “Hypervisor”
6
8. Classification of Virtual
Machines
Type I
VMM is implemented directly on the physical hardware.
VMM performs the scheduling and allocation of the
system’s resources.
IBM VM/370, Disco, VMware’s ESX Server, Xen
Type II
VMMs are built completely on top of a host OS.
The host OS provides resource allocation and standard
execution environment to each “guest OS.”
User-mode Linux (UML), UMLinux
8
9. Non-Virtualizable Architectures
According to Popek and Goldberg,
” an architecture is virtualizable if the set of
sensitive instructions is a subset of the set of
privileged instructions.”
x86
Several instructions can read system state in
register CPL 3 without trapping
MIPS
KSEG0 bypasses TLB, reads physical memory
directly
9
10. Type I contd..
Hardware Support for Virtualization
Figure: The hardware support approach to x86 Virtualization
E.g. Intel Vanderpool/VT and AMD-V/SVM
10
11. Type I contd..
Full Virtualization
Figure : The binary translation approach to x86 Virtualization
E.g. VMware ESX server
11
12. Type I contd..
Paravirtualization
Figure: The Paravirtualization approach to x86 Virtualization
E.g. Xen
12
13. Type II
Hosted VM Architecture
E.g. VMware Workstation, Connectix VirtualPC
13
14. Disco : VMM Prototype
Goals
Extend modern OS to run efficiently on shared
memory multiprocessors without large changes to
the OS.
A VMM built to run multiple copies of Silicon
Graphics IRIX operating system on a Stanford
Flash shared memory multiprocessor.
14
15. Problem Description
Multiprocessor in the market (1990s)
Innovative Hardware
Hardware faster than System Software
Customized OS are late, incompatible, and
possibly bug
Commodity OS not suited for multiprocessors
Do not scale cause of lock contention, memory
architecture
Do not isolate/contain faults
More Processors More failures
15
16. Solution to the problems
Resource-intensive Modification of OS (hard and
time consuming, increase in size, etc)
Make a Virtual Machine Monitor (software)
between OS and Hardware to resolve the problem
16
17. Two opposite Way for System
Software
Address these challenges in the operating system:
OS-Intensive
Hive , Hurricane, Cellular-IRIX, etc
innovative, single system image
But large effort.
Hard-partition machine into independent failure units:
OS-light
Sun Enterprise10000 machine
Partial single system image
Cannot dynamically adapt the partitioning
17
18. Return to Virtual Machine
Monitors
One Compromise Way between OS-intensive & OS-
light – VMM
Virtual machine monitors, in combination with
commodity and specialized operating systems, form a
flexible system software solution for these machines
Disco was introduced to allow trading off between
the costs of performance and development cost.
18
20. Advantages of this approach
Scalability
Flexibility
Hide NUMA effect
Fault Containment
Compatibility with legacy applications
20
21. Challenges Facing Virtual
Machines
Overheads
Trap and emulate privileged instructions of
guest OS
Access to I/O devices
Replication of memory in each VM
Resource Management
Lack of information to make good policy
decisions
Communication and Sharing
Stand alone VM’s cannot communicate
21
22. Disco’s Interface
Processors
MIPS R10000 processor
Emulates all instructions, the MMU, trap architecture
Extension to support common processor operations
Enabling/disabling interrupts, accessing privileged registers
Physical memory
Contiguous, starting at address 0
I/O devices
Virtualize devices like I/O, disks, n/w interface exclusive to VM
Physical devices multiplexed by Disco
Special abstractions for SCSI disks and network interfaces
Virtual disks for VMs
Virtual subnet across all virtual machines
22
23. Disco Implementation
Multi threaded shared memory program
Attention to NUMA memory placement, cache aware
data structures and IPC patterns
Code segment of DISCO copied to each flash
processor – data locality
Communicate using shared memory
23
24. Virtual CPUs
Direct Execution
execution of virtual CPU on real CPU
Sets the real machine’s registers to the virtual CPU’s
Jumps to the current PC of the virtual CPU, Direct execution
on the real CPU
Challenges
Detection and fast emulation of operations that cannot be
safely exported to the virtual machine privileged
instructions such as TLB modification and Direct access to
physical memory and I/O devices.
Maintains data structure for each virtual CPU for trap
emulation
Scheduler multiplexes virtual CPU on real processor
24
25. Virtual Physical Memory
Address translation & maintains a physical-to-
machine address (40 bit) mapping.
Virtual machines use physical addresses
Software reloaded translation-lookaside buffer (TLB)
of the MIPS processor
Maintains pmap data structure for each VM –
contains one entry for each physical to virtual
mapping
pmap also has a back pointer to its virtual address to
help invalidate mappings in the TLB
25
26. Contd..
Kernel mode references on MIPS processors access
memory and I/O directly - need to re-link OS code
and data to a mapped address space
MIPS tags each TLB entry with Address space
identifiers (ASID)
ASIDs are not virtualized - TLB need to be flushed on
VM context switches
Increased TLB misses in workloads
Additional Operating system references
VM context switches
TLB misses expensive - create 2nd level software -
TLB . Idea similar to cache?
26
27. NUMA Memory management
Cache misses should be satisfied from local memory
(fast) rather than remote memory (slow)
Dynamic Page Migration and Replication
Pages frequently accessed by one node are migrated
Read-shared pages are replicated among all nodes
Write-shared are not moved, since maintaining consistency
requires remote access anyway
Migration and replacement policy is driven by cache-miss-
counting facility provided by the FLASH hardware
27
28. Transparent Page Replication
1. Two different virtual processors of the same virtual machine logically
read-share the same physical page, but each virtual processor accesses
a local copy.
2. memmap tracks which virtual page references each physical page.
Used during TLB shootdown 28
30. Virtual I/O Devices
Disco intercepts all device accesses from the virtual
machine and forwards them to the physical devices
Special device drivers are added to the guest OS
Disco device provide monitor call interface to pass all
the arguments in single trap
Single VM accessing a device does not require
virtualizing the I/O – only needs to assure exclusivity
30
31. Copy-on-write Disks
Intercept DMA requests to translate the physical
addresses into machine addresses.
Maps machine page as read only to destination
address page of DMA Sharing machine memory
Attempts to modify a shared page will result in a
copy-on-write fault handled internally by the monitor.
Logs are maintained for each VM Modification
Modification made in main memory
Non-persistent disks are copy on write shared
E.g. Kernel text and buffer cache
E.g. File systems root disks
31
32. Transparent Sharing of Pages
Creates a global buffer cache shared across VM's and reduces
memory foot print of the system
32
33. Virtual Network Interface
Virtual subnet and network interface use copy on
write mapping to share the read only pages
Persistent disks can be accessed using standard
system protocol NFS
Provides a global buffer cache that is transparently
shared by independent VMs
33
34. Transparent sharing of pages
over NFS
1. The monitor’s networking device remaps the data page from the source’s
machine address space to the destination’s.
2. The monitor remaps the data page from the driver’s mbuf to the clients
buffer cache. 34
35. Modifications to the IRIX 5.3
OS
Minor changes to kernel code and data
segment – specific to MIPS
Relocate the unmapped segment of the virtual
machine into the mapped supervisor segment of
the processor– Kernel relocation
Disco drivers are same as original device
drivers of IRIX
Patched HAL to use memory loads/stores
instead of privileged instructions
35
36. Modifications to the IRIX 5.3
OS
Added code to HAL to pass hints to monitor
for resource management
New Monitor calls to MMU to request zeroed
page, unused memory reclamation
Changed mbuf management to be page-
aligned
Changed bcopy to use remap (with copy-on-
write)
36
37. SPLASHOS: A specialized OS
Thin specialized library OS, supported
directly by Disco
No need for virtual memory subsystem
since they share address space
Used for the parallel scientific
applications that can span the entire
machine
37
38. Disco: Performance
Experimental Setup
Disco targets the FLASH machine not
available that time
Used SimOS, a machine simulator that
models the hardware of MIPS-based
multiprocessors for the Disco monitor.
Simulator was too slow to allow long work
loads to be studied
38
40. Disco: Performance
Execution Overhead
Pmake overhead due to I/O virtualization, others due to TLB mapping
Reduction of kernel time
40
On average virtualization overhead of 3% to 16%
41. Disco: Performance
Memory Overheads
V: Pmake memory used if there is no sharing
M: Pmake memory used if there is sharing 41
42. Disco: Performance
Scalability
Partitioning of problem into different VM’s increases scalability.
Kernel synchronization time becomes smaller. 42
44. Conclusion
Disco VMM hides NUMA-ness from non-
NUMA aware OS
Disco VMM is low(er) effort
Moderate overhead due to virtualization
44
45. Discussion
Was Disco- VMM done rightly?
Virtual Physical Memory on architectures other
than MIPS
MIPS TLB is software managed
Not sure of how well other OS perform on Disco
since IRIX was designed for MIPS
Not sure how HIVE, Hurricane performs
comparatively
Performance of long workloads on the system
Performance of heterogeneous VMs e.g. Pmake
case
45