Applications of Virtual Machines For Scalable, Reliable, And Interactive Internet Services Ph.D. Oral Examination Amr A. Awadallah June 24 th , 2006 [email_address] http://www.awadallah.com/vmatrix

What is The vMatrix?

Virtual Machine Monitors Virtual Machine Monitor (aka Hypervisor) Real Machine (CPU, Memory, Disks, Network) Virtual Machine 1: vCPU, vMem, vDisk, vNet OS1: Windows 2000 Oracle, IIS Virtual Machine 2: vCPU, vMem, vDisk, vNet OS2: Linux MySQL, Apache

Live Demo of VMware player:

The vMatrix! A network for delivering virtual machines (VMs) between real machines (RMs) running the virtual machine monitor (VMM) software.

A network for delivering virtual machines (VMs) between real machines (RMs) running the virtual machine monitor (VMM) software.

Contribution An overlay network of VMMs (the vMatrix) enables backward-compatible improvement of the scalability, reliability, and interactivity of Internet services. Applications: 1. Dynamic Content Distribution 2. Server Switching 3. Fair placement of Game Servers

An overlay network of VMMs (the vMatrix) enables backward-compatible improvement of the scalability, reliability, and interactivity of Internet services.

First Application: Dynamic Content Distribution Motivation: To enable distribution of dynamic content. (40% of web requests) Definition: Dynamic content is web pages which are constructed by programs that execute on the server at the time a request is made. (e.g. http://maps.yahoo.com)

Motivation: To enable distribution of dynamic content. (40% of web requests)

Today Is Static Mirroring

Dynamic Content Distribution

Two Tier Architecture

Main Problem It is very hard to copy services due to all the dependencies that code has on system libraries, third-party modules, operating systems, and server hardware.

It is very hard to copy services due to all the dependencies that code has on system libraries, third-party modules, operating systems, and server hardware.

Challenges: Mobility (NAT) VMM RM VM Agent VM1 VM2 192.168.1.10 NAT/LB Internet 64.58.77.28 DNS maps.yahoo.com  64.58.77.28

Challenges: Security (VPNs) VMM VM1 VM2 RM VM Agent VPN Intranet DNS 192.168.1.10 172.21.162.9 c009.proxy.yahoo.com maps.yahoo.com  64.58.77.28 NAT/LB Internet 64.58.77.28

Two Tier Challenges Response Time N 1 FRONT END BACK END FRONT END N 1 BACK END N 2 BACK END Perception!

Response Time

Perception!

Two Tier Challenges Availability FRONT END BACK END

Availability

The AJAX Solution

Advantages Faster Response Time Higher Availability Network Bandwidth Savings Backward Compatibility Disadvantage VM files are very large (order of gigabytes)!

Faster Response Time

Higher Availability

Network Bandwidth Savings

Backward Compatibility

VM files are very large (order of gigabytes)!

Second Application: Server Multiplexing Motivation: To allow statistical sharing of servers between services.

Motivation: To allow statistical sharing of servers between services.

Static Servers (ala Circuit Switching) FINANCE SPORTS Load Balancer

Server Switching (ala Packet Switching) FINANCE & SPORTS Load Balancer vMatrix Oracle Key: Peak of Sums < Sum of Peaks

What makes Switching hard ? Software Dependencies Hardware Dependencies People Dependencies

Software Dependencies

Hardware Dependencies

People Dependencies

vMatrix Framework: The Oracle: This is the main control program Hibernation Nest : VM files are suspended and stored here for later retrieval. Production Cluster : Live operational VMs, typically only one VM per RM Loading Chambers : Development and Maintenance Cluster, there could be more than one VM per RM here. RM VMM VM VM VM VM VM VM VM VM VM RM VMM VM VM VM RM VMM VM VM VM RM VMM VM RM VMM VM

Lifecycle of a server VM Loading Chambers Hibernation Nest Instantiated 1. A large number of virtual servers are stored as dormant files on a SAN or Netapps 2. A number of virtual servers are activated in a shared RM so that developers & system admins can upgrade, debug, or patch them. 3. A virtual server is activated on a dedicated RM and exposed to live load.

Sample Implementations: Real Machine (PIII-550MHz, 640MB RAM, 9GB hard disk) VMware ESX VMM Server (consumes 184MB RAM, 3.5GB hard disk and 5% CPU) Virtual Machine exposes a PIII-550MHz with 512MB RAM and 5.5GB hard disk. Operating System: Red Hat Linux 9 Apache Web Server MySQL Database PHP (Hyper Text Processor) PHP-Nuke and osCommerce Internet Services Real Machine (PIII-550MHz, 640MB RAM, 9GB hard disk) VMware ESX VMM Server (consumes 184MB RAM, 3.5GB hard disk and 5% CPU) Virtual Machine exposes a PIII-550MHz with 1024MB RAM and 5.5GB hard disk. Operating System: Yahoo FreeBSD 4.8 YLIB (Yahoo C/C++ Libraries) YSS (Yahoo Structured Search) Yahoo! Autos Search Network API

Advantages Presents developers with isolation model they expect Backward Compatibility Efficient Resource Utilization (People and Power) Peak Absorption Faster Recovery Disadvantages VM files are very large (order of gigabytes)! VM Software is not free  Operating System Licensing Issues

Presents developers with isolation model they expect

Backward Compatibility

Efficient Resource Utilization (People and Power)

Peak Absorption

Faster Recovery

VM files are very large (order of gigabytes)!

VM Software is not free 

Operating System Licensing Issues

Third Application: Equi-Ping Game Server Placement For Pre-Arranged First-Person-Shooter Multiplayer Matches Motivation: To place game server hosts optimally such that all participating players have a fairer game experience.

Motivation: To place game server hosts optimally such that all participating players have a fairer game experience.

Delay Sensitivity of Online Games: Extremely Lag sensitive (on the order of 50ms) First Person Shooters (FPS) (Halo, Battlefield2, Doom, CounterStrike) Moderate sensitivity to lag (on the order of 500ms) MMORPG and RTS (World of Warcraft, Everquest, Generals) Not sensitive to lag (on the order of seconds) Turn Based Strategy (Civilization 4, Risk, Chess, Pool)

Screenshot from Halo PC:

The Problem: Unfair Lag Clan A Clan B 130ms 30ms Game Server

The Solution: Equi-Ping Server Clan A Clan B 80ms Real Machine 80ms VMM Virtual Game Server in VM

Equi-ping placement problem Red dots are servers, green dots are players, we need to pick the Red dot that is equi-distant from all players.

Red dots are servers, green dots are players, we need to pick the Red dot that is equi-distant from all players.

S = set of m servers available to host the match. P = set of n players participating in this match. RTT s,p = round-trip-time (ping) from server s to player p The Algorithm has 3 main steps: 1. Disregard all servers that have any RTT s,p larger than 180ms For each server s, we compute a closeness factor C s representing the average differential ping between the players if server s is picked: Pick the server with smallest C s The order of complexity of this algorithm is O(n 2 m) ≈ O(m) since m » n Equi-ping placement Algorithm

S = set of m servers available to host the match.

P = set of n players participating in this match.

RTT s,p = round-trip-time (ping) from server s to player p

The Algorithm has 3 main steps:

1. Disregard all servers that have any RTT s,p larger than 180ms

For each server s, we compute a closeness factor C s representing the average differential ping between the players if server s is picked:

Pick the server with smallest C s

The order of complexity of this algorithm is O(n 2 m) ≈ O(m) since m » n

Halo PC VM Server Stack Real Machine (PIII-550MHz, 640MB RAM, 9GB hard disk) VMware ESX VMM Server (consumes 184MB RAM, 3.5GB hard disk and 5% CPU) Virtual Machine exposes a PIII-550MHz with 512MB RAM and 5.5GB hard disk. Operating System: Windows XP (1.8GB) Halo PC Game Server (278MB)

Advantages: Equi-ping placement of game servers to minimize round-trip-time difference between participants Backward compatibility leading to zero cost for converting existing Economies of scale: this network can be shared among many games

Equi-ping placement of game servers to minimize round-trip-time difference between participants

Backward compatibility leading to zero cost for converting existing

Economies of scale: this network can be shared among many games

Applications Summary Dynamic Content Distribution which improves interactivity for Internet services. Server Switching which improves resource utilization, availability and enables peek absorption. Equi-ping server placement for game host servers which is very important for the fairness of online FPS games.

Dynamic Content Distribution which improves interactivity for Internet services.

Server Switching which improves resource utilization, availability and enables peek absorption.

Equi-ping server placement for game host servers which is very important for the fairness of online FPS games.

Related Work Common drawback: no backward compatibility. Disk Imaging / Ghosting, (no concurrent VMs). Application Servers: IBM WebSphere, ATG Dynamo, BEA WebLogic, JBoss, (no strict isolation, interface violations). Java Virtual Machine (J#/C#/.Net), (needs recoding, low performance). Packagers: Debian APT, Redhat RPM, PRC, (no isolation). Light Weight OSes (Denali), (new OS API, needs recoding). OS Virtualization (Ensim, Ejasent, EXETender, Chrooting), (tied to OS). Active Proxy Caches (ICAP, Active cache, OPES)

Common drawback: no backward compatibility.

Disk Imaging / Ghosting, (no concurrent VMs).

Application Servers: IBM WebSphere, ATG Dynamo, BEA WebLogic, JBoss, (no strict isolation, interface violations).

Java Virtual Machine (J#/C#/.Net), (needs recoding, low performance).

Packagers: Debian APT, Redhat RPM, PRC, (no isolation).

Light Weight OSes (Denali), (new OS API, needs recoding).

OS Virtualization (Ensim, Ejasent, EXETender, Chrooting), (tied to OS).

Active Proxy Caches (ICAP, Active cache, OPES)

Papers &quot;The vMatrix: A Network of Virtual Machine Monitors for Dynamic Content Distribution&quot;, Amr Awadallah and Mendel Rosenblum, 7th International Workshop on Web Content Caching and Distribution (WCW 2002), Boulder, Colorado, August 2002 &quot;The vMatrix: Server Switching&quot;, Amr A. Awadallah and Mendel Rosenblum, IEEE 10th International Workshop on Future Trends in Distributed Computing Systems (IEEE FTDCS 2004), Suzhou, China, May 2004. &quot;The vMatrix: Equi-Ping Game Server Placement For Pre-Arranged First-Person-Shooter Multiplayer Matches&quot;, Amr A. Awadallah and Mendel Rosenblum, The 4th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA 2006), Dubai/Sharjah, UAE, March 2006

&quot;The vMatrix: A Network of Virtual Machine Monitors for Dynamic Content Distribution&quot;, Amr Awadallah and Mendel Rosenblum, 7th International Workshop on Web Content Caching and Distribution (WCW 2002), Boulder, Colorado, August 2002

&quot;The vMatrix: Server Switching&quot;, Amr A. Awadallah and Mendel Rosenblum, IEEE 10th International Workshop on Future Trends in Distributed Computing Systems (IEEE FTDCS 2004), Suzhou, China, May 2004.

&quot;The vMatrix: Equi-Ping Game Server Placement For Pre-Arranged First-Person-Shooter Multiplayer Matches&quot;, Amr A. Awadallah and Mendel Rosenblum, The 4th ACS/IEEE International Conference on Computer Systems and Applications (AICCSA 2006), Dubai/Sharjah, UAE, March 2006

Other Papers &quot;TCP-BFA: Buffer Fill Avoidance&quot;, Amr A. Awadallah and Chetan Rai, IFIP High Performance Networking Conference, Vienna, Austria, September 1998. &quot;Analysis of a packet switch with memories running slower than the line-rate&quot;, Sundar Iyer, Amr A. Awadallah, and Nick McKeown, IEEE INFOCOM 2000, Tel-Aviv, Israel, March 2000.

&quot;TCP-BFA: Buffer Fill Avoidance&quot;, Amr A. Awadallah and Chetan Rai, IFIP High Performance Networking Conference, Vienna, Austria, September 1998.

&quot;Analysis of a packet switch with memories running slower than the line-rate&quot;, Sundar Iyer, Amr A. Awadallah, and Nick McKeown, IEEE INFOCOM 2000, Tel-Aviv, Israel, March 2000.

Conclusion The vMatrix is an overlay network of virtual machine monitors enabling delivery of virtual machines between real machines. The vMatrix is a practical backward-compatible solution for improving the scalability, reliability, and interactivity of existing Internet applications.

The vMatrix is an overlay network of virtual machine monitors enabling delivery of virtual machines between real machines.

The vMatrix is a practical backward-compatible solution for improving the scalability, reliability, and interactivity of existing Internet applications.

Thank You

Appendix: VMware Perl API use VMware::Control; use VMware::Control::Server; use VMware::Control::VM; my $VMMserver = VMware::Control::Server ::new($hostname, $port, $user, $password); $VMMserver->connect(); my $VM = VMware::Control::VM ::new($server, $vmconfig); $VM->connect(); # To get a list of all VMs on a server my @vmlist = $VMMserver->enumerate(); # To register a new VM on a server $VMMserver->register($vmconfig); # To start and stop a VM $VM->start(); $VM->stop(); # To suspend and resume a VM $VM->suspend(); $VM->resume(); # To get CPU, Memory, Net, IO stats $VM->get(“Status.Stats.vm.cpuUsage”, 5*60); # Check if VM is running (heart-beat) $VM->get(“Status.Power”);

use VMware::Control;

use VMware::Control::Server;

use VMware::Control::VM;

my $VMMserver = VMware::Control::Server ::new($hostname, $port, $user, $password);

$VMMserver->connect();

my $VM = VMware::Control::VM ::new($server, $vmconfig);

$VM->connect();

# To get a list of all VMs on a server

my @vmlist = $VMMserver->enumerate();

# To register a new VM on a server

$VMMserver->register($vmconfig);

# To start and stop a VM

$VM->start();

$VM->stop();

# To suspend and resume a VM

$VM->suspend();

$VM->resume();

# To get CPU, Memory, Net, IO stats

$VM->get(“Status.Stats.vm.cpuUsage”, 5*60);

# Check if VM is running (heart-beat)

$VM->get(“Status.Power”);

Two Tier Challenges Replication BACK END FRONT END FRONT END FRONT END

Replication