Art of Using Xen at Scale

Art of Using Xen at Scale Pradeep Vincent Principal Engineer, Amazon Web Services

Agenda •  Amazon Web Services -‐ Intro •  Virtualiza@on in the Cloud •  Scale Impact on Virtualiza@on •  Xen Enhancements.. •  Future Features – Some Thoughts

AMAZON WEB SERVICES -‐ INTRO

Amazon Web Services “..delivers a highly scalable cloud compu@ng plaIorm with high availability and dependability, and the ﬂexibility to enable customers to build a wide range of applica@ons.”

Amazon Web Services •  Includes more than 20 Services –  Compute Services •  EC2 –  Storage Services •  S3 •  EBS –  Database Services –  Content Delivery Services –  Messaging Services –  And many more.. •  More Details : aws.amazon.com

EC2 Instance Types High High Cluster Standard Micro Memory Compute Compute Instances Instances Instances Instances Instances

VIRTUALIZATION IN THE CLOUD

Virtualiza@on and Cloud •  Virtualiza@on a key enabler for Cloud Compu@ng •  Virtualiza@on Provides.. –  Secure Control Plane –  Fine grained resource alloca@on –  Cloud Speciﬁc Business Logic in Dom-‐0

Virtualiza@on in the Cloud •  Guest OS(Kernel) chosen by customer –  Large and ever changing set •  Aggressive inges@on of Security ﬁxes –  Security is paramount •  Unique Challenges at Scale..

SCALE IMPACT ON VIRTUALIZATION

Scale Ma_ers..

Intra Host ‘Scaling’ Mul@-‐tenancy

Mul@-‐tenancy •  Scaling Challenges –  Performance Scaling: I/O performance etc –  Tools Scaling: XM etc •  Isola@on Challenges –  Performance Isola@on –  VM Fault Isola@on

Fault Isola@on

VM-‐level Fault Isola@on •  Failure related to one VM isolated from Peer VMs •  Good Hypervisor Level Fault Isola@on in Xen •  End-‐to-‐End Fault Isola@on Important –  Including Dom-‐0 I/O Stack –  Including Control Plane (e.g. Xenwatch)

Performance

Consistency of Performance Performance Isola@on

Performance Scaling •  Cri@cal for Mul@-‐tenancy •  Par@cularly I/O Performance •  Good in Xen •  Deserves con@nued Focus..

I/O Performance Direct AHached Device Performance Good But.. Loss of Control and Flexibility Not Great

Para-‐virtualized I/O Performance Important

Reduce Cost

Hardware Enablement •  Xen Enablement bit ‘Slow’ for some hardware •  Faster ‘Produc@on’ quality enablement in Xen –  Help Cloud adopt new hardware faster

Upstream Linux IntegraOon GREAT !!

Interface Compa@bility

Interface Compa@bility •  Applies to Many Types of Interfaces –  Paravirtualized Driver Interface •  Blkfront – Blkback •  NeIront – Netbk –  Hypercall Interface –  Control Plane Interface (XM vs XL) •  Has Been Good in Xen –  Deserves con@nued focus

XEN ENHANCEMENTS..

Serve Spiky Workloads For Less

Micro instance in EC2

Micro Instance and CPU Hogs

Large ‘CPU Time’ Bucket Bigger than 10s of millisecs 10s of milliseconds

Some Thoughts on FUTURE FEATURES

Mul@-‐@er Scheduling ? •  Lots of use cases in the Cloud •  Classes with priori@za@on –  Beyond ‘CPU Pools’ –  Similar to Linux Scheduler ? –  Real Time Scheduling/Advanced Func@onality •  Preserve exis@ng Scheduler Seman@cs to the extent possible

Dom-‐0 Resource Accoun@ng •  CPU consumed in Dom-‐0 for each VM –  Isn’t accounted against the VM per se –  Fairness Issues –  Priori@za@on Issues

Dom-‐0 Resource Accoun@ng •  Possible Solu@ons –  Group Scheduling in Dom-‐0 – Par@al Solu@on –  Xen and Dom-‐0 Scheduler Coopera@on ?? •  Hypervisor – Dom-‐0 Interface Blurred.. –  Something else ?? •  Scheduler deals with Threads.. –  One Thread serving Mul@ple VMs not great –  ‘Thread Aligned’ VM Resource Consump@on Good

Key Takeaways •  Virtualiza@on Key For Cloud •  End-‐to-‐End Fault Isola@on Important •  Performance Isola@on And Performance Consistency Important •  Para-‐virtualized I/O Performance Important •  Faster ‘Produc@on’ Quality Hardware Enablement •  Interface Compa@bility Cri@cal

SHAMELESS MARKETING

We are Hiring !

We are Hiring ! We are Hiring !

We are Hiring ! We are Hiring ! We are Hiring ! aws.amazon.com/careers We are Hiring !

THANK YOU •  EMAIL: PRADEEPV@AMAZON.COM •  MORE INFO: WWW.AWS.AMAZON.COM

http://www.xen.org	7746
http://xen.org	7093
http://www.xenproject.org	131
http://lars.1.xen.org	100
http://translate.googleusercontent.com	44
http://xenproject.org	35
http://xenorg.cloudaccess.net	31
http://staging.xen.org	19
http://xen.xensource.com	9
http://www.xen.org.	1
http://www.xenproject.org.	1

Art of Using Xen at Scale

by XenProject.org on Aug 11, 2011

Accessibility

Upload Details

Usage Rights

11 Embeds 15,210

Statistics

Art of Using Xen at Scale Presentation Transcript