Introduction and Overview of OpenStack for IaaS


Published on

These slides supported a presentation at the 2013 Red Hat Summit.

It covers:
✦ Introduction to OpenStack
✦ OpenStack Architecture
✦ Understanding the Elastic Cloud
✦ OpenStack in the Real World

Published in: Technology

Introduction and Overview of OpenStack for IaaS

  1. 1. 1
  2. 2. Introduction & Overview ofOpenStack for IaaS CloudsKeith BasilPrincipal Product Manager, OpenStackRed HatJune 12, 20132
  3. 3. Your presenter..personalVirginia hare scrambler, plays chess..professionalRed HatCloudscaling, Time Warner Cable,, Cisco anda couple of startupsblendedskype/twitter/github/irc, life: noslzzp3
  4. 4. Agenda✦ Introduction to OpenStack✦ OpenStack Architecture✦ Understanding the Elastic Cloud✦ OpenStack in the Real World4
  5. 5. What Is OpenStack?OpenStack is both Software and a Community5
  6. 6. What Problem Does OpenStack Solve?OpenStack provides a framework for buildingelastic cloud infrastructure at massive scale.✦Facilitates management of✦COMPUTE,✦NETWORK, and✦STORAGE resources✦Provides natural infrastructure for elastic applicationsWe will cover the “elastic cloud” later in more detail..6
  7. 7. OpenStack Components Provide..✦Compute✦Virtual machine management✦Comprehensive hypervisor support✦Networks✦IP address management✦Security services✦Storage✦Volumes (block storage)✦Object storage for VM images and files7
  8. 8. OpenStack as CommunityTechnical Committee - defines and stewards technical directionBoard of Directors - provides strategic and financial oversight ofFoundation ResourcesUser Committee - created to represent enterprise, academic andservice provider users8
  9. 9. OpenStack History and CadenceHavana(Oct 2013)Grizzly(Apr 2013)Folsom(Oct 2012)Essex(Apr 2012)Diablo(Sep 2011)Cactus(Apr 2011)Bexar(Feb 2011)Austin(Oct 2010)Ceilometer & Heat integrated(Oct 2013)Maturation of Quantum and Cinder, focus on upgrade support(Apr 2013)Quantum (Networking) full inclusion, Volume Service added(Oct 2012)Dashboard and Identity service released, Quantum incubated(Apr 2012)First “Production Ready” release(Sep 2011)Scaling enhancements, support for many hypervisors(Apr 2011)OpenStack Compute ready, initial release of Image Service(Feb 2011)Initial release, Object Storage Production Ready, Compute in testing6-monthcadence9
  10. 10. OpenStack Trends, Growth & MilestonesHavana(Oct 2013)Grizzly(Apr 2013)Folsom(Oct 2012)Essex(Apr 2012)Diablo(Sep 2011)Cactus(Apr 2011)Bexar(Feb 2011)Austin(Oct 2010)First OpenStackrelease with Red Hat codeOpenStack Foundation formed(Sep 2012)47 committers acrossthe top ten companiesRed Hat assigns its first developer to the OpenStack community(Aug 2011)71 committers230committersLargest OpenStack Summit to date,enterprise customer keynotesData extracted from Google Trends (keyword: OpenStack) and Bitergia reports.“Committers” shown above indicate the number of individual committers across the top ten contributing companies.10
  14. 14. Red Hat ContributorsMany contributors. One community.No coincidence.14
  15. 15. Agenda✦ Introduction to OpenStack✦ OpenStack Architecture✦ Understanding the Elastic Cloud✦ OpenStack in the Real World15
  16. 16. OpenStack Design and ArchitectureDASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)Modular architectureBased on a (growing) set of core servicesDesigned for Scalability and ElasticityBLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)16
  17. 17. OpenStack Dashboard (Horizon)Provides simple self service UI for end-usersBasic cloud administrator functions(No infrastructure management)Define users, tenants and quotasDASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)BLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)17
  18. 18. OpenStack Compute (Nova)Supports multiple hypervisors (KVM, Xen, LXC, Hyper-V, ESX)Native OpenStack API and Amazon EC2 API supportDistributed controller services handle scheduling, API calls, etc.DASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)BLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)18
  19. 19. OpenStack Block Storage (Cinder)Provides block storage for virtual machines (persistent disks)Similar to Amazon EBS servicePlugin architecture for vendor extensionsDASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)BLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)19
  20. 20. OpenStack Networking (Quantum)Network ServiceProvides framework for Software Defined Networking (SDN)Allows integration of hardware and software based networksolutionsDASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)BLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)20
  21. 21. OpenStack Image Service (Glance)Stores and retrieves disk images (virtual machine templates)Supports Raw, QCOW, VMDK, VHD, ISO, OVF & AMI/AKIBackend storage : Filesystem, Swift, Amazon S3DASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)BLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)21
  22. 22. OpenStack Object Storage (Swift)Modeled after Amazons S3 serviceProvides simple service for storing and retrieving arbitrary dataNative API and S3 compatible APIDASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)BLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)22
  23. 23. OpenStack Identity Service (Keystone)Common authorization frameworkManages users, tenants and rolesPluggable backends (SQL, PAM, LDAP, etc)DASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)BLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)23
  24. 24. Connecting the ComponentsDASHBOARD(Horizon)COMPUTE(Nova)IDENTITYSERVICE(Keystone)OpenStack uses message queues for communicationbetween componentsSupported queueing backends: RabbitMQ, Qpid and ZeroMQBLOCKSTORAGE(Cinder)OBJECTSTORE(Swift)NETWORKING(Quantum)IMAGESERVICE(Glance)Message Queue24
  25. 25. Incubation Project: CeilometerMetering & Monitoring(Ceilometer)- Collects meter data (CPU, network, etc)- Designed for integration and extensibility- Data collected is made available via REST API- Message signing provides non-repudiationGraduated from Incubation to Integratedstatus for the Havana release25
  26. 26. Incubation Project: HeatApplication Orchestration(Heat)- Provides template driven cloud applicationorchestration- Modeled after AWS CloudFormation- Targeted to provide advanced functionalitysuch as high availability and autoscaling- Introduced by !Graduated from Incubation to Integratedstatus for the Havana release26
  27. 27. Spin meup a VM!And make it LARGE!Umm, Do Iknow you? I need tosee some papers!!KeystoneOk, weneed to find aplace to buildthis VM.NovaTag -you’re it!VMcapacity capacitycapacity Papers aregood. Time toget to work!NovaNodeQuantum, Ineed a network with allthe trimmings!QuantumHere’s your IP,default route andFW settings.Cinder, havethat volumeready for me?NodeIndeed I do.Don’t forget tomount it!SwiftGlanceHey Glance, can Iget the RHEL 6.4image?Node8)Let’s Follow a Request..Thank youOpenStack!!8)It’srenderingtime!27
  28. 28. Agenda✦ Introduction to OpenStack✦ OpenStack Architecture✦ Understanding the Elastic Cloud✦ OpenStack in the Real World28
  29. 29. Elastic Cloud != Enterprise VirtualizationOn-demand self-serviceBroad network accessResource poolingRapid elasticityMeasured serviceMany applications on each serverMaximum server utilizationMinimum server count29
  30. 30. Workload EvolutionCLOUD WORKLOADS✦Smaller stateless VMs✦Lifecycle measured in hours to months✦Applications scale out horizontallywith new VMs✦Applications expect failureTRADITIONAL WORKLOADS✦Larger stateful VMs✦Lifecycle measured in years✦Applications scale up (more vCPU, vRAM)✦Applications NOT designed totolerate failure30
  31. 31. “Pets vs Cattle” (Scale Up vs Scale Out)The above adapted from Tim Bell, CERN “Pets vs. Cattle” is attributed to Bill Baker, Microsoft Distinguished Engineer.“Future application architectures should use Cattle but Pets withstrong configuration management are viable and still needed”- Tim Bell, CERN“Scale Up- Servers are like pets.Pets are given names, are unique,lovingly hand raised and cared for.When they get ill, you nurse them backto healthScale Out- Servers are like cattle.Cattle are given numbers and arealmost identical to each other.When they get ill, you get anotherone.31
  32. 32. Why the Elastic Cloud Is NeededOur Data is too large✦We are past the point of single computers being able toefficiently handle our dataService Requests are too large✦Client devices are more plentiful than everOr, BOTH..32
  33. 33. Why the Elastic Cloud Is NeededServers fail - Deal with it!1✦Assume you could start with super reliable servers(MTBF of 30 years!)✦If you build a system with 10,000 of those servers✦You will watch one fail every dayFault-tolerant software is inevitable![1] Adapted from Jeff Dean’s presentation on Designs, Lessons and Advice from Building Large Distributed Systems
  34. 34. “Fault Tolerance” (survival) in Natureelastic, agile, scaled with small failure domains34
  35. 35. Pets in the Water35
  36. 36. but.. Unnatural Acts Can Be Impressive36
  37. 37. Agenda✦ Introduction to OpenStack✦ OpenStack Architecture✦ Understanding the Elastic Cloud✦ OpenStack in the Real World37
  38. 38. OpenStack’s Increasing MaturityWe are seeing organizations struggle withelastic cloud adoption✦Unfortunately, Frankenclouds arebeing white boarded daily!✦“Pet friendly” strategies are underway✦Green field deployments work wellKnowledge capture is early but evolving✦OpenStack Operations Guide✦OpenStack Security Guide38
  39. 39. OpenStack Deployment ConsiderationsOpenStack’s two most important deployment questions:✦What does the NETWORK look like?✦“The 90’s called. They want their network architecture back.”✦ What does the cloud HARDWARE look like?✦Amazon and Google have figured it out: embrace and extend!39
  40. 40. Question: The Network?40
  42. 42. Because your cloud will grow..Each unit here could be a server, or a rack of servers.42
  43. 43. And continue to grow..1,152 servers shown? Or 1,152 racks with 16 servers each (18,432 servers)?hint: it shouldn’t matter43
  44. 44. Spine and Leaf TopologyAsk your friendly network vendor for guidance✦Cisco, ARISTA, Brocade, Juniper, Force10, etc.
  45. 45. Question: The Hardware?45
  46. 46. 1/11/21/41/8n1-standard-8-dn1-standard-4-dn1-standard-2-dn1-standard-1-dm1.xlargem1.largem1.mediumm1.smallm1.classn1-standard.classxlargelargemediumsmallPublic Cloud VM Instances Exposed!46
  47. 47. Deployment: Sizing a Compute Nodexlargelarge mediumsmallSolve for thebiggest VM inthe classSmaller VMs are fractional proportions of the largest.This facilitates efficient hardware use and scheduling.Compute Hardware Node (m1.class)128GB memory, (16) 1TB disks, (2) E5-2670 CPU1/1 1/2 1/4 1/847
  48. 48. Deployment: Sizing a Compute NodexlargeCompute Hardware Node (m1.class)128GB memory, (16) 1TB disks, (2) E5-2670 CPUxlargesmallsmallsmallsmallsmallsmallsmallsmallmedium mediummedium mediumlargexlarge xlargelargesmallsmallsmallsmallsmallsmallsmallsmallGiven the machine config above, it would support:(4) n1-standard-8-d, (8) n1-standard-4-d, (16) n1-standard-2-d, (32) n1-standard-1-d(8) m1.xlarge, (16) m1.large, (32) m1.medium, (64) m1.small48
  49. 49. Plan for the Resource Service LevelCompute/StorageNetwork FabricCloud ControllerResourceServiceLevel49
  50. 50. OpenStack Deployment Profiles50
  51. 51. PayPalProfile Highlights:✦113 million registered accounts; PCs or mobile devicesin 190 different markets with 25 different currencies✦Targeting 90 percent coverage for several thousandnodes in nine to 12 months✦DIY team leveraging OpenStack community support51
  52. 52. OpenStack Accelerating ScienceProfile Highlights:✦Using OpenStack to support particlephysics research✦Currently running 500 nodes and 2000 VMs✦Immediate plans:✦Deploying production OpenStack running Grid software✦Intends to use Ceilometer, bare metal for tenants and LBaaS✦Ramping to 15,000 hypervisors with 100k - 300k VMsby 201552
  53. 53. National Security AgencyProfile Highlights:✦One of the NSAs largest hosting platforms✦Number of users, systems, servers, storage, applicationsusers: [REDACTED]✦Agility, flexibility and scalability providing bettersupport for mission systems✦Big Data is truly big✦Has been working with OpenStack since Cactus53
  54. 54. In Summary54
  55. 55. OpenStack ...✦Is open source software and vibrant community✦Provides a framework for an elastic cloud.✦Requires fresh thinking for deployments✦Is being deployed successfully at scale today55
  56. 56. And finally..56
  57. 57. 57
  58. 58. Thank You!Red Hat IaaS Overview & RoadmapAndrew Cathrow — Sr. Virt. Product Manager, Red HatSimon Grinberg — Principal Product Manager, Red HatWednesday, June 123:40 pm - 4:40 pmRed Hat OpenStack Performance & ScaleMark Wagner — Senior Principal Engineer, Red HatWednesday, June 124:50 pm - 5:50 pmOpenStack ArchitectureRussell Bryant — Principal Software Engineer, Red HatThursday, June 133:40 pm - 4:40 pmCheck out these sessions!58
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.