Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

State of the Stack April 2013


Published on

A comprehensive review of OpenStack then and now, each project's architecture, and hard data on why the race for open cloud is over. (First edition delivered April 2013 at OpenStack Summit. This version is from SPDEcon on June 10, 2013.)

Published in: Technology, News & Politics
  • Login to see the comments

State of the Stack April 2013

  1. 1. CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licensesState of the Stack - 2013Game. Over. OpenStack is The Stack.June 10th, 2013 - SNIA SPDEcon(first edition delivered at OpenStack Summit, April 2013)@randybias
  2. 2. Introduction
  3. 3. Who3OpenStack FoundationBoard of DirectorsProd. OpenStack pioneer, Cloudscaling:Wins: KT, Internap, LivingSocial, Seagate (EVault),IBS Datafort, major U.S. carriers, & othersPart of OpenStack community since July 2010 (launch)Top 10 Cloud Computing Pioneer
  4. 4. I run an OpenStack product companyI believe the pioneers to emulate are:I have run big data centers100K+ sq ft, 1,000s of physical servers, 100s of switchesMy Bias4
  5. 5. 51 What is OpenStack?3 History & Momentum4 Stackology - a stack taxonomy5 Stacking it Up - a dive into the projects6 Stack Gaps - what’s missing?7 Stack Politics - who’s playing?9 Summary2 Why the Success?8 Who’s using it and how?
  6. 6. What is OpenStack?
  7. 7. OpenStack From 10km7NetworkingOPENSTACKCLOUD OPERATING SYSTEMStandard HardwareCompute StorageYour ApplicationsOpenStack DashboardOpenStack Shared ServicesAPIs
  8. 8. OpenStack Mission8"To produce the ubiquitous Open Source cloud computingplatform that will meet the needs of public and privatecloud providers regardless of size, by being simple toimplement and massively scalable."Code Community
  9. 9. OpenStack Foundation Mission9The OpenStack Foundation is an independent body providing sharedresources to help achieve the OpenStack Mission by Protecting,Empowering, and Promoting OpenStack software and thecommunity around it, including users, developers and the entireecosystem.The ubiquitous cloudcomputing platform
  10. 10. What it isSome say ...... it’s an Infra-as-a-Service (IaaS)... it’s a cloud operating system... it’s a tool for building private cloudsWe say it’s “The Stack”... think Linux... think Java... think ubiquitous open source cloud toolkit... think Game Changer10
  11. 11. The Battle is Over (open src)11OpenStack LaunchOpenStack CloudStack Eucalyptus OpenNebulaSource:
  12. 12. Battle is Nearly Over (closed src)12OpenStack vSphere vCloudSource:
  13. 13. Linux 2000 vs. Linux 200913Is this OpenStack’s Trajectory?Operating system family market share2000 2009Unix LinuxLinux UnixSource: Linux Magazine
  14. 14. Fastest Growing Global Open Source Community14COMPANIESTOTAL DEVELOPERS AVERAGE MONTHLYCONTRIBUTORSCODE CONTRIBUTIONS929 245 3,241189 TOP 10 COUNTRIES9,000+INDIVIDUAL MEMBERS“OpenStack appears to be a more advanced or more modern open source project thansome of its predecessors because its a highly coordinated effort.”– Charlie BabcockInformation WeekCOUNTRIES100United States, China, India,Great Britain, Australia, France,Russia, Canada, Ireland, Germany
  15. 15. 15Grizzly StatsCONTRIBUTORSPATCHES / DEV NEW DRIVERS TEST CLOUDS DEPLOYED DAILY~14 15 700517 (+56%)TOP 10 CONTRIBUTING COMPANIES7,620PATCHES SUBMITTED“OpenStack appears to be a more advanced or more modern open source project thansome of its predecessors because its a highly coordinated effort.”– Charlie BabcockInformation WeekNEW FEATURES230Red Hat, Rackspace, IBM, HP, Nebula,Intel, eNovance, VMware, Cloudscaling,DreamHost
  16. 16. Why the Success?
  17. 17. Who or What Should We Thank?Rackspace for Letting GoOpenStack Foundation & CommunityParticularly, all of the companies who realized this could be bigHype CurveThe OpenStack Infrastructure TeamOslo Project (openstack-common)A thankless job allowing shared code & cleaner projectsThe Big Enterprises for Driving InterestPTL Generational Shift17
  18. 18. Infrastructure TeamMassive Effort -> Improved QualityGated CommitsAll Code Has to Jump Through GatesTempest Test FrameworkCode Reviews & ContinuousIntegrationJenkins, GerritAt scale: jenkins.openstack.org181/4 pages
  19. 19. History &Momentum
  20. 20. JulInauguralDesignSummit inAustin201020OpenStacklauncheswith 25+partnersFirst ‘Austin’code releasewith 35+partnersOct NovFirst publicDesignSummit inSan AntonioAustinOpenStack Object Storage prodOpenStack Compute dev previewLaunch!2010 - The Launch YearSource: Too many to list; blame me for inaccuracies
  21. 21. 2011 - Growing Pains & Early AdoptersFeb2nd Summit21Rackspaceannounces plansto launchindependentFoundation in2012OctApr3rd Summit(Santa Clara)addsConferenceGovernance moves forward withproject technical leads (PTL),policy board elections (PPB)JulFirstAnniversaryBexarOpenStack Compute for mid-size prodOpenStack Image Service added to coreCactusOpenStack Compute for larger-scale prodSepDiabloMajor stability releaseFirst 6-mo cycle release2011Decision to shift from 3-mo to6-mo dev cycleJan1st SwiftPublic CloudInternap w/ CloudscalingHappyBirthday!1st NovaPublic CloudInternap w/ Cloudscaling
  22. 22. Createdframework forFoundation asa communityFeb Apr Aug19 companiesannouncepublic supportfor FoundationFramework &documents ratifiedby community22Drafting committeeformed – creatinglegal documentsOpenStackFoundation“officially”launchesSepEssexOpenStack Identity in coreOpenStack Dashboard in coreFolsomOpenStack BlockStorage in coreOpenStackNetworking in core2012MayHPCloudLaunch(Beta)Citrix Bails(how’s that going for ya?)JanAT&T JoinsOpenStackInternal production (private)OctGartnerReport(teeth gnashing followed)Inaugural OpenStackFoundation BoardmeetingVMware, Intel, & NECaccepted as Gold membersBoardElections2012 - Rise of the Foundation & Prod Deployments
  23. 23. 2013/2014 - Breakout Growth YearsApr Oct23Q1HavanaOpenStack Metering in integrationOpenStack Orchestration in integrationLBaaS?“I” Release2014GrizzlyOpenStack Metering in incubationOpenStack Orchestration in incubation2013First Summit100% run andfunded byFoundationFirstInternationalSummit(APAC?)
  24. 24. In every single category, thetop 3 vendors support OpenStackIncredible Industry Support24top 3 switch vendors top 3 storage vendors top 3 hypervisorstop 3 router vendors top 3 blade vendors top 3 linux vendorstop 3 x86 vendors
  25. 25. Developer Growth25Contributors per month (ohloh)
  26. 26. Developer Growth ComparisonContributors per month (ohloh)26
  27. 27. Dev Growth by Git ContributorsQingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1
  28. 28. Accumulated CommunityQingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1
  29. 29. Growth by Domain (company - roughly)Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1 the CloudStackprojects, influence fromCitrix is quite obvious, over45% of github.comcommits come fromaccounts belonging and
  30. 30. 0750150022503000Austin Santa Clara San Francisco PortlandSummit Growth30
  31. 31. Established Marketing 241k/visits month:Software: 300K downloadsMembership: 9000+, Over 90% subscribe to newsletterRelationships with Tier 1 publications and analysts3117,693Followers(+50% from 8/12)
  32. 32. Stackology
  33. 33. OpenStack From 10km33NetworkingOPENSTACKCLOUD OPERATING SYSTEMStandard HardwareCompute StorageYour ApplicationsOpenStack DashboardOpenStack Shared ServicesAPIs
  34. 34. CLI toolsDashboard Other toolsComputeNetworkingOrchestrationHypervisor(s) QueuingDatabase/ KVS/ CacheExternalBlockProviderPhysicalNetworkProviderProvisioningLogAggregationHealthMonitoringetc.REST Meter DataRESTSQL, etc. Varies Varies Varies AMQP/0MQTopology &MetadataMeteringRESTREST RESTDNSImageManagementIdentityManagementRESTBlockStorageObjectStorageOpenStack (m)Architecture Slide34UI LayerElasticServicesLayerOtherstuff,youprobablyneed/wantData &ResourceLayerSharedServicesLayer
  35. 35. ProjectNameDescription LayerAWSEquivalentCodenameDashboard Self-service, role-based webinterface for users and administratorsUI Console HorizonCompute Provision and manage large pools ofon-demand computing resourcesElasticServiceEC2 NovaBlock Storage Volumes on commodity storage gear,and drivers for turn-key block storagesolutionsElasticServiceEBS CinderObjectStoragePetabytes of reliable storage onstandard gearElasticServiceS3 SwiftNetworking L2-focused on-demand networkingwith some L3 capabilitiesElasticServiceVPC QuantumOrchestration Application orchestration layer thatruns on top of and managesOpenStack ComputeElasticServiceCloudFormation,CloudWatchHeatMetering Centralized metering data for allservices for integration to externalbillingSharedServiceN/A CeilometerIdentity Multi-tenant authentication systemthat ties to existing stores (e.g.LDAP) and Image ServiceSharedServiceNone KeystoneImageManagementUpload, download, and manage VMimages for the compute serviceSharedServiceVM Import/ExportGlance35
  36. 36. 6 month integrated release cycleEvery 6 months, we coordinate and integrate:Thousands of patches & commitsAcross hundreds of developersAnd 9 “integrated” or “core” projectsCompletely impossible without:The OpenStack infrastructure team (CI, etc.)Dedicated PTLs and individual developersNo other similar project does this36
  37. 37. OpenStack is Well OrganizedQingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1“[the coordinatedreleases are] anindicator that theOpenStackproject is wellorganized in termsof sub-projectmanagement.”
  38. 38. Stacking It Up
  39. 39. A Quick Note of Thanks39These diagrams would not have beenpossible without the prior work of:Ken Pepple, Solinea (@ken_pepple)Dina Belova, Mirantis... and the help of several Cloudscalers:Eric Windisch (@ewindisch)Joe Gordon ( Joyce (@openfly, Sneddon (@dxs)Joseph Glanville (@jpgvm)
  40. 40. Caveat Emptor40The focus for these diagrams was ease ofreading, not accuracy.See Ken Pepple’s originals or the code if you need truth.That being said, our team tried really hard for accuracy.Blame me for any errors.
  41. 41. Architecture Diagrams Legend41
  42. 42. OpenStack RPC42{oslo.version: 2,oslo.message: json ( {method: method_name,args: { keyword: value }} )}nova-api nova-schedulerRemote Procedure Call(invoked via (a)synchronous message passing)
  43. 43. OpenStack Compute (Nova)43
  44. 44. Compute Thoughts44Nova still runs best w/ KVMDo we need another hypervisor? What’s the biz case?Multiple Availability Zones still not solvedCells are for making one AZ biggercomplex, tight-couplingWe need a clean sharding mechanism for AZesalso what about Cinder and Quantum?Integ. to Cinder & Quantum needs rethinkMore information needs to be able to be passed back
  45. 45. Compute (Networking) Thoughts45nova-network still requiredQuantum has been L2 focused & L3 gap still existscentralized nova-networking is #faildecentralized is more #failnova-conductor security for hypervisor obviatednova-metadata-api & nova-network on every hypervisor?security implicationsreconciling Quantum and nova-network?Quantum needs more L3 capability, but ...
  46. 46. OpenStack Block Storage (Cinder)46
  47. 47. Block Storage Thoughts47Default “nova-volume” func. is too minimalWhen people think Block Storage service they assume:Persistent, Network-based, & Performant - it isn’tCinder scheduler needs info from NovaAssuming you want to do anything interestingPoint of lock-in since default isn’t usefulYou have to place a bet on a block storage solutionThese are expensive, experimental or proprietary
  48. 48. OpenStack Networking (Quantum)48
  49. 49. Networking Thoughts49Default networking functionality is minimalThe APIs have been L2-centricL3 functionality is same as existed with nova-networkSame architecture, same basic layout, with all of the downsidesNeeds a Quantum plugin for full func.Can’t run more than one plugin at a time per functionOnly truly baked plugin is probably Nicira?Others in process, but it’s not clear how many production deployments there areGood news is that this area is hotSo hopefully this is unstuck soon
  50. 50. OpenStack Object Storage (Swift)50
  51. 51. Object Storage Thoughts51Swift has continued to lag OpenStack dev3 yrs on, auditor is slow & does not prioritize replicationContainer replication is a bad hackUgly stepchild of OpenStackKeystone authentication woes (integration, performance)Isn’t universally loved like NovaSad, since it was the more mature of the two projects at launch in 2010Good news: this area has new playersEVault, Seagate, SwiftStack
  52. 52. OpenStack Image Mgmt (Glance)52
  53. 53. Image Mgmt Thoughts53Still not clear why this is standalone projectReally a sub-function of ComputeSemi-pluggable (but not really)Uses different backends for image storageTo be really useful it needs more features:P2V, V2V, and other image conversionAbility to slipstream PV drivers into imagesConvert from popular formats: OVF, AMI, etc.
  54. 54. OpenStack Identity (Keystone)54
  55. 55. Identity Mgmt Thoughts55Mixed identity / schizophreniaVerifies identity, authorization, AND service registryService registry is one of manyNova, Cinder, et al have their own internal registriesSlows everything downSee LivingSocial preso from Folsom SummitSee caching tricks with memcache some projects useOpenStack needed to reinvent wheel here?This could have just been LDAP with a schema + caching
  56. 56. OpenStack Dashboard (Horizon)56Source: Wikipedia (–view–controller)
  57. 57. Dashboard Thoughts57It’s gotten a lot betterSame UI for end-user and admin is bad ideaCloudStack did this and it was a messThe workflows and views are too differentSecurity considerations existGeneral lag: many things aren’t accessiblee.g. HeatNeed better docs on extending, w/o harmCustomers, product companies, SPs all want to modifyAllow for customizations, while supporting upgrades, etc.
  58. 58. OpenStack Metering (Ceilometer)58
  59. 59. Metering Thoughts59Metering systems are hardBad or incomplete data for SPs is existentialMetering system should be *very* baked (is 1yr enough?)No tokenized meter dataInstance hours not enoughHow do you bill for Windows, Oracle, RHEL licenses?Tokens stack: size of instance, OS and app licenses, etc.Needs to get flow data from edge switchesNetflow and/or Sflow support for physical switchesData from the vSwitches is not the best source in the real world
  60. 60. OpenStack Orchestration (Heat)*60* Source:
  61. 61. Orchestration Thoughts61Huge potentialAdds additional AWS func: CloudWatch, CloudFormationProvides clean templates for stacks, which means:OpenStack on OpenStack (OoO) for testing, etc.First primary project that rides “on top”Clear differentiator over other projectsInitiative: Heat templates for Ref ArchVendors, customers, etc. could feed to prov systems:Crowbar, Piston, Cloudscaling, etc.
  62. 62. Strengths:De facto winnerIncredible communityUnstoppable velocityClear innovation curveSWOT: OpenStack62Opportunities:Build an SQL92 base for cloudcompute (see Threats)Public cloud compatibility as firstorder initiativevCloud private cloud compatibilityas first order initiativeWeaknesses:No benevolent dictatorLack of IaaS experience for manydevelopersInteroperability will be difficultNot impossible, *difficult*Threats:Splintering, fragmentation, andcustomizationForking or ivory tower thinking
  63. 63. Stack Gaps
  64. 64. What’s In a Complete Cloud OS?64OpenStackRelationshipWho?OpenStackScoreEcosystemScoreVendorTarget*User Interface(s) Horizon, CLI, ...OpenStack, Vendor,Ecosystem 4 2 4Elastic ResourceManagementNova, Swift,Quantum, Cinder, ...OpenStack, Vendor,Ecosystem 4 1 4Service DiscoveryScattered:Nova, Keystone, ...OpenStack, Vendor 2 0 4Authentication,Authorization, andAccess Controls (AAA)Keystone (authen/author), variousprojects (ACLs)OpenStack 2 0 4HW/SW Life CycleManagementN/A Vendor, Ecosystem 1 2 4Service Management N/A Vendor0 1 4Health & Logging N/A Vendor, Ecosystem0 0 4Topology & Inventory N/A Vendor0 0 4Hardware Certifications N/AOpenStack, Vendor,Ecosystem 1 1 2*** We’re all _trying_ to close this gap ** It’s a hard problem no one will solve individually
  65. 65. Your Basic ChoicesDownload OpenStack and DIYOpenStack DistributionsTurn-key Systemspowered by OpenStack65123
  66. 66. Stack Politiks
  67. 67. Types of OpenStack Players67Type Description ExampleHardware VendorSelling hardware that integrates or supportsOpenStackJuniper, NetApp, Cisco, EMCComponent VendorPoint solution, usually software, thatprovides subset of OpenStack functionality orsupports itMidokura, NexentaDistribution /PackagerBasic packaging, some installation/setup,etc.RedHat, SUSE, CanonicalTurn-key SystemComplete, integrated, OpenStack solution,with value addsCloudscaling, Nebula, PistonService CompaniesProfessional or managed services tocustomize or operate OpenStackMirantis, Metacloud,Rackspace PrivatePublic Clouds Public IaaS HP, Rackspace PublicPaaS / ISVs Value add on top of OpenStack deploymentsScalr, ActiveState (Stackato),CloudFoundryPrivate Clouds Users Wikimedia, AT&T, Yahoo!
  68. 68. components -computecomponents -storagesystemsLinux distros public clouds private cloudsPaaS / layered ISVsservice companiescomponents - networkhardwareWho’s Playing the OpenStack Game?68
  69. 69. components -computecomponents -storagesystemsLinux distros public clouds private cloudsPaaS / layered ISVsservice companiescomponents - networkhardwarePlayer Motivations69Sell HardwareSell SDNSoftwareSell StorageSoftwareSell HVSoftware &SupportSell Turn-keySystems &SupportSell Labor (T&M),MonthlyManagement, etc,Sell Software onTop of IaaSSell Support via“owning” thecommunitySell Online CloudResourcesUse OpenStack forBusiness Leverage
  70. 70. Who’s Using It?
  71. 71. First OpenStack Survey71414#survey#responses#16%7%8%4%11%17%37%More#than#10,000#employees#5,001#to#10,000#employees#1,001#to#5,000#employees#501#to#1,000#employees#101#to#500#employees#217100#employees#1720#employees#Company SizeInformation Technology 60%#Academic / Research 15%#Telecommunication 10%#IndustriesGovernment / Defense 3%#CC Icons 29 28 23 1856countriesCountry124$100$77$151$ServiceProviderEcosystemVendorCloudConsumerCloudOperatorType of Involvement
  72. 72. Deployments at a GlanceType35#Hosted#Private# 15#Hybrid##37#Public#106#On#Premise#Private#Trunk&8%&Grizzly&15%&Folsom&47%&Essex&25%&Diablo&5%&Version849294Production Proof ofConceptDev/QAStage134#94#94#89#66#46#DashboardObject StorageSnapshotting to new imagesLive MigrationEC2 Compatibility APIS3 Compatibility APIFeatures181#171#169#153#147#121#103#20#16#NovaGlanceKeystoneHorizonQuantumCinderSwiftCeilometerHeatComponents197&Deployments&
  73. 73. Size of 98 Production Compute Systems731"100$$52%$101"500$18%$ 501"1,000$$8%$1,001"5,000$$8%$5,000"10,000$$3%$>$10,000$$6%$Unspecified$5%$Other$30%$Instances1"50$$71%$51"100$$8%$101"500$$9%$501"1,000$$2%$>1,000$$4%$unspecified$6%$Other$29%$Nodes1"100$$51%$101"500$$21%$ 501"1,000$$4%$1,001"5,000$$12%$5,001"10,000$3%$>10,000$$4%$unspecified$5%$Other$16%$Cores
  74. 74. Usage: KVM, LVM, OVS & SQL74KVM$71%$ESX$8%$Xen$8%$Xenserver$5%$Lxc$5%$Hyperv$3%$Other$29%$HypervisorsLVM$36%$NFS$19%$Ceph$RBD$13%$ Netapp$10%$GlusterFS$8%$SAN/HP$5%$Windows$4%$EMC$3%$Solidfire$2%$Other$32%$Open$Vswitch$39%$Linux$Bridge$31%$Cisco$11%$ HyperUv$5%$Nicira$5%$Brocade$3%$Ryu$2%$big$switch$2%$NEC$2%$Other$19%$Storage DriversNetwork DriversSQL$55%$LDAP$34%$ PAM$8%$KVS$3%$Other$11%$Identity Drivers150$62$33$JSON$ XML$ Both$API Format
  75. 75. Summary
  76. 76. OpenStack by TKO?OH: “Finish him!”We still have work to doYour participation mattersRegardless of whether you: build, develop, or operateGet involved
  77. 77. Q & A blog77Randy BiasCTO & Co-founder, CloudscalingDirector, OpenStack Foundation@randybiasOriginal Summit Slides are at: