A comprehensive review of OpenStack then and now, each project's architecture, and hard data on why the race for open cloud is over. (First edition delivered April 2013 at OpenStack Summit. This version is from SPDEcon on June 10, 2013.)
CCA - NoDerivs 3.0 Unported License - Usage OK, no modifications, full attribution** All unlicensed or borrowed works retain their original licensesState of the Stack - 2013Game. Over. OpenStack is The Stack.June 10th, 2013 - SNIA SPDEcon(first edition delivered at OpenStack Summit, April 2013)@randybias
Who3OpenStack FoundationBoard of DirectorsProd. OpenStack pioneer, Cloudscaling:Wins: KT, Internap, LivingSocial, Seagate (EVault),IBS Datafort, major U.S. carriers, & othersPart of OpenStack community since July 2010 (launch)Top 10 Cloud Computing Pioneer
I run an OpenStack product companyI believe the pioneers to emulate are:I have run big data centers100K+ sq ft, 1,000s of physical servers, 100s of switchesMy Bias4
51 What is OpenStack?3 History & Momentum4 Stackology - a stack taxonomy5 Stacking it Up - a dive into the projects6 Stack Gaps - what’s missing?7 Stack Politics - who’s playing?9 Summary2 Why the Success?8 Who’s using it and how?
OpenStack Mission8"To produce the ubiquitous Open Source cloud computingplatform that will meet the needs of public and privatecloud providers regardless of size, by being simple toimplement and massively scalable."Code Community
OpenStack Foundation Mission9The OpenStack Foundation is an independent body providing sharedresources to help achieve the OpenStack Mission by Protecting,Empowering, and Promoting OpenStack software and thecommunity around it, including users, developers and the entireecosystem.The ubiquitous cloudcomputing platform
What it isSome say ...... it’s an Infra-as-a-Service (IaaS)... it’s a cloud operating system... it’s a tool for building private cloudsWe say it’s “The Stack”... think Linux... think Java... think ubiquitous open source cloud toolkit... think Game Changer10
The Battle is Over (open src)11OpenStack LaunchOpenStack CloudStack Eucalyptus OpenNebulaSource: trends.google.com
Battle is Nearly Over (closed src)12OpenStack vSphere vCloudSource: trends.google.com
Linux 2000 vs. Linux 200913Is this OpenStack’s Trajectory?Operating system family market share2000 2009Unix LinuxLinux UnixSource: Linux Magazinehttp://www.linux-mag.com/id/7749/LinuxUNIXBSDWindows/OtherMixed
Fastest Growing Global Open Source Community14COMPANIESTOTAL DEVELOPERS AVERAGE MONTHLYCONTRIBUTORSCODE CONTRIBUTIONS929 245 3,241189 TOP 10 COUNTRIES9,000+INDIVIDUAL MEMBERS“OpenStack appears to be a more advanced or more modern open source project thansome of its predecessors because its a highly coordinated eﬀort.”– Charlie BabcockInformation WeekCOUNTRIES100United States, China, India,Great Britain, Australia, France,Russia, Canada, Ireland, Germany
15Grizzly StatsCONTRIBUTORSPATCHES / DEV NEW DRIVERS TEST CLOUDS DEPLOYED DAILY~14 15 700517 (+56%)TOP 10 CONTRIBUTING COMPANIES7,620PATCHES SUBMITTED“OpenStack appears to be a more advanced or more modern open source project thansome of its predecessors because its a highly coordinated eﬀort.”– Charlie BabcockInformation WeekNEW FEATURES230Red Hat, Rackspace, IBM, HP, Nebula,Intel, eNovance, VMware, Cloudscaling,DreamHost
Who or What Should We Thank?Rackspace for Letting GoOpenStack Foundation & CommunityParticularly, all of the companies who realized this could be bigHype CurveThe OpenStack Infrastructure TeamOslo Project (openstack-common)A thankless job allowing shared code & cleaner projectsThe Big Enterprises for Driving InterestPTL Generational Shift17
Infrastructure TeamMassive Effort -> Improved QualityGated CommitsAll Code Has to Jump Through GatesTempest Test FrameworkCode Reviews & ContinuousIntegrationJenkins, GerritAt scale: jenkins.openstack.org181/4 pages
JulInauguralDesignSummit inAustin201020OpenStacklauncheswith 25+partnersFirst ‘Austin’code releasewith 35+partnersOct NovFirst publicDesignSummit inSan AntonioAustinOpenStack Object Storage prodOpenStack Compute dev previewLaunch!2010 - The Launch YearSource: Too many to list; blame me for inaccuracies
2011 - Growing Pains & Early AdoptersFeb2nd Summit21Rackspaceannounces plansto launchindependentFoundation in2012OctApr3rd Summit(Santa Clara)addsConferenceGovernance moves forward withproject technical leads (PTL),policy board elections (PPB)JulFirstAnniversaryBexarOpenStack Compute for mid-size prodOpenStack Image Service added to coreCactusOpenStack Compute for larger-scale prodSepDiabloMajor stability releaseFirst 6-mo cycle release2011Decision to shift from 3-mo to6-mo dev cycleJan1st SwiftPublic CloudInternap w/ CloudscalingHappyBirthday!1st NovaPublic CloudInternap w/ Cloudscaling
Createdframework forFoundation asa communityFeb Apr Aug19 companiesannouncepublic supportfor FoundationFramework &documents ratiﬁedby community22Drafting committeeformed – creatinglegal documentsOpenStackFoundation“ofﬁcially”launchesSepEssexOpenStack Identity in coreOpenStack Dashboard in coreFolsomOpenStack BlockStorage in coreOpenStackNetworking in core2012MayHPCloudLaunch(Beta)Citrix Bails(how’s that going for ya?)JanAT&T JoinsOpenStackInternal production (private)OctGartnerReport(teeth gnashing followed)Inaugural OpenStackFoundation BoardmeetingVMware, Intel, & NECaccepted as Gold membersBoardElections2012 - Rise of the Foundation & Prod Deployments
2013/2014 - Breakout Growth YearsApr Oct23Q1HavanaOpenStack Metering in integrationOpenStack Orchestration in integrationLBaaS?“I” Release2014GrizzlyOpenStack Metering in incubationOpenStack Orchestration in incubation2013First Summit100% run andfunded byFoundationFirstInternationalSummit(APAC?)
In every single category, thetop 3 vendors support OpenStackIncredible Industry Support24top 3 switch vendors top 3 storage vendors top 3 hypervisorstop 3 router vendors top 3 blade vendors top 3 linux vendorstop 3 x86 vendors
Developer Growth25Contributors per month (ohloh)
Developer Growth ComparisonContributors per month (ohloh)26
Dev Growth by Git ContributorsQingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=312027
Accumulated CommunityQingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=312028
Growth by Domain (company - roughly)Qingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=312029For the CloudStackprojects, inﬂuence fromCitrix is quite obvious, over45% of github.comcommits come fromaccounts belonging tocitrix.com and cloud.com.
0750150022503000Austin Santa Clara San Francisco PortlandSummit Growth30
Established Marketing ReachOpenStack.org 241k/visits month:Software: 300K downloadsMembership: 9000+, Over 90% subscribe to newsletterRelationships with Tier 1 publications and analysts3117,693Followers(+50% from 8/12)
CLI toolsDashboard Other toolsComputeNetworkingOrchestrationHypervisor(s) QueuingDatabase/ KVS/ CacheExternalBlockProviderPhysicalNetworkProviderProvisioningLogAggregationHealthMonitoringetc.REST Meter DataRESTSQL, etc. Varies Varies Varies AMQP/0MQTopology &MetadataMeteringRESTREST RESTDNSImageManagementIdentityManagementRESTBlockStorageObjectStorageOpenStack (m)Architecture Slide34UI LayerElasticServicesLayerOtherstuff,youprobablyneed/wantData &ResourceLayerSharedServicesLayer
ProjectNameDescription LayerAWSEquivalentCodenameDashboard Self-service, role-based webinterface for users and administratorsUI Console HorizonCompute Provision and manage large pools ofon-demand computing resourcesElasticServiceEC2 NovaBlock Storage Volumes on commodity storage gear,and drivers for turn-key block storagesolutionsElasticServiceEBS CinderObjectStoragePetabytes of reliable storage onstandard gearElasticServiceS3 SwiftNetworking L2-focused on-demand networkingwith some L3 capabilitiesElasticServiceVPC QuantumOrchestration Application orchestration layer thatruns on top of and managesOpenStack ComputeElasticServiceCloudFormation,CloudWatchHeatMetering Centralized metering data for allservices for integration to externalbillingSharedServiceN/A CeilometerIdentity Multi-tenant authentication systemthat ties to existing stores (e.g.LDAP) and Image ServiceSharedServiceNone KeystoneImageManagementUpload, download, and manage VMimages for the compute serviceSharedServiceVM Import/ExportGlance35
6 month integrated release cycleEvery 6 months, we coordinate and integrate:Thousands of patches & commitsAcross hundreds of developersAnd 9 “integrated” or “core” projectsCompletely impossible without:The OpenStack infrastructure team (CI, etc.)Dedicated PTLs and individual developersNo other similar project does this36
OpenStack is Well OrganizedQingye Jiang (John) - Open Source IaaS Community Analysis CY13 - Q1http://www.qyjohn.net/?p=312037“[the coordinatedreleases are] anindicator that theOpenStackproject is wellorganized in termsof sub-projectmanagement.”
A Quick Note of Thanks39These diagrams would not have beenpossible without the prior work of:Ken Pepple, Solinea (@ken_pepple)Dina Belova, Mirantis... and the help of several Cloudscalers:Eric Windisch (@ewindisch)Joe Gordon (http://github.com/jogo)Matt Joyce (@openfly, http://www.music-piracy.com)Dan Sneddon (@dxs)Joseph Glanville (@jpgvm)
Caveat Emptor40The focus for these diagrams was ease ofreading, not accuracy.See Ken Pepple’s originals or the code if you need truth.That being said, our team tried really hard for accuracy.Blame me for any errors.
Compute Thoughts44Nova still runs best w/ KVMDo we need another hypervisor? What’s the biz case?Multiple Availability Zones still not solvedCells are for making one AZ biggercomplex, tight-couplingWe need a clean sharding mechanism for AZesalso what about Cinder and Quantum?Integ. to Cinder & Quantum needs rethinkMore information needs to be able to be passed back
Compute (Networking) Thoughts45nova-network still requiredQuantum has been L2 focused & L3 gap still existscentralized nova-networking is #faildecentralized is more #failnova-conductor security for hypervisor obviatednova-metadata-api & nova-network on every hypervisor?security implicationsreconciling Quantum and nova-network?Quantum needs more L3 capability, but ...
Block Storage Thoughts47Default “nova-volume” func. is too minimalWhen people think Block Storage service they assume:Persistent, Network-based, & Performant - it isn’tCinder scheduler needs info from NovaAssuming you want to do anything interestingPoint of lock-in since default isn’t usefulYou have to place a bet on a block storage solutionThese are expensive, experimental or proprietary
Networking Thoughts49Default networking functionality is minimalThe APIs have been L2-centricL3 functionality is same as existed with nova-networkSame architecture, same basic layout, with all of the downsidesNeeds a Quantum plugin for full func.Can’t run more than one plugin at a time per functionOnly truly baked plugin is probably Nicira?Others in process, but it’s not clear how many production deployments there areGood news is that this area is hotSo hopefully this is unstuck soon
Object Storage Thoughts51Swift has continued to lag OpenStack dev3 yrs on, auditor is slow & does not prioritize replicationContainer replication is a bad hackUgly stepchild of OpenStackKeystone authentication woes (integration, performance)Isn’t universally loved like NovaSad, since it was the more mature of the two projects at launch in 2010Good news: this area has new playersEVault, Seagate, SwiftStack
Image Mgmt Thoughts53Still not clear why this is standalone projectReally a sub-function of ComputeSemi-pluggable (but not really)Uses different backends for image storageTo be really useful it needs more features:P2V, V2V, and other image conversionAbility to slipstream PV drivers into imagesConvert from popular formats: OVF, AMI, etc.
Identity Mgmt Thoughts55Mixed identity / schizophreniaVerifies identity, authorization, AND service registryService registry is one of manyNova, Cinder, et al have their own internal registriesSlows everything downSee LivingSocial preso from Folsom SummitSee caching tricks with memcache some projects useOpenStack needed to reinvent wheel here?This could have just been LDAP with a schema + caching
OpenStack Dashboard (Horizon)56Source: Wikipedia (http://en.wikipedia.org/wiki/Model–view–controller)
Dashboard Thoughts57It’s gotten a lot betterSame UI for end-user and admin is bad ideaCloudStack did this and it was a messThe workflows and views are too differentSecurity considerations existGeneral lag: many things aren’t accessiblee.g. HeatNeed better docs on extending, w/o harmCustomers, product companies, SPs all want to modifyAllow for customizations, while supporting upgrades, etc.
Metering Thoughts59Metering systems are hardBad or incomplete data for SPs is existentialMetering system should be *very* baked (is 1yr enough?)No tokenized meter dataInstance hours not enoughHow do you bill for Windows, Oracle, RHEL licenses?Tokens stack: size of instance, OS and app licenses, etc.Needs to get flow data from edge switchesNetflow and/or Sflow support for physical switchesData from the vSwitches is not the best source in the real world
Orchestration Thoughts61Huge potentialAdds additional AWS func: CloudWatch, CloudFormationProvides clean templates for stacks, which means:OpenStack on OpenStack (OoO) for testing, etc.First primary project that rides “on top”Clear differentiator over other projectsInitiative: Heat templates for Ref ArchVendors, customers, etc. could feed to prov systems:Crowbar, Piston, Cloudscaling, etc.
Strengths:De facto winnerIncredible communityUnstoppable velocityClear innovation curveSWOT: OpenStack62Opportunities:Build an SQL92 base for cloudcompute (see Threats)Public cloud compatibility as ﬁrstorder initiativevCloud private cloud compatibilityas ﬁrst order initiativeWeaknesses:No benevolent dictatorLack of IaaS experience for manydevelopersInteroperability will be difﬁcultNot impossible, *difﬁcult*Threats:Splintering, fragmentation, andcustomizationForking or ivory tower thinking
First OpenStack Survey71414#survey#responses#16%7%8%4%11%17%37%More#than#10,000#employees#5,001#to#10,000#employees#1,001#to#5,000#employees#501#to#1,000#employees#101#to#500#employees#217100#employees#1720#employees#Company SizeInformation Technology 60%#Academic / Research 15%#Telecommunication 10%#IndustriesGovernment / Defense 3%#CC Icons http://vathanx.deviantart.com/175 29 28 23 1856countriesCountry124$100$77$151$ServiceProviderEcosystemVendorCloudConsumerCloudOperatorType of Involvement
Deployments at a GlanceType35#Hosted#Private# 15#Hybrid##37#Public#106#On#Premise#Private#Trunk&8%&Grizzly&15%&Folsom&47%&Essex&25%&Diablo&5%&Version849294Production Proof ofConceptDev/QAStage134#94#94#89#66#46#DashboardObject StorageSnapshotting to new imagesLive MigrationEC2 Compatibility APIS3 Compatibility APIFeatures181#171#169#153#147#121#103#20#16#NovaGlanceKeystoneHorizonQuantumCinderSwiftCeilometerHeatComponents197&Deployments&
Size of 98 Production Compute Systems731"100$$52%$101"500$18%$ 501"1,000$$8%$1,001"5,000$$8%$5,000"10,000$$3%$>$10,000$$6%$Unspeciﬁed$5%$Other$30%$Instances1"50$$71%$51"100$$8%$101"500$$9%$501"1,000$$2%$>1,000$$4%$unspeciﬁed$6%$Other$29%$Nodes1"100$$51%$101"500$$21%$ 501"1,000$$4%$1,001"5,000$$12%$5,001"10,000$3%$>10,000$$4%$unspeciﬁed$5%$Other$16%$Cores