The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

8,030 views
8,012 views

Published on

Published in: Technology
0 Comments
8 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
8,030
On SlideShare
0
From Embeds
0
Number of Embeds
6,007
Actions
Shares
0
Downloads
0
Comments
0
Likes
8
Embeds 0
No embeds

No notes for slide

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

  1. 1. Apache Cloudstack : A not-so-cloudy future @chiradeep FUTURE Collab 2012Tuesday, December 4, 2012 1
  2. 2. DisclaimerTuesday, December 4, 2012 2
  3. 3. Disclaimer • Prognostications – From experience – But feel free to disagreeTuesday, December 4, 2012 2
  4. 4. Disclaimer • Prognostications – From experience – But feel free to disagree • Not laying out a roadmap – Or timelineTuesday, December 4, 2012 2
  5. 5. Disclaimer • Prognostications – From experience – But feel free to disagree • Not laying out a roadmap – Or timeline • I reserve – The right to change my mind – The right to be wildly optimistic – The right to be biased (to AWS) – The right to make a fool of myselfTuesday, December 4, 2012 2
  6. 6. Not covered • In the long run …we are all dead • In the very short term … point features, critical bug fixes … important butTuesday, December 4, 2012 3
  7. 7. Not covered • In the long run …we are all dead • In the very short term … point features, critical bug fixes … important but • Consult jira and the wiki and mailing list – https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack +Next+ReleaseTuesday, December 4, 2012 3
  8. 8. InfluencersTuesday, December 4, 2012 4
  9. 9. Influencers End-usersTuesday, December 4, 2012 4
  10. 10. Influencers Developers End-usersTuesday, December 4, 2012 4
  11. 11. Influencers Developers Vendors End-usersTuesday, December 4, 2012 4
  12. 12. Influencers Developers Vendors End-users OperatorsTuesday, December 4, 2012 4
  13. 13. Influencers Contributors Developers Vendors End-users OperatorsTuesday, December 4, 2012 4
  14. 14. Influencers Contributors Developers Vendors End-users Operators These are in turn influenced by…Tuesday, December 4, 2012 4
  15. 15. Outside InfluencesTuesday, December 4, 2012 5
  16. 16. Outside Influences AWSTuesday, December 4, 2012 5
  17. 17. Outside Influences OTHER STACKS AWSTuesday, December 4, 2012 5
  18. 18. Outside Influences OTHER STACKS Other clouds (GCE) AWSTuesday, December 4, 2012 5
  19. 19. Outside Influences Secular Trends OTHER STACKS Other clouds (GCE) AWSTuesday, December 4, 2012 5
  20. 20. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etcTuesday, December 4, 2012 6
  21. 21. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etc Can’t impact, but:Tuesday, December 4, 2012 6
  22. 22. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etc Can’t impact, but: • Benefit from itTuesday, December 4, 2012 6
  23. 23. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etc Can’t impact, but: • Benefit from it • Anticipate itTuesday, December 4, 2012 6
  24. 24. End-usersTuesday, December 4, 2012 7
  25. 25. End-users • Features, Features, Features – Clear roadmap – Well documented guides – Fast pace of deliveryTuesday, December 4, 2012 7
  26. 26. End-users • Features, Features, Features – Clear roadmap – Well documented guides – Fast pace of delivery • What does the future hold? – More features, quicker delivery – Javelin refactor should help – https://cwiki.apache.org/confluence/display/ CLOUDSTACK/JavelinTuesday, December 4, 2012 7
  27. 27. End-usersTuesday, December 4, 2012 8
  28. 28. End-users • Stability – Just works – Understandable error messages – MonitoringTuesday, December 4, 2012 8
  29. 29. End-users • Stability – Just works – Understandable error messages – Monitoring • What does the Future hold? – Continuous improvement / bug fixes – Continuous integration, automated testingTuesday, December 4, 2012 8
  30. 30. End-usersTuesday, December 4, 2012 9
  31. 31. End-users • Smooth upgrades – Zero – to – limited – downtime (API availability) during upgradesTuesday, December 4, 2012 9
  32. 32. End-users • Smooth upgrades – Zero – to – limited – downtime (API availability) during upgrades • What does the Future hold? – Javelin branch: Modular architecture implies individually upgradable components – Separate api endpoints and orchestrationTuesday, December 4, 2012 9
  33. 33. End-usersTuesday, December 4, 2012 10
  34. 34. End-users • Principle of least surprise – Looks like a cloud they are used toTuesday, December 4, 2012 10
  35. 35. End-users • Principle of least surprise – Looks like a cloud they are used to • What does the Future hold? – More AWS API fidelity • Access control like IAM • Multiple AZ per region – CLI mode fully supported – GCE module anybody?Tuesday, December 4, 2012 10
  36. 36. AWS PlatformTuesday, December 4, 2012 11
  37. 37. AWS Platform AWS Global InfrastructureTuesday, December 4, 2012 11
  38. 38. AWS Platform Networking AWS Global InfrastructureTuesday, December 4, 2012 11
  39. 39. AWS Platform Compute Storage Database Networking AWS Global InfrastructureTuesday, December 4, 2012 11
  40. 40. AWS Platform Application Services Compute Storage Database Networking AWS Global InfrastructureTuesday, December 4, 2012 11
  41. 41. AWS Platform Deployment and Administration Application Services Compute Storage Database Networking AWS Global InfrastructureTuesday, December 4, 2012 11
  42. 42. Mapping to CloudStack FEATURE means not core to CloudStack but CloudStack will work extremely well with itTuesday, December 4, 2012 12
  43. 43. Networking PlatformTuesday, December 4, 2012 13
  44. 44. Networking Platform • VPC – Partial implementation already in CloudStackTuesday, December 4, 2012 13
  45. 45. Networking Platform • VPC – Partial implementation already in CloudStack • Direct Connect – Somewhat covered by VPC private gateway – Specific to AWS infrastructure design.Tuesday, December 4, 2012 13
  46. 46. Networking Platform • VPC – Partial implementation already in CloudStack • Direct Connect – Somewhat covered by VPC private gateway – Specific to AWS infrastructure design. • Route 53 (DNS anycast infrastructure)Tuesday, December 4, 2012 13
  47. 47. Networking Platform • VPC – Partial implementation already in CloudStack • Direct Connect – Somewhat covered by VPC private gateway – Specific to AWS infrastructure design. • Route 53 (DNS anycast infrastructure) • Future – Full-fidelity of VPC implementation using SDN – ENI – ELB and EIP in VPC – (More) Commercial Network Devices / Appliances – IPv6Tuesday, December 4, 2012 13
  48. 48. Storage ServicesTuesday, December 4, 2012 14
  49. 49. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS)Tuesday, December 4, 2012 14
  50. 50. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semanticsTuesday, December 4, 2012 14
  51. 51. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semantics • GlacierTuesday, December 4, 2012 14
  52. 52. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semantics • Glacier • Storage GatewayTuesday, December 4, 2012 14
  53. 53. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semantics • Glacier • Storage Gateway • What does the Future hold? – Storage refactor == More EBS back-ends – Zone-wide visibility of volumes – HDFS backend to S3 – NFS secondary -> S3 syncTuesday, December 4, 2012 14
  54. 54. Compute Services • EC2 – Full-featured already • Autoscale – Netscaler provider • ELB – Virtual router – Netscaler – F5Tuesday, December 4, 2012 15
  55. 55. Compute Services • What does the Future hold? – EC2 • Dedicated instances • Multi-instance launch • Better API fidelity • GCE api endpoint? – Autoscale • Non-netscaler provider – ELB • Health checks • SSL termination • IPV6 • Brocade ADX • Auto-scaled virtual routerTuesday, December 4, 2012 16
  56. 56. Database ServicesTuesday, December 4, 2012 17
  57. 57. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBaseTuesday, December 4, 2012 17
  58. 58. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBase • RDSTuesday, December 4, 2012 17
  59. 59. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBase • RDS • ElastiCacheTuesday, December 4, 2012 17
  60. 60. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBase • RDS • ElastiCache • What does the Future Hold? – RDS and ElastiCache could be subprojects? – Or new Apache projectsTuesday, December 4, 2012 17
  61. 61. Application Services • CloudFront • Elastic MapReduce (EMR) • CloudSearch • Simple Email Service (SES) • Simple Queue Service (SQS) • Simple Notification Service (SNS) • Simple Workflow Service (SWF)Tuesday, December 4, 2012 18
  62. 62. Application Services • CloudFront • Elastic MapReduce (EMR) • CloudSearch • Simple Email Service (SES) • Simple Queue Service (SQS) • Simple Notification Service (SNS) • Simple Workflow Service (SWF) • CloudStack will enable other projects / vendors that provide this – Like S3 – IAM a key enablerTuesday, December 4, 2012 18
  63. 63. Deployment and Administration • CloudWatch • IAM • Management Console • CloudFormation • Elastic Beanstalk (PAAS)Tuesday, December 4, 2012 19
  64. 64. Deployment and Administration • CloudWatch • IAM • Management Console • CloudFormation • Elastic Beanstalk (PAAS) • What does the Future hold? – New subprojects for CloudWatch and CloudFormation ? – Better UI/UX experience – IAM: • Who, what, from where, when, for how long • Expand to include more Governance: – Call-out to governance service, e.g., approval, verify ticketsTuesday, December 4, 2012 19
  65. 65. DevelopersTuesday, December 4, 2012 20
  66. 66. Developer NeedsTuesday, December 4, 2012 21
  67. 67. Developer Needs • Add new features faster – Continuous integration – Modular / loosely coupled architecture – Well documented interfaces • Principle of least surprise – Use well-known libraries / frameworks – Use best tool / language for task at handTuesday, December 4, 2012 21
  68. 68. Developer Needs • Add new features faster – Continuous integration – Modular / loosely coupled architecture – Well documented interfaces • Principle of least surprise – Use well-known libraries / frameworks – Use best tool / language for task at hand • What does the future hold? – Javelin branch for loose coupled architecture – Well known components for DI, messaging, packaging, build – Jenkins improvements – continuous automated testsTuesday, December 4, 2012 21
  69. 69. Developer NeedsTuesday, December 4, 2012 22
  70. 70. Developer Needs • Fast code-debug-test cycle – DevCloud – Bare minimum moving parts (no system vms)Tuesday, December 4, 2012 22
  71. 71. Developer Needs • Fast code-debug-test cycle – DevCloud – Bare minimum moving parts (no system vms) • What does the Future Hold? – Improvements to DevCloud – Deployment without system vms • Enable new hypervisors • Laptop-based hypervisors (vbox, v. workstation, v.fusion) – System VM and virtual router will be decoupled/ separate modulesTuesday, December 4, 2012 22
  72. 72. Developer NeedsTuesday, December 4, 2012 23
  73. 73. Developer Needs • Process – Visibility into roadmap – Code submission criteria needs to be easy and non-onerous – Clear guidelines on what belongs in CloudStackTuesday, December 4, 2012 23
  74. 74. Developer Needs • Process – Visibility into roadmap – Code submission criteria needs to be easy and non-onerous – Clear guidelines on what belongs in CloudStack • What does the Future Hold? – Process improvements – Release managers to keep on top of thingsTuesday, December 4, 2012 23
  75. 75. Cloud OperatorsTuesday, December 4, 2012 24
  76. 76. Cloud operator needsTuesday, December 4, 2012 25
  77. 77. Cloud operator needs • Easy install – #1 predictor whether the POC goes forward – System VMs are boon or bane?Tuesday, December 4, 2012 25
  78. 78. Cloud operator needs • Easy install – #1 predictor whether the POC goes forward – System VMs are boon or bane? • What does the Future Hold: – Optional System VMs – More forgiving install methodsTuesday, December 4, 2012 25
  79. 79. Cloud operator needsTuesday, December 4, 2012 26
  80. 80. Cloud operator needs • Scalable – Already proven in # of hypervisors – Need scalable networking (SDN)Tuesday, December 4, 2012 26
  81. 81. Cloud operator needs • Scalable – Already proven in # of hypervisors – Need scalable networking (SDN) • What does the Future Hold: – Optional System VMs – Productize native SDN (GRE-based) – IPv6Tuesday, December 4, 2012 26
  82. 82. Cloud operator needsTuesday, December 4, 2012 27
  83. 83. Cloud operator needs • Flexibility – Choice of hypervisors, networking providers, storage providers – Flexibility in orchestrationTuesday, December 4, 2012 27
  84. 84. Cloud operator needs • Flexibility – Choice of hypervisors, networking providers, storage providers – Flexibility in orchestration • What does the Future Hold: – More hypervisors supported • Hyper-V, Oracle VM, PowerVM, Solaris Zones – Choice of SDN and networking providers – Choice of storage backends (volume and object) – Customizable orchestration flows?Tuesday, December 4, 2012 27
  85. 85. Cloud operator needsTuesday, December 4, 2012 28
  86. 86. Cloud operator needs • Resilient – Occam’s razor. Should not be more complicated than necessary. • Enterprise needs different from SP needs – Handle zone failures – Automated, semi-automated and manual recoveryTuesday, December 4, 2012 28
  87. 87. Cloud operator needs • Resilient – Occam’s razor. Should not be more complicated than necessary. • Enterprise needs different from SP needs – Handle zone failures – Automated, semi-automated and manual recovery • What does the Future Hold? – Modular architecture means components are optional – Databases are split – Per-zone orchestration layer – Well documented upgrade proceduresTuesday, December 4, 2012 28
  88. 88. Cloud operator needsTuesday, December 4, 2012 29
  89. 89. Cloud operator needs • Adaptive – Graceful degradation and admission control – Easy well-defined upgrades – Continuous deployment?Tuesday, December 4, 2012 29
  90. 90. Cloud operator needs • Adaptive – Graceful degradation and admission control – Easy well-defined upgrades – Continuous deployment? • What does the Future Hold? – Independently scaled stateless API endpoints – Well documented zero-downtime upgrades – Individual service upgradeTuesday, December 4, 2012 29
  91. 91. Cloud operator needs • Monitoring • SecureTuesday, December 4, 2012 30
  92. 92. Cloud operator needs • Monitoring • Secure • What does the Future Hold? – Well documented attack surface so that it can be defended. – Informative logs – Lots of instrumentation – Well integrated monitoring backendsTuesday, December 4, 2012 30
  93. 93. Cloud operator needs • Supportability • Chargeback and usageTuesday, December 4, 2012 31
  94. 94. Cloud operator needs • Supportability • Chargeback and usage • What does the Future Hold? – Troubleshoot / replicate end-user problems – Override automated recovery processes when required – Powerful end-user GUI – Basic admin GUI – AlertsTuesday, December 4, 2012 31
  95. 95. Cloud Operator Needs • Support for more hypervisors • Integrate existing virtualization deployments into CloudStackTuesday, December 4, 2012 32
  96. 96. Cloud Operator Needs • Support for more hypervisors • Integrate existing virtualization deployments into CloudStack • What does the Future Hold: – Optional system vm means support for • LXC, PowerVM, LPAR, Solaris Zones, Baremetal • Laptop hypervisors (Workstation, Fusion,vBox) – Virtualization ImportTuesday, December 4, 2012 32
  97. 97. Cloud Operator NeedsTuesday, December 4, 2012 33
  98. 98. Cloud Operator Needs • Integrate with existing datacenter infrastructure / processesTuesday, December 4, 2012 33
  99. 99. Cloud Operator Needs • Integrate with existing datacenter infrastructure / processes • What does the Future Hold? – DNS, DHCP, LDAP, IPAM integration hooks – Monitoring systems hooks – Event-based integration into • CMDB, Ticketing systems • Service management, change management, service desk, capacity planning, release planningTuesday, December 4, 2012 33
  100. 100. VendorsTuesday, December 4, 2012 34
  101. 101. Vendors • Strong communityTuesday, December 4, 2012 34
  102. 102. Vendors • Strong community • Well-documented integration points – Plugins – vs – servicesTuesday, December 4, 2012 34
  103. 103. Vendors • Strong community • Well-documented integration points – Plugins – vs – services • Testability outside of full-fledged CS deploymentTuesday, December 4, 2012 34
  104. 104. Vendors • Strong community • Well-documented integration points – Plugins – vs – services • Testability outside of full-fledged CS deployment • What does the Future hold? – More documentation – Integration test suites that can run without CSTuesday, December 4, 2012 34
  105. 105. AWS Virtuous Cycle Reduced   Prices Lower   More   Infra  costs Customers More   Economie AWS   s  of   Usage More   Infrastruct ureTuesday, December 4, 2012 35
  106. 106. What is CloudStack’s Virtuous Great   So>ware More   More   features operators More  Tuesday, December 4, 2012 36
  107. 107. What is CloudStack’s Virtuous Great   So>ware More vendors Expanding ecosystem More   More   features operators More  Tuesday, December 4, 2012 36

×