Successfully reported this slideshow.
Your SlideShare is downloading. ×

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

More Related Content

Related Books

Free with a 30 day trial from Scribd

See all

The Future of Apache CloudStack (Not So Cloudy) (Collab 2012)

  1. 1. Apache Cloudstack : A not-so-cloudy future @chiradeep FUTURE Collab 2012 Tuesday, December 4, 2012 1
  2. 2. Disclaimer Tuesday, December 4, 2012 2
  3. 3. Disclaimer • Prognostications – From experience – But feel free to disagree Tuesday, December 4, 2012 2
  4. 4. Disclaimer • Prognostications – From experience – But feel free to disagree • Not laying out a roadmap – Or timeline Tuesday, December 4, 2012 2
  5. 5. Disclaimer • Prognostications – From experience – But feel free to disagree • Not laying out a roadmap – Or timeline • I reserve – The right to change my mind – The right to be wildly optimistic – The right to be biased (to AWS) – The right to make a fool of myself Tuesday, December 4, 2012 2
  6. 6. Not covered • In the long run …we are all dead • In the very short term … point features, critical bug fixes … important but Tuesday, December 4, 2012 3
  7. 7. Not covered • In the long run …we are all dead • In the very short term … point features, critical bug fixes … important but • Consult jira and the wiki and mailing list – https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack +Next+Release Tuesday, December 4, 2012 3
  8. 8. Influencers Tuesday, December 4, 2012 4
  9. 9. Influencers End-users Tuesday, December 4, 2012 4
  10. 10. Influencers Developers End-users Tuesday, December 4, 2012 4
  11. 11. Influencers Developers Vendors End-users Tuesday, December 4, 2012 4
  12. 12. Influencers Developers Vendors End-users Operators Tuesday, December 4, 2012 4
  13. 13. Influencers Contributors Developers Vendors End-users Operators Tuesday, December 4, 2012 4
  14. 14. Influencers Contributors Developers Vendors End-users Operators These are in turn influenced by… Tuesday, December 4, 2012 4
  15. 15. Outside Influences Tuesday, December 4, 2012 5
  16. 16. Outside Influences AWS Tuesday, December 4, 2012 5
  17. 17. Outside Influences OTHER STACKS AWS Tuesday, December 4, 2012 5
  18. 18. Outside Influences OTHER STACKS Other clouds (GCE) AWS Tuesday, December 4, 2012 5
  19. 19. Outside Influences Secular Trends OTHER STACKS Other clouds (GCE) AWS Tuesday, December 4, 2012 5
  20. 20. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etc Tuesday, December 4, 2012 6
  21. 21. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etc Can’t impact, but: Tuesday, December 4, 2012 6
  22. 22. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etc Can’t impact, but: • Benefit from it Tuesday, December 4, 2012 6
  23. 23. Secular forces • Moore’s Law • DevOps / Developer as kingmaker • OSS ecosystems • Mobility • Power / green computing • etc Can’t impact, but: • Benefit from it • Anticipate it Tuesday, December 4, 2012 6
  24. 24. End-users Tuesday, December 4, 2012 7
  25. 25. End-users • Features, Features, Features – Clear roadmap – Well documented guides – Fast pace of delivery Tuesday, December 4, 2012 7
  26. 26. End-users • Features, Features, Features – Clear roadmap – Well documented guides – Fast pace of delivery • What does the future hold? – More features, quicker delivery – Javelin refactor should help – https://cwiki.apache.org/confluence/display/ CLOUDSTACK/Javelin Tuesday, December 4, 2012 7
  27. 27. End-users Tuesday, December 4, 2012 8
  28. 28. End-users • Stability – Just works – Understandable error messages – Monitoring Tuesday, December 4, 2012 8
  29. 29. End-users • Stability – Just works – Understandable error messages – Monitoring • What does the Future hold? – Continuous improvement / bug fixes – Continuous integration, automated testing Tuesday, December 4, 2012 8
  30. 30. End-users Tuesday, December 4, 2012 9
  31. 31. End-users • Smooth upgrades – Zero – to – limited – downtime (API availability) during upgrades Tuesday, December 4, 2012 9
  32. 32. End-users • Smooth upgrades – Zero – to – limited – downtime (API availability) during upgrades • What does the Future hold? – Javelin branch: Modular architecture implies individually upgradable components – Separate api endpoints and orchestration Tuesday, December 4, 2012 9
  33. 33. End-users Tuesday, December 4, 2012 10
  34. 34. End-users • Principle of least surprise – Looks like a cloud they are used to Tuesday, December 4, 2012 10
  35. 35. End-users • Principle of least surprise – Looks like a cloud they are used to • What does the Future hold? – More AWS API fidelity • Access control like IAM • Multiple AZ per region – CLI mode fully supported – GCE module anybody? Tuesday, December 4, 2012 10
  36. 36. AWS Platform Tuesday, December 4, 2012 11
  37. 37. AWS Platform AWS Global Infrastructure Tuesday, December 4, 2012 11
  38. 38. AWS Platform Networking AWS Global Infrastructure Tuesday, December 4, 2012 11
  39. 39. AWS Platform Compute Storage Database Networking AWS Global Infrastructure Tuesday, December 4, 2012 11
  40. 40. AWS Platform Application Services Compute Storage Database Networking AWS Global Infrastructure Tuesday, December 4, 2012 11
  41. 41. AWS Platform Deployment and Administration Application Services Compute Storage Database Networking AWS Global Infrastructure Tuesday, December 4, 2012 11
  42. 42. Mapping to CloudStack FEATURE means not core to CloudStack but CloudStack will work extremely well with it Tuesday, December 4, 2012 12
  43. 43. Networking Platform Tuesday, December 4, 2012 13
  44. 44. Networking Platform • VPC – Partial implementation already in CloudStack Tuesday, December 4, 2012 13
  45. 45. Networking Platform • VPC – Partial implementation already in CloudStack • Direct Connect – Somewhat covered by VPC private gateway – Specific to AWS infrastructure design. Tuesday, December 4, 2012 13
  46. 46. Networking Platform • VPC – Partial implementation already in CloudStack • Direct Connect – Somewhat covered by VPC private gateway – Specific to AWS infrastructure design. • Route 53 (DNS anycast infrastructure) Tuesday, December 4, 2012 13
  47. 47. Networking Platform • VPC – Partial implementation already in CloudStack • Direct Connect – Somewhat covered by VPC private gateway – Specific to AWS infrastructure design. • Route 53 (DNS anycast infrastructure) • Future – Full-fidelity of VPC implementation using SDN – ENI – ELB and EIP in VPC – (More) Commercial Network Devices / Appliances – IPv6 Tuesday, December 4, 2012 13
  48. 48. Storage Services Tuesday, December 4, 2012 14
  49. 49. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) Tuesday, December 4, 2012 14
  50. 50. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semantics Tuesday, December 4, 2012 14
  51. 51. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semantics • Glacier Tuesday, December 4, 2012 14
  52. 52. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semantics • Glacier • Storage Gateway Tuesday, December 4, 2012 14
  53. 53. Storage Services • S3 – Essential for a Multi-AZ deployment • Multiple backends (NFS, Cloudian, Caringo, Basho CS) • EBS – CloudStack supports EBS semantics • Glacier • Storage Gateway • What does the Future hold? – Storage refactor == More EBS back-ends – Zone-wide visibility of volumes – HDFS backend to S3 – NFS secondary -> S3 sync Tuesday, December 4, 2012 14
  54. 54. Compute Services • EC2 – Full-featured already • Autoscale – Netscaler provider • ELB – Virtual router – Netscaler – F5 Tuesday, December 4, 2012 15
  55. 55. Compute Services • What does the Future hold? – EC2 • Dedicated instances • Multi-instance launch • Better API fidelity • GCE api endpoint? – Autoscale • Non-netscaler provider – ELB • Health checks • SSL termination • IPV6 • Brocade ADX • Auto-scaled virtual router Tuesday, December 4, 2012 16
  56. 56. Database Services Tuesday, December 4, 2012 17
  57. 57. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBase Tuesday, December 4, 2012 17
  58. 58. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBase • RDS Tuesday, December 4, 2012 17
  59. 59. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBase • RDS • ElastiCache Tuesday, December 4, 2012 17
  60. 60. Database Services • Dynamo DB (NoSQL) – Apache Cassandra, Riak, Apache HBase • RDS • ElastiCache • What does the Future Hold? – RDS and ElastiCache could be subprojects? – Or new Apache projects Tuesday, December 4, 2012 17
  61. 61. Application Services • CloudFront • Elastic MapReduce (EMR) • CloudSearch • Simple Email Service (SES) • Simple Queue Service (SQS) • Simple Notification Service (SNS) • Simple Workflow Service (SWF) Tuesday, December 4, 2012 18
  62. 62. Application Services • CloudFront • Elastic MapReduce (EMR) • CloudSearch • Simple Email Service (SES) • Simple Queue Service (SQS) • Simple Notification Service (SNS) • Simple Workflow Service (SWF) • CloudStack will enable other projects / vendors that provide this – Like S3 – IAM a key enabler Tuesday, December 4, 2012 18
  63. 63. Deployment and Administration • CloudWatch • IAM • Management Console • CloudFormation • Elastic Beanstalk (PAAS) Tuesday, December 4, 2012 19
  64. 64. Deployment and Administration • CloudWatch • IAM • Management Console • CloudFormation • Elastic Beanstalk (PAAS) • What does the Future hold? – New subprojects for CloudWatch and CloudFormation ? – Better UI/UX experience – IAM: • Who, what, from where, when, for how long • Expand to include more Governance: – Call-out to governance service, e.g., approval, verify tickets Tuesday, December 4, 2012 19
  65. 65. Developers Tuesday, December 4, 2012 20
  66. 66. Developer Needs Tuesday, December 4, 2012 21
  67. 67. Developer Needs • Add new features faster – Continuous integration – Modular / loosely coupled architecture – Well documented interfaces • Principle of least surprise – Use well-known libraries / frameworks – Use best tool / language for task at hand Tuesday, December 4, 2012 21
  68. 68. Developer Needs • Add new features faster – Continuous integration – Modular / loosely coupled architecture – Well documented interfaces • Principle of least surprise – Use well-known libraries / frameworks – Use best tool / language for task at hand • What does the future hold? – Javelin branch for loose coupled architecture – Well known components for DI, messaging, packaging, build – Jenkins improvements – continuous automated tests Tuesday, December 4, 2012 21
  69. 69. Developer Needs Tuesday, December 4, 2012 22
  70. 70. Developer Needs • Fast code-debug-test cycle – DevCloud – Bare minimum moving parts (no system vms) Tuesday, December 4, 2012 22
  71. 71. Developer Needs • Fast code-debug-test cycle – DevCloud – Bare minimum moving parts (no system vms) • What does the Future Hold? – Improvements to DevCloud – Deployment without system vms • Enable new hypervisors • Laptop-based hypervisors (vbox, v. workstation, v.fusion) – System VM and virtual router will be decoupled/ separate modules Tuesday, December 4, 2012 22
  72. 72. Developer Needs Tuesday, December 4, 2012 23
  73. 73. Developer Needs • Process – Visibility into roadmap – Code submission criteria needs to be easy and non-onerous – Clear guidelines on what belongs in CloudStack Tuesday, December 4, 2012 23
  74. 74. Developer Needs • Process – Visibility into roadmap – Code submission criteria needs to be easy and non-onerous – Clear guidelines on what belongs in CloudStack • What does the Future Hold? – Process improvements – Release managers to keep on top of things Tuesday, December 4, 2012 23
  75. 75. Cloud Operators Tuesday, December 4, 2012 24
  76. 76. Cloud operator needs Tuesday, December 4, 2012 25
  77. 77. Cloud operator needs • Easy install – #1 predictor whether the POC goes forward – System VMs are boon or bane? Tuesday, December 4, 2012 25
  78. 78. Cloud operator needs • Easy install – #1 predictor whether the POC goes forward – System VMs are boon or bane? • What does the Future Hold: – Optional System VMs – More forgiving install methods Tuesday, December 4, 2012 25
  79. 79. Cloud operator needs Tuesday, December 4, 2012 26
  80. 80. Cloud operator needs • Scalable – Already proven in # of hypervisors – Need scalable networking (SDN) Tuesday, December 4, 2012 26
  81. 81. Cloud operator needs • Scalable – Already proven in # of hypervisors – Need scalable networking (SDN) • What does the Future Hold: – Optional System VMs – Productize native SDN (GRE-based) – IPv6 Tuesday, December 4, 2012 26
  82. 82. Cloud operator needs Tuesday, December 4, 2012 27
  83. 83. Cloud operator needs • Flexibility – Choice of hypervisors, networking providers, storage providers – Flexibility in orchestration Tuesday, December 4, 2012 27
  84. 84. Cloud operator needs • Flexibility – Choice of hypervisors, networking providers, storage providers – Flexibility in orchestration • What does the Future Hold: – More hypervisors supported • Hyper-V, Oracle VM, PowerVM, Solaris Zones – Choice of SDN and networking providers – Choice of storage backends (volume and object) – Customizable orchestration flows? Tuesday, December 4, 2012 27
  85. 85. Cloud operator needs Tuesday, December 4, 2012 28
  86. 86. Cloud operator needs • Resilient – Occam’s razor. Should not be more complicated than necessary. • Enterprise needs different from SP needs – Handle zone failures – Automated, semi-automated and manual recovery Tuesday, December 4, 2012 28
  87. 87. Cloud operator needs • Resilient – Occam’s razor. Should not be more complicated than necessary. • Enterprise needs different from SP needs – Handle zone failures – Automated, semi-automated and manual recovery • What does the Future Hold? – Modular architecture means components are optional – Databases are split – Per-zone orchestration layer – Well documented upgrade procedures Tuesday, December 4, 2012 28
  88. 88. Cloud operator needs Tuesday, December 4, 2012 29
  89. 89. Cloud operator needs • Adaptive – Graceful degradation and admission control – Easy well-defined upgrades – Continuous deployment? Tuesday, December 4, 2012 29
  90. 90. Cloud operator needs • Adaptive – Graceful degradation and admission control – Easy well-defined upgrades – Continuous deployment? • What does the Future Hold? – Independently scaled stateless API endpoints – Well documented zero-downtime upgrades – Individual service upgrade Tuesday, December 4, 2012 29
  91. 91. Cloud operator needs • Monitoring • Secure Tuesday, December 4, 2012 30
  92. 92. Cloud operator needs • Monitoring • Secure • What does the Future Hold? – Well documented attack surface so that it can be defended. – Informative logs – Lots of instrumentation – Well integrated monitoring backends Tuesday, December 4, 2012 30
  93. 93. Cloud operator needs • Supportability • Chargeback and usage Tuesday, December 4, 2012 31
  94. 94. Cloud operator needs • Supportability • Chargeback and usage • What does the Future Hold? – Troubleshoot / replicate end-user problems – Override automated recovery processes when required – Powerful end-user GUI – Basic admin GUI – Alerts Tuesday, December 4, 2012 31
  95. 95. Cloud Operator Needs • Support for more hypervisors • Integrate existing virtualization deployments into CloudStack Tuesday, December 4, 2012 32
  96. 96. Cloud Operator Needs • Support for more hypervisors • Integrate existing virtualization deployments into CloudStack • What does the Future Hold: – Optional system vm means support for • LXC, PowerVM, LPAR, Solaris Zones, Baremetal • Laptop hypervisors (Workstation, Fusion,vBox) – Virtualization Import Tuesday, December 4, 2012 32
  97. 97. Cloud Operator Needs Tuesday, December 4, 2012 33
  98. 98. Cloud Operator Needs • Integrate with existing datacenter infrastructure / processes Tuesday, December 4, 2012 33
  99. 99. Cloud Operator Needs • Integrate with existing datacenter infrastructure / processes • What does the Future Hold? – DNS, DHCP, LDAP, IPAM integration hooks – Monitoring systems hooks – Event-based integration into • CMDB, Ticketing systems • Service management, change management, service desk, capacity planning, release planning Tuesday, December 4, 2012 33
  100. 100. Vendors Tuesday, December 4, 2012 34
  101. 101. Vendors • Strong community Tuesday, December 4, 2012 34
  102. 102. Vendors • Strong community • Well-documented integration points – Plugins – vs – services Tuesday, December 4, 2012 34
  103. 103. Vendors • Strong community • Well-documented integration points – Plugins – vs – services • Testability outside of full-fledged CS deployment Tuesday, December 4, 2012 34
  104. 104. Vendors • Strong community • Well-documented integration points – Plugins – vs – services • Testability outside of full-fledged CS deployment • What does the Future hold? – More documentation – Integration test suites that can run without CS Tuesday, December 4, 2012 34
  105. 105. AWS Virtuous Cycle Reduced   Prices Lower   More   Infra  costs Customers More   Economie AWS   s  of   Usage More   Infrastruct ure Tuesday, December 4, 2012 35
  106. 106. What is CloudStack’s Virtuous Great   So>ware More   More   features operators More   Tuesday, December 4, 2012 36
  107. 107. What is CloudStack’s Virtuous Great   So>ware More vendors Expanding ecosystem More   More   features operators More   Tuesday, December 4, 2012 36

×