DevOps in a Public OpenStack Cloud - Hui Cheng

2,182 views

Published on

Presented at OpenStack Summit Fall 2012, San Diego, by Hui Cheng. Detail notes see here: http://freedomhui.com/2012/10/devops-in-openstack-public-cloud/

Published in: Technology

DevOps in a Public OpenStack Cloud - Hui Cheng

  1. 1. 在这里写上你的标题DevOps in OpenStack Public Cloud 副标题副标题副标题 副标题文字副标题文字 Presented at OpenStack Summit, Fall 2012, San Diego Hui Cheng freedomhui@gmail.com | freedomhui.com Community Manager of COSUG Technical Manager in Sina Corporation 作者名字/日期 2012/10/17
  2. 2. OpenStack Public Cloud 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05However They never tell you how to operatetheir public cloud based on OpenStack!
  3. 3. Content SinaCloud Introduction Challenges to build a OpenStack Public Cloud 1. Network topology 2. Security Enhancement 3. Storage Solution 4. Identity Integration 5. Billing & Monitoring 6. Dashboard Improvement Operate an production OpenStack 7. Platform stack 8. Automated Deployment 9. Continuous Integration 10. Project Management StackLab: A community OpenStack Public Cloud Summary
  4. 4. Cloud Requirement 目录 00 00 写上你的文字你的文字 Sina.com 01 • Largest infotainment web portal in China 01 • Provides various on-line services, like news, 02 Finance, video, email, blog hosting, etc. 02 • Needs unified infrastructure & app platform to 03 host heterogeneous services and apps. 03Sina Weibo 04• twitter-like microblog service• 05 over 350m users, #1 SNS in China.• huge influence on Chinas society• Weibo Open Platform to build a social ecosystem through Open API and cloud environmental. We are building a reliable, scalable and secure cloud platform to support our business and external customers.
  5. 5. 目录SinaCloud Portfolio • First and most popular PaaS cloud 00 in China, launched in 2009 00 写上你的文字你的文字 • Support PHP, Python and Java 01 runtime. 01 • 250,000 developers, 380,000 apps 02 running on SAE. 02 03 03  First commercial cloud app market in China. 04  SaaS cloud based on SAE tech.  Design for the common users, 05 (Sina Cloud Market) 1-Click purchase and install apps.  First OpenStack based public IaaS cloud in China
  6. 6. 目录 Sina OpenStack dev Team 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05More info: http://www.openstack.org/blog/2012/10/how-sina-contributes-to-openstack/
  7. 7. 目录 Sina OpenStack dev Team 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05More info: http://www.openstack.org/blog/2012/10/how-sina-contributes-to-openstack/
  8. 8. Content SinaCloud Introduction Challenges to build a OpenStack Public Cloud 1. Network topology 2. Security Enhancement 3. Storage Solution 4. Identity Integration 5. Billing & Monitoring 6. Dashboard Improvement Operate an production OpenStack 7. Platform stack 8. Automated Deployment 9. Continuous Integration 10. Project Management StackLab: A community OpenStack Public Cloud Summary
  9. 9. 目录 Network Topology 00 00 写上你的文字你的文字 vs Quantum Nova-network 01 01Multi Multi Flat, Tunneling SDN Sec Dashboard 02 host Talent FlatDHCP Group Support 02Nova-Network 03Quantum 03 04  Nova-Network is simple, robust and reliable, except lack of some 05 advanced features.  Quantum is not ready for production use, it’s OVS plugin has great potential to be open-source NVP solution.  I would suggestion to continue use nova-network for production deployment until next release.
  10. 10. 目录Nova-Network 00 Flat00 写上你的文字你的文字  01 Need external DHCP Server, and human intervention, not 01 flexible, hardly use in practical deployment. FlatDHCP 02 02  Like Amazon EC2 networking(not VPC, VPC corresponds to 03 Quantum), VM get IP from single network pools. 03  Simple, easy to hack.  04 Widely used in public cloud, also preferred topology in many scenarios. 05 VLAN  A little complex, hardware configuration may be involved.  Not suggest to use except strong requirement of tenant isolation,
  11. 11. Network Topology —— Real User CaseNova Network(FlatDHCP+Multi-host)Capability:• Accessibility of all VMs in the fixed IP range• VM is able to access public network• VM can be accessible from public networkBonus:• Totally distributed architecture avoid single-point failure.• Multiple gateway eliminates NAT bottleneck• High speed between OS regionsDrawback:• Tenant isolation lessens• Need security facility(SWS-filter) to protect intranet
  12. 12. 目录Security Enhancement 00 SWS Filter: a extension to security group in nova-network 00 写上你的文字你的文字  01 Used to filter egress traffic from VM to internal network  01 Define whose traffic could be able to reach which internal 02 network IP/segment. 02 03 03 04 05
  13. 13. 目录Storage Solution 00  00 Object写上你的文字你的文字 we choose Swift Storage: Definitely 01  Block Storage  01Cinder is not Amazon EBS, just a framework to include 02 multiple open-source/commercial storage solution. 02  Nova-volume/Cinder(iSCSI) is not applicable to public cloud. 03 Sheepdog/Gluster/Ceph plugins need time to be stable.  03  Island: Local Storage Volume plugin for Cinder is 04 coming. High performance local storage 05   Incremental & independent snapshot  Snapshot store in swift
  14. 14. 目录 Identify Integration: Keystone 00 AWS-like Multi-region support  00 写上你的文字你的文字 01 01 Dashboard 02 02 Keystone Select Region KeystoneNova Beijing Shanghai Nova 03 03Swift Glance 04 Glance Swift 05 MySQL
  15. 15. Kanyun: Monitoring system 目录 00 Worker Dashboard Nova 00 写上你的文字你的文字 01NovaCompute01 Compute 02 Metrics: 02 API daemon CPU、mem、disk、Worker 03 03 Retrieve network traffic usage Responds to client 04 info request 05 Billing Aggregator NoSQLCalculates/stores metrics Repo: https://github.com/sinacloud/kanyun
  16. 16. Kanyun demo 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05
  17. 17. Dough:Billing system 目录  Keep track of billing info to charge tenants 00 Flexible 00 写上你的文字你的文字  deduct customization of 01 payment policies 01 RDBMS Dashboard  How much/often to Kanyun API 02 charge for resource 02 (Metering) unit 03 Handles prepaid or 03  pay-as-you-go Coupon Support 04  05 Farmer API daemon NoSQL Check status / Subscribe or Retrieve usage / unsubscribe Create purchases Query infoRepo: https://github.com/sinacloud/dough, you should also consider Celiometer project.
  18. 18. Dough:Billing info page 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05
  19. 19. 目录Dashboard Improvement 00  00 We did not use Horizon, because: 写上你的文字你的文字 01  Horizon’s UI is not easy to customize  01Front end and back end is tightly coupled 02  we need much customization, it’s hard to keep pace with 02Horizon. 03 03  04 What we do?  Decouple the frontend design and backend implementation. 05  Make dashboard a lightweight frontend.  Separate user console and admin console.
  20. 20. 目录Horizon Dashboard 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05
  21. 21. SWS v1 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05
  22. 22. SWS v2 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05
  23. 23. SWS v3 – User Dashboard 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05
  24. 24. SWS v3 - Monitoring 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05
  25. 25. Content SinaCloud Introduction Challenges to build a OpenStack Public Cloud 1. Network topology 2. Security Enhancement 3. Storage Solution 4. Identity Integration 5. Billing & Monitoring 6. Dashboard Improvement Operate an production OpenStack 7. Platform stack 8. Automated Deployment 9. Continuous Integration 10. Project Management StackLab: A community OpenStack Public Cloud Summary
  26. 26. Platform Stack 目录 00 00 写上你的文字你的文字 01 01 x86 rack Server 2U 02 02 Ubuntu 03 OpenStack 03 KVM 04 05 SAS SSD Raid10/5 Raid10/5
  27. 27. Challenges in Deploying Cloud Cloud in essence are big data centers Requirement:  Provision large scale physical infrastructures  Software deployment  Orchestrate all the heterogeneous components
  28. 28. SWS automation toolchain. 目录 Openstack Montoring 00 Zabbix 00 写上你的文字你的文字 Development tools Cluster 01 Configuration management 01 Deb 02 Puppet Repo 02 03 Build Packages 03 Services Provision Gitorious Operation Tools 04 Peer Review Foreman Provides DHCP/TFTP/DNS 05 and puppetCA for puppet Gerrit OS provision git review Dev Bare Metal
  29. 29. SWS continuous integration 目录 00 Dev 00 写上你的文字你的文字 01 01 Something failed… Wish my code passed 02 02 03 03Packaging Need 04 change! Hey, test PASS! Peer review PASS! Newbie 05 Old BirdIt looks good to me, Good, Approve!But need someone approve…
  30. 30. Project Management 目录 00 00 写上你的文字你的文字 01 01 02 02 03 03 04 05Deploy open-source version Launchpad in-house as project management system.
  31. 31. Content SinaCloud Introduction Challenges to build a OpenStack Public Cloud 1. Network topology 2. Security Enhancement 3. Storage Solution 4. Identity Integration 5. Billing & Monitoring 6. Dashboard Improvement Operate an production OpenStack 7. Platform stack 8. Automated Deployment 9. Continuous Integration 10. Project Management StackLab: A community OpenStack Public Cloud Summary
  32. 32. StackLab.org A Community free OpenStack Public Cloud, more than just a OpenStack sandbox.StackLab is initiated and operated by Sina OpenStack team, as well astech volunteers from community, while resources sponsored by Sina, Intel. StackLab news report: http://freedomhui.com/2012/10/coscl-launches-stacklab/
  33. 33. Why StackLabNot everyone has the opportunity to run a OpenStack publiccloud when no resources, no users, no market, but StackLabwill change this.
  34. 34. StackLab GoalsA community OpenStack public cloud which benefits users,contributors and sponsors.For OpenStack Users who experience StackLab Understand what exactly OpenStack is and what does it provide Develop application on StackLab or using OpenStack API Build faith on OpenStack, possibly become real adopters and supporterFor OpenStack contributors involved in StackLab Testing patches on real production-like environmental, and get feedback from users, thus facilitate development and QA processes Gains experiences through operating StackLab without risk of SLA Better understand the requirement of OpenStack usersFor StackLab sponsors Build band acknowledgement in OpenStack community Own one StackLab region in their own data center Prior access to free technical support, consultant, of StackLab DevOps team
  35. 35. How to join StackLabFor OpenStack Users who want to experience StackLabReally Easy! Goto StackLab.org, register a free account instantly withoutapprovement by admin.For OpenStack contributors to join StackLab DevOps team 1. Persuade your company to become a StackLab sponsor, thus you will have a StackLab region in your own DC, and you are one admin of StackLab. 2. Contact us to join as an individual member. For StackLab sponsors 1. Send a email to openstacklab@gmail.com, expressing your willing to join StackLab 2. Sign a agreement with StackLab DevOps team 3. StackLab team help you build one StackLab region in your own DC. The minimum requirement is 3 servers with access to public network. 4. List your StackLab region in Stacklab.org portal.
  36. 36. How to join StackLab(cont.)StackLab: http://stacklab.orgStackLab Discussion Group:https://groups.google.com/group/stacklabstacklab@googlegroups.comStackLab DevOps Team:https://groups.google.com/group/stacklab-devopsstacklab-devops@googlegroups.com
  37. 37. What does StackLab Look Like?Choose the region before login Or choose the region after login
  38. 38. Content SinaCloud Introduction Challenges to build a OpenStack Public Cloud 1. Network topology 2. Security Enhancement 3. Storage Solution 4. Identity Integration 5. Billing & Monitoring 6. Dashboard Improvement Operate an production OpenStack 7. Platform stack 8. Automated Deployment 9. Continuous Integration 10. Project Management StackLab: A community OpenStack Public Cloud Summary
  39. 39. Summary 目录 00 00 写上你的文字你的文字 OpenStack definitely the best open-source cloud 01 01 platform for building public cloud 02  Open, open, open, open  02 Fast growing ecosystem around OpenStack 03 No vendor lock-in  03  Etc. 04 OpenStack Public cloud needs much more 05 customization and development to differentiate. Require strong technical skills and involvement in community.
  40. 40. 目录 00 00 写上你的文字你的文字 01 01 02Thank you, OpenStack Community 03 02and Foundation. 03 04 05
  41. 41. 目录00 00 写上你的文字你的文字01 01 Q&A02 0203 03 04 05 freedomhui@gmail.com Freedomhui.com

×