Published on

This is the slide Will Chan used a

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Developer Day – 7/21/2012Will Chan – Director of Engineering
  2. 2. Agenda• Welcome and Introduction• What is CloudStack? • CloudStack Basics • Cloudstack Deployment Architecture • Networking Deep Dive • Software Architecture • Current Architecture • Future Architecture • Q&A• Lunch
  3. 3. Agenda (cont.)• CloudStack Integration • UI Customization • API Deep Dive • Future UI Plugin Framework• Q&A/Break• Apache Community • Why Apache and the Apache Server Foundation (ASF)? • How to contribute to CloudStack• Closing Remarks
  4. 4. Welcome and Introduction
  5. 5. Welcome and Introduction• Will Chan • Founding Engineer/Director of Engineering @ since 2008 • Director of Engineering @ Citrix Systems since 2011 • PPMC member @ ASF CloudStack since 2012 • Committer @ ASF CloudStack since 2012• Sheng Liang • Cloud Visionary and Founder of • CTO, CloudPlatforms Group at Citrix Systems
  6. 6. What is CloudStack?
  7. 7. Apache CloudStack • Secure, multi-tenant cloud orchestration platform – Turnkey platform for delivering IaaS clouds – Over 100 commercial deployments: private and public Build your cloud the way the world’s – Full featured GUI, end-user API and admin API most successful clouds are built
  8. 8. Apache CloudStack • Open Source • Apache License • Incubating in the Apache Software Foundation since April 2012 • Open Source since May 2010 Build your cloud the way the world’s most successful clouds are built • In production since 2009
  9. 9. Apache CloudStack • Flexibility and scale • Hypervisor agnostic • Flexible network topologies • Multiple storage options • Proven to scale to tens of thousands of Build your cloud the way the world’s most successful clouds are built hypervisors
  10. 10. CloudStack is Full-Service Orchestration Platform API (EC2 & CS) Self-service Portal Metering Image Mgmt. Dashboard Identity Mgmt. Load FWs & VPNs Balancers Storage Compute Network
  11. 11. 146 Companies 238 Developers Global User Groups Service Providers 100’s of Production Clouds 32,000 Community Members Enterprises Universities
  12. 12. Server Virtualization++ CloudBuilt for traditional enterprise apps Designed around big data, massive& client-server compute scale & next-gen apps• Enterprise arch for 100s of hosts • Cloud architecture for 1000s of hosts• Scale-up (server clusters) • Scale-out (multi-site server farms)• Apps assume reliability • Apps assume failure• VLAN (or no) isolation • L3 isolation or overlays• Bonding, multi-link, multi-path, • Generally do not support multicast or redundant networks, STP broadcast• Proprietary vendor stack • Open, value-added stack Think: vCloud Director Think: AWS, RAX, zCloud, eBay, etc.
  13. 13. CloudStack Supports Multiple Cloud Strategies Private Clouds Public Clouds On-premise Hosted Multi-tenant Enterprise Cloud Enterprise Cloud Public Cloud • Dedicated resources • Dedicated resources • Mix of shared and • Security & total control • Security dedicated resources • Internal network • SLA bound • Elastic scaling • Managed by • 3rd party owned and • Pay as you go Enterprise or 3rd party operated • Public internet, VPN access
  14. 14. Designing a zone for a traditional workload Hypervisor Traditional-Style Availability Zone vSphere or XenServer Enterprise vCenter/XenCenter Storage Enterprise Networking (e.g., VLAN) SAN Networking Hyperviso Hyperviso Hyperviso L2 VLANs r Cluster r Cluster r Cluster Network Services Enterprise Storage (e.g., SAN) Load Balancing VPN Multi-tier Apps Multi-tier VLANs OVF
  15. 15. Designing a zone for an Amazon-style workload Amazon-Style Availability Zone Software Defined Networks Hypervisor (e.g., Security Groups, EIP, ELB,...) XenServer Advanced Server Server Server Server Storage Racks Racks Racks Racks Local EBS Object store Server Server Server Server Racks Racks Racks Racks Networking L3 SDN based L2 Elastic IP Server Server Server Server Racks Racks Racks Racks Network Services Security Groups ELB GSLB Elastic Block Storage Multi-tier Apps 3rd Party Tools (e.g., RightScale, enStratus) CloudFormation
  16. 16. Deployment CloudArchitecture
  17. 17. Deployment ArchitectureZone 1  Host is the basic unit of scale. Runs a hypervisor or is bare metal Load Balancer Firewall  Cluster consists of one ore more hosts of same hypervisor L3 switch  All hosts in cluster have access to shared (primary) storage Pod 1 L2 switch Pod N Secondary  Pod is one or more clusters, usually …. Storage with a L2 switch. Represents a rack Cluster N  Availability Zone has one or more pods, has access to secondary …. storage.  Firewall and Load balancers separate Cluster 1 public and private networks Host 1 Primary  One or more zones represent cloud Storage Host 2
  18. 18. Deployment Architecture (Storage) Primary Storage• Configured at Cluster-level. Close to hosts for better performance• Stores all disk volumes for VMs in a cluster L3 switch• Cluster can have one or more primary storages Pod 1 L2 switch• Local disk, iSCSI, FC or NFS Secondary Cluster 1 Storage Host 1 Primary Secondary Storage Storage Host 2• Configured at Zone-level• Stores all Templates, ISOs and Snapshots• Zone can have one or more secondary storages• NFS, OpenStack Swift
  19. 19. Deployment Architecture Data Center 1 Data Center 3 Zone1 Zone 4 CloudStack Clouds can have one or more Availability Zones. Data Center 2 Zone 2 Zone 3
  20. 20. Management Server Managing Multiple Zones Cloud Data Center 1 Data Center 2  Single Management Server can manage Data Center 2 Management Data Center 3 multiple zones Server Zone 2  Zones can be geographically distributed but Zone 2 low latency links are expected for better Zone 3 performance Zone1 Zone 4 3 Zone  Single MS node can manage up to 10K hosts.  Multiple MS nodes can be deployed as cluster for scale or redundancy Data Center 2 Data Center 2 Data Center 2 Zone 2 Zone 2 Zone 2Zone 3 Zone 3 Zone 3
  21. 21. Management Server Deployment Architecture Single-node Deployment Multi-node Deployment Management User API User API Server Management Load MySQL Server DB BalancerAdmin API Admin API Management Server MySQL DB Back Up Replication DB  MS is stateless. MS can be deployed as physical server or VM  Single MS node can manage up to 5K hosts. Infrastructure Infrastructure Multiple nodes can be deployed for scale or Resources Resources redundancy  Commercial: RHEL 5.4+; FOSS: Ubuntu 10.0.4, Fedora 16 Citrix Confidential - Do Not Distribute
  22. 22. Management Server Interaction with Hypervisors Management Server XAPI HTTP vCenter Agent Agent XenServer KVM OVM ESX• XS 5.6, 5.6FP1, 5.6 SP2, 6.0 • ESX 4.1, 5.0 (coming) • RHEL 6.0, 6.1, 6.2 (coming) • OVM 2.2• Incremental Snapshots • Full Snapshots • Full Snapshots (not live) • No Snapshots• VHD • VMDK • QCOW2 • RAW• NFS, iSCSI, FC & Local disk • NFS, iSCSI, FC & Local disk • NFS, iSCSI & FC • NFS & iSCSi• Storage over-provisioning: NFS • Storage over-provisioning: • Storage over-provisioning: NFS • No storage over-provisioning NFS, iSCSI
  23. 23. Networking Deep Dive
  24. 24. Network Flexibility Network Services Service Providers Network Isolation • L2 connectivity  Virtual • No isolation • IPAM appliances • DNS • VLAN isolation  Hardware • Routing firewalls • Overlays • ACL  LB appliances • Firewall • L3 isolation • NAT  SDN controllers • VPN  IDS /IPS • LB appliances • IDS  VRF • IPS  Hypervisor
  25. 25. Layer-3 Guest Network Network Services Managed Externally Network Services Managed by CS Public Network Security Group 1 Security Group 1 Public Network/Internet Guest VM 1 Guest VM 1 NetScaler Guest VM 2 Guest VM 2 Load Blancer EIP, ELB Guest VM 3 Guest VM 3 Guest VM 4 Guest VM 4 CS CS DHCP, Virtual Security Group 2 DHCP, Virtual Security Group 2 Router Router DNS DNS
  26. 26. Layer-2 Guest Virtual Network CS Virtual Router provides Network Services External Devices provide Network Services Guest Virtual Network Guest Virtual Network VLAN 100 VLAN 100Public PublicNetwork/Internet Network/Internet Guest VM 1 Public IP Private IP Guest VM 1 Juniper SRX Public IP Gateway address 1 Firewall CS Guest VM 2 Guest VM 2 Virtual Router Public IP Private IP DHCP, DNS NAT NetScaler Guest VM 3 2 Load Blancer Guest VM 3 Load Balancing VPN Guest VM 4 Guest VM 4 CS DHCP, Virtual Router DNS
  27. 27. Network Offerings• Same concept with disk and service offerings• What can you control? • Name • Enable Redundant Router • Control Network Rate • Specify Network Services (Firewall, Loadbalancer, etc…) • Specify Network Provider (VR, SRX, Netscaler, F5, etc…) • Specify access (All, Domain, Account)• Allow upgrade and downgrade across offerings.
  28. 28. Multi-tier virtual networking Guest Virtual Network Guest Virtual Network Guest Virtual Network VLAN 100 VLAN 101 VLAN 102PublicNetwork/Internet Web VM 1 App VM 1 DB VM 1 Gateway address Public IP Gateway address CS Gateway address Virtual Router DHCP, DNS NAT Web VM 3 App VM 2 DB VM 2 Load Balancing VPN Web VM 4 App VM 3
  29. 29. Current SoftwareArchitecture
  30. 30. UI Cloud Portal CLI Other Clients Management Server REST API OAM&P API End User API EC2 API Other APIs Pluggable Service API Engine Console Proxy ACL & Authentication Security Adapters Management - Accounts, Domains, and Projects - ACL, limits checking Account Management Connectors Services APITemplate Access DB Plugin API Deployment Planning HA Orchestration Engine Services API - Drives long running VM operations Network Gurus - Syncs between resources managed and DBUsage Calculations - Generates events Network ElementsAdditional Services Hypervisor Gurus Cluster Resource Job Alert & Event Database Management Management Management Management Access Event Bus Message Bus Usage Server Resource API Hypervisor Network Storage Image Snapshot Resources Resources Resources Resources Resources
  31. 31. Orchestration Engine• Understands how to orchestrate long running processes (i.e. VM starts, Snapshot copies, Template propagation)• Well defined process steps• Calls Plugin API to execute functionalities that it needs
  32. 32. Plugins• Various ways to add more capability to CloudStack• Implements clearly defined interfaces• All calls are at transaction boundaries• Compiles only against the Plugin API module
  33. 33. Anatomy of a Plugin • Can be two jars: server component to be deployed on management server and an optional ServerResource Rest API - Optional. Required only if needs to expose component to be deployed co-located configuration API to admin. with the resource • Server component can implement Plugin API Implementation multiple Plugin APIs to add its feature Data Access Layer • Can expose its own API through Pluggable Service so administrators can configure the plugin - ServerResource Optional. Required if Plugin needs to be co-located with • As an example, OVS plugin actually - the resource Implements translation layer to talk to resource implements both NetworkGuru and - Communicates with server component via JSON NetworkElement
  34. 34. Plugin Interfaces Available• NetworkGuru – Implements various network isolation and ip address technologies• NetworkElement – Facilitate network services on network elements to support a VM (i.e. DNS, DHCP, LB, VPN, Port Forwarding, etc)• DeploymentPlanner – Different algorithms to place a VM and volumes.• Investigator – Ways to find out if a host is down or VM is down.• Fencer – Ways to fence off a VM if the state is unknown• UserAuthenticator – Methods of authenticating a user• SecurityChecker – ACL access• HostAllocator – Provides different ways to allocate host• StoragePoolAllocator – Provides different ways to allocate volumes
  35. 35. Future CloudStackArchitecture (by Sheng)
  36. 36. CloudStack Integration
  37. 37. What you will learn• How to customize the CloudStack 3.0.x user interface • Showcase changes specific in the CSS to alter the look and feel of CloudStack • Showcase an example of how to add your own side navigation • Dealing with Cross Site Request Forgery (CSRF) • Simple Single Signon • Localization
  38. 38. What you will learn• Working with the API • Session Based Auth vs API Key Auth • How to sign a request with apiKey/secretKey • Asynchronous commands • Response Format • Pagination
  39. 39. Demo
  40. 40. Customizing User Interface
  41. 41. CloudStack UI• Reference implementation of the CloudStack API• Built on HTML 4.0, CSS, and jQuery• Uses Java Server Pages for localization only• Three types of customizations • Minor customizations – logo changes, minor CSS changes • Major customizations – Changing tabs, adding additional links • Complete rewrite – user UI is completely offloaded to a portal
  42. 42. Editing the Logo, Navigation, and Title Background#header div.logo { background: url("../images/logo.png") no-repeat scroll 0 center transparent; float: left; height: 47px; #navigation ul li { .dashboard.admin margin: 4px 0 0 19px; background: url("../images/bg-nav- .dashboard-container .top { position: relative; item.png") background: url("../images/ width: 170px; repeat-x scroll 0 0 bg-breadcrumb.png")} transparent; repeat-x scroll ¥0 -1px cursor: pointer; transparent; height: 50px; border-radius: 7px 7px 0 0; text-shadow: 0 1px 1px #FFFFFF; color: #FFFFFF; } float: left; margin: 0 0 9px; padding: 4px 4px 8px; width: 100%; }
  43. 43. Adding navigation buttons 1. Go to /ui/scripts/cloudStack.js 2. Add a new section to the array: sections: { dashboard: {}, instances: {}, storage: {}, network: {}, templates: {}, events: {}, accounts: {}, domains: {}, system: {}, projects: {}, global-settings: {}, configuration: {}, // New section testSection: {} }
  44. 44. Adding navigation buttons (cont.) 3. Open /ui/index.jsp. Create HTML 4. Enclose a function in somewhere in the template div to testSection, which returns a contain your HTML content, which will jQuery object be drawn in the browser pane: containing your template code, and whatever other content you <!-- Templates --> wish to <div id="template"> be shown: <div class="testSection-tmpl"> sections: { <h1>Test section</h1> dashboard: {}, instances: {}, </div> storage: {}, </div> network: {}, templates: {}, events: {}, accounts: {}, domains: {}, system: {}, projects: {}, global-settings: {}, configuration: {}, // New section testSection: { title: Title for section, show: function(args) { return $(#template .testSection- tmpl).clone(); } } }
  45. 45. Adding navigation buttons (cont.) 5. Add the section to the pre-filter, so that it isnt filtered out for the admin account: -- sectionPreFilter: function(args) { if(isAdmin()) { return ["dashboard", "instances", "storage", "network", "templates", "accounts", "domains", "events", "system", "global-settings", "configuration", "projects"]; }, sectionPreFilter: function(args) { if(isAdmin()) { return ["dashboard", "instances", "storage", "network", "templates", "accounts", "domains", "events", "system", "global-settings", "configuration", "projects", // New section "testSection"]; }, ...
  46. 46. Adding navigation buttons (cont.) 7. (optional) Add an icon for your new section in the CSS, either at the bottom of /ui/css/cloudstack3.css or in your own CSS file under /ui/css folder. Make sure the size of the icon is ~32x32 pixels: #navigation ul li.testSection span.icon { background: url(../images/testSection-icon.png) no-repeat 0px 0px; }
  47. 47. Cross Site Request Forgery (CSRF)• Type of malicious exploit of a website whereby unauthorized commands are transmitted from a user that the website trusts. Unlike cross-site scripting (XSS), which exploits the trust a user has for a particular site, CSRF exploits the trust that a site has in a users browse• What does CS do to prevent this? • After execution of the login command you will get two session variables • JSESSIONID – default cookie • SESSIONKEY – random token that is passed along every API request • http://<API URL>?sessionkey=<SESSIONKEY>&…
  48. 48. Simple Single Signon•http://<api_url>?command=login&username=XXX&domainid=NNN&timestamp=YYY&signature=<secure-hash>• You do not need to pass in the API Key• The four parameters that must be passed in for the login command are domainId, username, timestamp, and signature• security.singlesignon.key• security.singlesignon.tolerance.millis• SAML?
  49. 49. Localization • Support for Japanese and Simplified Chinese • Takes advantage of the Java ResourceBundle to do localization • Simply create a /WEB-INF/classes/resources/messages_<language code>.properties • Server side vs Client side processing
  50. 50. Working with the API
  51. 51. Session-based Auth vs API Key Auth• CloudStack supports two ways of authenticating via the API.• Session-based Auth • Uses default Java Servlet cookie based sessions • Use the “login” API to get a JSESSIONID cookie and a SESSIONKEY token • All API commands require both cookie and token to authenticate • Has a timeout as configured within Tomcat• API Key Auth • Works similarly to AWS API • Requires a bit more coding to generate the signature • All API commands require a signature hash
  52. 52. SIGNING REQUEST WITH API KEY / SECRET KEYStep 1:commandString = command name + parameters + api keyURL encode each field-value pair within the commandstringStep 2:Lower case the entire commandString and sort it alphabetically via the field for each field-value pair.sortedCommandString :apiKey=vmwijj…&command=createvolume&diskofferingid=1&name=smallvolume=zoneid=1Step 3:Take the sortedCommandString and run it through the HMAC SHA-1 hashing algorithm (mostprogramming languages offer a utility method to do this) with the user’s Secret Key. Base64 encodethe resulting byte array in UTF-8 so that it can be safely transmitted via HTTP. The final stringproduced after Base64 encoding should be SyjAz5bggPk08I1DE34lnH9x%2f4%3D
  53. 53. Asynchronous Commands• Starting with 3.0, in your standard CRUD (Create, Read, Update, Delete) of any first class objects in CloudStack, CUD are automatically asynchronous. R is synchronous.• Rather than returning a response object, it will return a job ID.• If it is a “Create” command, it will also return the object ID.• With the job ID, you can query the async job status via the queryAsyncJobResult command.• The queryAsyncJobResult response will return the following possible job status code: • 0 - Job is still in progress. Continue to periodically poll for any status changes. • 1 - Job has successfully completed. The job will return any successful response values associated with command that was originally executed. • 2 - Job has failed to complete. Please check the <jobresultcode> tag for failure reason code and <jobresult> for the failure reason.
  54. 54. Response Formats• CloudStack supports two formats as the response to an API call.• The default response is XML. If you would like the response to be in JSON, add &response=json to the Command String.
  55. 55. Response Formats (cont.)Sample XML Response:<listipaddressesresponse> <allocatedipaddress> <ipaddress></ipaddress> <allocated>2009-09-18T13:16:10-0700</allocated> <zoneid>4</zoneid> <zonename>WC</zonename> <issourcenat>true</issourcenat></allocatedipaddress> </listipaddressesresponse>Sample JSON Response:{ "listipaddressesresponse" : { "allocatedipaddress" : [ { "ipaddress" : "", "allocated" :"2009-09-18T13:16:10-0700", "zoneid" : "4", "zonename" : "WC", "issourcenat" : "true" } ]
  56. 56. Pagination• Using the page and pagesize parameter • page defines the current cursor to the list • pagesize defines the number of items per request • Pagesize is limited by the administrator • Sample: • listVirtualMachines&page=1&pagesize=500 • listVirtualMachines&page=2&pagesize=500
  57. 57. UI Plugin Framework• Problems today? • Any major customizations require modification of CloudStack UI code. • Modifications require deep knowledge of CloudStack UI code. • Versioning becomes difficult.• Future Plugin Framework • Creating UI widgets that are re-usable • A JS configuration file that will allow partners/developers to specify how to include their UI into the CloudStack UI without having to modify core CloudStack UI code. • Example could be left navigation link or possibly new actions. These are TBD.
  58. 58. Apache Community
  59. 59. Why Apache License? • Ecosystem above • Vendors within • User adoption
  60. 60. Why Apache Software Foundation? • Best governance • 15+ years, 100+ projects • 2500+ Developers
  61. 61. The Road to an Apache “Top Level Project”• April: convert source code to Apache License• April: announce intent to donate• April: proposal for donation; get accepted to Incubator• May: donation, mailing lists, enter Incubation• Sept : Apache CloudStack 4.0 release• 2012: work in the “Apache Way”• Graduate to Top Level Project, contingent on: • Community involvement • Follow legal requirements and Apache standards
  62. 62. Implications for Partners and Customers• CloudStack awareness increased• CloudStack on path to be #1 orchestration software• Apache license provides more options for enhancements• More direct influence possible• Better visibility into CloudStack development
  63. 63. Citrix CloudPlatform• Citrix released CloudPlatform 3.0.3 mid June.• Citrix plans to contribute 100% of development back into CloudStack• Monetization remains the same before and after Apache.• We expect Apache CloudStack to be 3 months ahead of CloudPlatform• Citrix CloudPlatform will have a release schedule separate from CloudStack and will be determined by business needs.
  64. 64. How to Contribute
  65. 65. Apache Roles• User • A user is someone that uses our software. They contribute to the Apache projects by providing feedback to developers in the form of bug reports and feature suggestions. Users participate in the Apache community by helping other users on mailing lists and user support forums.• Developer • A developer is a user who contributes to a project in the form of code or documentation. They take extra steps to participate in a project, are active on the developer mailing list, participate in discussions, provide patches, documentation, suggestions, and criticism. Developers are also known ascontributors .
  66. 66. Apache Roles (cont.)• Committer • A committer is a developer that was given write access to the code repository and has a signed Contributor License Agreement (CLA) on file. They have an mail address. Not needing to depend on other people for the patches, they are actually making short-term decisions for the projectDeveloper• PMC Member • A PMC member is a developer or a committer that was elected due to merit for the evolution of the project and demonstration of commitment. They have write access to the code repository, an mail address, the right to vote for the community-related decisions and the right to propose an active user for committership. The PMC as a whole is the entity that controls the project, nobody else.
  67. 67. Development Environment• Development Machine • Apache Tomcat, version 6.0.33. Set environment variable CATALINA_HOME to point to your apache install directory. • Mysql, version 5.1.58 • Git, the latest version • Java, the latest version • Ant, the latest version
  68. 68. Development Environment (cont.)• To setup a Windows environment: • +on+Windows• To setup a Mac OS environment: • environment+on+Mac+OSX
  69. 69. Development Environment (cont.)• To get the CloudStack source code • git clone • git checkout master• To build CloudStack • ant clean-all build-all deploy-server deploydb• To start the Management Server • ant debug
  70. 70. Demo
  71. 71. CloudStack Developer Mailing List• This is where all CloudStack development discussion are mostly held.• All new features should be discussed on this mailing list.• If you want to contribute to CloudStack, you are highly encouraged to subscribe to the cloudstack-dev list if you haven’t done so. • To subscribe, email to • You can also subscribe to the users list (cloudstack-users- • And to the commit list (
  72. 72. How to Contribute• Clone ASF cloudstack repo: • git clone• Checkout master branch: • git checkout master• Write code, make sure its properly unit-tested. Unit-tests have to be submitted as a part of the patch• Create the patch for review: • git format-patch -o <dir of patch> --signoff master^
  73. 73. How to Contribute (cont.)• Create Jira ticket (or use existing ticket) and attach the patch: •• Submit the patch for review on Reviewboard for repository "cloudstack-git": •
  74. 74. How to Contribute (cont.)• Post on developer mailing list for review. Either the patch will be directly merged into the master branch or a topic branch will be created if it’s a large feature.• If you contribute a lot of good patches to CloudStack, a PMC member may decide to initiate a vote on your behalf to become a full-time committer.
  75. 75. Resources• CloudStack docs and knowledge base: • •• CloudStack architecture review: •• CloudStack packages and dependencies: • s
  76. 76. Resources (Cont.)• Exceptions handling in CloudStack: • +handling• DB upgrade development for CloudStack: •• Git workflow and coding standards in CloudStack: • itworkflowinthebravenewworld-Creatingpatches
  77. 77. devCloud• What is devCloud? • DevCloud is a VirtualBox image, on which CloudStack management server + Xen hypervisor are installed. CloudStack management server is running on Ubuntu 12.04 dom0, can also add dom0 itself as a Xen hypervisor host and create Linux virtual machines on it. • As a developer, you can push your modified CloudStack code into DevCloud, then deploy and run the CloudStack management server in DevCloud. • As an user, you can access CloudStack management server running inside DevCloud through web UI, large part of functionality of CloudStack are supported in DevCloud, such as creating VM, taking snapshot, creating template, console proxy, etc. •