• Like
SUSEcon Crowbar Presentation
Upcoming SlideShare
Loading in...5
×

Thanks for flagging this SlideShare!

Oops! An error has occurred.

SUSEcon Crowbar Presentation

  • 1,583 views
Published

Presentation given by Greg Althaus & Rob Hirschfeld @ SUSEcon 2012 about Dell Crowbar, DevOps and Cloud Deployment.

Presentation given by Greg Althaus & Rob Hirschfeld @ SUSEcon 2012 about Dell Crowbar, DevOps and Cloud Deployment.

Published in Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
1,583
On SlideShare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
44
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Crowbar:The Cloud OperationsFrameworkGreg AlthausDell Principal Engineer and Crowbar Co-Founder
  • 2. Agenda• What Led to Crowbar?• What is Crowbar?• Crowbar Architecture• SUSE + Crowbar = A Path to Clouds• Crowbar 2.0 and Beyond• Q&A2 Revolutionary Cloud & Big Data Solutions
  • 3. What Led to Crowbar?3 Revolutionary Cloud & Big Data Solutions
  • 4. Crowbar <3 OpenStack“Boxes to working OpenStack in <4 hours”• Crowbar’s birthright was “an OpenStack installer” – But we started it knowing there was a much bigger problem• Like many open source projects, OpenStack is complex and changes constantly – Want the innovation and afraid of change• To create an OpenStack project, needed: – To get customers running quickly – Have a way to cope with frequent updates – Connect a lot of parts together – Be part of the open source community• Got & get substantial uplift from the OpenStack community4 Confidential Revolutionary Cloud & Big Data Solutions
  • 5. Problem: Clouds do not operate themselves! Operations SW Ops Servers HW Networks Storage Etc. Expertise5 Confidential Revolutionary Cloud & Big Data Solutions
  • 6. Ops automation (“devops”) jumpstarts learning,reduces complexity and manages scale Operations Automation (DevOps) SW Servers HW Networks Storage Ops Etc. Operations Expertise6 Confidential Revolutionary Cloud & Big Data Solutions
  • 7. What is DevOps? Why is it important?• DevOps extended to infrastructure is “CloudOps”• Operational Model for Software – “Operations as Code” – Highly automated – Layered deployments (no image based) – Continuous deployment• Shaped by Amazon Cloud – API only access to infrastructure – Elastic resources that are easier to create and destroy than change• Configuration Management Tooling – Chef & Puppet – Builds applications from open source modules/recipes – Not quite coding, but far from manual7 Confidential Revolutionary Cloud & Big Data Solutions
  • 8. OpenStack has a lot of interlocking parts!8 Confidential Revolutionary Cloud & Big Data Solutions
  • 9. …and it matters how they are deployed! Cookbooks9 Confidential Revolutionary Cloud & Big Data Solutions
  • 10. Images vs. Layers: Lifecycle Images: Replacement Layers: Upgrade Config Config Config I I Foo Foo Config Config I+A+U+ I+A+U+ I+A+U+ Bar v1 Bar v2 O/S O/S O/S U U OS OS Config Bar v2 I+A+U+ O/S10 Confidential Revolutionary Cloud & Big Data Solutions
  • 11. CloudOps for OpenStack There are lot of pieces, but they share many elements! I I I I I I Crowbar Dash Nova Nova Nova Config Config Config Config Config Config Nagios Chef Board Cntrl DB API U U U U U U OS OS OS OS OS OS I I I I I I Nova Nova Nova Nova Nova Nova Config Config Config Config Config Config Node Node Node Node Node Node U U U U U U OS OS OS OS OS OS11 Confidential Revolutionary Cloud & Big Data Solutions
  • 12. CloudOps for OpenStack Plus interconnections are highly complex and dynamic I I I I I I Crowbar Dash Nova Nova Nova Config Config Config Config Config Config Nagios Chef Board Cntrl DB API U U U U U U OS OS OS OS OS OS I I I I I I Nova Nova Nova Nova Nova Nova Config Config Config Config Config Config Node Node Node Node Node Node U U U U U U OS OS OS OS OS OS12 Confidential Revolutionary Cloud & Big Data Solutions
  • 13. What is Crowbar?13 Revolutionary Cloud & Big Data Solutions
  • 14. Crowbar, the Cloud Deployer++• What is Crowbar – Not only reduces the effort for deploying clouds – Also provides an operational model for managing them• Field proven technologies – Build on locally deployed Chef Server – Raw servers to full cloud in <2 hours, multi-node from the ground up – Hardened with over a year of deployments• Apache 2 open source (on github/dellcloudedge) – Multi-apps (OpenStack & Hadoop), multi-OS (Ubuntu, RHEL, CentOS, SUSE), Development “in the open” w/ community engagement• Hadoop & OpenStack releases (dell.com/OpenStack) – Cactus, Diablo and Essex!• Open version is not Dell hardware Limited – This is been an important factor in adoption14 Confidential Revolutionary Cloud & Big Data Solutions
  • 15. Crowbar in the Open (Community!)• Community – Focused on OpenStack, expanding in other areas – Engineering benefits from community test and dev – Dell engineering active participants• Developed in the Open – Most code development is current & live on github – Needed for collaboration – Allows faster response to customers and partners• Active Community – 300+ email list users & nearly 500 Github followers – 500+ downloads of the Crowbar image – Partners: Intel, Opscode, SUSE, etc – Customers: telecom, cloud, financial, etc15 Confidential Revolutionary Cloud & Big Data Solutions
  • 16. Cloud Taxonomy Software as a Service IT as a Service Platform as a Service Infrastructure as a Service Everything as a Service Admin Software Web Services & APIs Self Service Portal SSO Customer Management Entitlement, rights Billing Metering Information Service Management LDAP/AD Infrastructure Software Workload Lifecycle Legacy Management Ser Gov/Workflow Automation Monitoring Management Orchestration Intelligent Resource Manager Platform Provisioning ReportingAnalytics Security Abstraction Software OS Data Store Operating System Application Run-Time Hardware Virtualization Virtualization Virtualization IPS Physical Compute Switch Storage Firewall Network HVAC Power Facility Overarching Environmentals Revolutionary Cloud & Big Data Solutions Systems
  • 17. Crowbar Builds the Cloud Taxonomy Software as a Service IT as a Service Platform as a Service Infrastructure as a Service Everything as a Service Admin Software OpenStack APIs OpenStack Dashboard SSO Customer Management Entitlement, rights Billing Metering Information Service Management LDAP/AD Infrastructure Software Workload Lifecycle Legacy Management Ser Gov/Workflow Automation Monitoring Management Orchestration Intelligent Resource Manager Crowbar ReportingAnalytics Keystone Abstraction Software OS Data Store Operating System Application Run-Time HardwareKVM Virtualization Virtualization Virtualization IPS Physical Nova Switch Glance Swift Firewall Quantum BIOS RAID NETWORK Overarching Environmentals Revolutionary Cloud & Big Data Solutions Systems
  • 18. Crowbar Builds Up Clouds in layers APIs, User Access, & Ecosystem Partners Ops Management Dell “Crowbar” Cloud Infrastructure Core Components & Operating Systems Physical Resources18 Confidential Revolutionary Cloud & Big Data Solutions
  • 19. 1) Hardware Configuration APIs, User Access, & Ecosystem Partners Ops Management Dell “Crowbar” Cloud Infrastructure Core Components & Operating Systems Physical Resources19 Confidential Revolutionary Cloud & Big Data Solutions
  • 20. 2) OS and Ops Infrastructure APIs, User Access, & Ecosystem Partners Ops Management Dell “Crowbar” Cloud Infrastructure Core Components & Operating Systems Physical Resources20 Confidential Revolutionary Cloud & Big Data Solutions
  • 21. 3) Cloud Platform Integration APIs, User Access, & Ecosystem Partners Ops Management Dell “Crowbar” Cloud Infrastructure Core Components & Operating Systems Physical Resources21 Confidential Revolutionary Cloud & Big Data Solutions
  • 22. 4) Cloud App Management APIs, User Access, & Ecosystem Partners Ops Management Dell “Crowbar” Cloud Infrastructure Core Components & Operating Systems Physical Resources22 Confidential Revolutionary Cloud & Big Data Solutions
  • 23. Late Binding Matters! It’s “Just-in-time” DeploymentLate binding is a critical component of Crowbar flexiblity • Integrated configuration of RAID, BIOS, Allocate with System View provision & networking, so no longer have to do that work in advance. • Do NOT need to make these decisions early in the process Inventory Provision • Can adapt and change over time. Discover Deploy Layer N User App & Ops decisions drive the entire Network Config config & deploy process!23 Confidential Revolutionary Cloud & Big Data Solutions
  • 24. Crowbar Architecture + Demo24 Revolutionary Cloud & Big Data Solutions
  • 25. OpenStack Orchestration: Chef & Crowbar Orchestration State Machine OpenStack Components Crowbar Magic Happens! Ops Management Dell “Crowbar” Run List Network Configuration Chef Server Operating System Config. Hardware Config (uses “Sledgehammer”) DHCP Server(s)25 Confidential Revolutionary Cloud & Big Data Solutions
  • 26. Dell Cloud Installer: Crowbar Barclamps APIs, User Access, & Nagios Ganglia Dashboard Ecosystem Partners enStratus Cloudera Horizon Ops Management Nova Swift Glance Dell “Crowbar” Cloud Infrastructure & Dell IP Extensions Block Hadoop Keystone Core Components & Crowbar DNS Logging Operating Systems Deployer NTP Provisioner BIOS IPMI Physical Resources Network RAID * Crowbar has potential to service other programs beyond OpenStack26 Confidential Revolutionary Cloud & Big Data Solutions
  • 27. What is a Barclamp? Barclamp Crowbar API O/S App Components Chef Recipes & Partial UI Packages & Scripts Ops Management • Crowbar API & Partial UI Dell “Crowbar” • Adds states and transitions to orchestration • Customizes UI specific to barclamp • Visual extensions to nodes/networks/utils • Chef Recipes • DevOps description of application • Enforced repeatable configuration • O/A App Packages • Operating system dependencies (Debs & RPMs) • Components & Scripts • Ad hoc configuration not covered by Chef27 Confidential Revolutionary Cloud & Big Data Solutions
  • 28. What is a Barclamp? Barclamp Crowbar API O/S App Components Chef Recipes & Partial UI Packages & Scripts Ops Management • Interact as service layers Dell “Crowbar” • Have dependencies and pre-requisites • Can operate on multiple operating systems • Have independent version streams • Leverage/reuse Chef cookbooks Barclamp Barclamp28 Confidential Revolutionary Cloud & Big Data Solutions
  • 29. Crowbar’s Proposal Workflow• Proposals manage deployments• Proposals are a core design objective to ensure customer flexibility and system level management• Workflow: • Templates are default configurations without resources • Proposals all users to choose config and nodes (resources) • Applying a proposal tells Crowbar to “make it so” Discovered Nodes Proposal Nodes Apply Active Barclamp Deployed Services Proposal Nodes Apply Active29 Confidential Revolutionary Cloud & Big Data Solutions
  • 30. Crowbar Orchestration = State Machine30 Confidential Revolutionary Cloud & Big Data Solutions
  • 31. Networking in Crowbar IP Allocations Network Barclamp DHCP Chef Client Crowbar IMPI Discovers VLAN Switches VLAN and Ports eth0 VLAN VLAN eth1 Crowbar Teamed Teamed Does NOT Configure eth# The Switch! not O/S Only* mgm’d Server * For now31 Confidential Revolutionary Cloud & Big Data Solutions
  • 32. Networking in Crowbar IP Allocations Network Barclamp DHCP Other Chef Client Barclamps IMPI Request VLAN Conduit “Conduits” VLAN 1g1 eth0 VLAN VLAN Conduit Interface & 1g2 eth1 Conduits Teamed Teamed Conduit Normalize 1g3 eth# Networks Between not Nodes mgm’d Server32 Confidential Revolutionary Cloud & Big Data Solutions
  • 33. SUSE + Crowbar = A Path to Clouds33 Revolutionary Cloud & Big Data Solutions
  • 34. SUSE Cloud with Crowbar• Version of Crowbar that runs on SUSE• Packaged to take advantage of SUSE repositories and methods• Provides OpenStack Essex• Extends and builds SUSE Studio• Released and available as of the beginning of September34 Confidential Revolutionary Cloud & Big Data Solutions
  • 35. Team Work • A huge key to the success has been team work • Joint development • Design and planning discussion • Not just limited to SUSE, but definite leader35 Confidential Revolutionary Cloud & Big Data Solutions
  • 36. Crowbar 2.0 and Beyond36 Confidential Revolutionary Cloud & Big Data Solutions
  • 37. Crowbar 2.0 (currently in progress) http://bit.ly/crowbar2• Simplify use of Chef, eliminate Crowbar requirements in Chef recipes – Reduce the initial effort required to leverage Crowbar – Opens Crowbar to a broader audience (see Upstreaming)• Provide heterogeneous/multiple operating system deployments to enable: – Multiple versions of the same OS running for upgrades – Different OS operating simultaneously (and deal with heterogeneous packaging issues) – Accommodation of no-agent systems like locked systems (e.g.: virtualization hosts) and switches (aka external entities) – UEFI booting in Sledgehammer• Strengthen networking abstractions – Allow networking configurations to be created dynamically (so that users are not locked into choices made before Crowbar deployment) – Better manage connected operations – Enable pull-from-source deployments that are ahead of (or forked from) available packages37 Confidential Revolutionary Cloud & Big Data Solutions
  • 38. Crowbar 2.0 (currently in progress) http://bit.ly/crowbar2• Improve Crowbars core database and state machine to enable – Larger scale concerns – Controlled production migrations and upgrades• Packaging rework – Lead by some of the SUSE work – Create distribution-based packages• Upgrade components – Start the process of enabling security updates – Staged Updates from test to production environments – Start considering application• Other important items – Make documentation more coupled to current features and easier to maintain – Upgrade to Rails 3 to simplify code base, security and performance – Deepen automated test coverage and capabilities38 Confidential Revolutionary Cloud & Big Data Solutions
  • 39. Crowbar 2.0 (currently in progress) http://crowbar.sync.in/crowbar2• Network models (http://youtu.be/eTSTLecNlGM ) - http://crowbar.sync.in/crowbar2-network• Online mode (http://youtu.be/U05apbOqq0k ) - http://crowbar.sync.in/crowbar2-online• Pull from source (http://youtu.be/Bo4V6Dax3LU ) - http://crowbar.sync.in/crowbar2-pullsource• Heterogeneous OS (http://youtu.be/xL-2eefqH8s) - http://crowbar.sync.in/crowbar2-multi-os• Attribute driven recipes (http://youtu.be/wtUNJfrHU1Q) - http://crowbar.sync.in/crowbar2-attributeconfig39 Confidential Revolutionary Cloud & Big Data Solutions
  • 40. Q&A40 Confidential Revolutionary Cloud & Big Data Solutions
  • 41. Thanks for Listening! • Greg Althaus Gregory_Althaus@dell.com Twitter: @galthaus41 Confidential Revolutionary Cloud & Big Data Solutions