Your SlideShare is downloading. ×
0
Open Source Toolchainsto Manage CloudInfrastructureMark R. HinkleVP of CommunityCloud.comTwitter:@clouddotcom@mrhinkleEmai...
2Introduction•  Responsible for Driving Adoption of   CloudStack Open Source Cloud Computing   Software•  Former manager o...
3Agenda•  Introductions•  Cloud & Systems Management Overview•  Open Source Tools for Cloud   Management•  Creating Open S...
Cloud Computing &Systems Management Overview
5Jevon’s Paradox and the Cloud…the efficiency with which a resource is used tendsto increase (rather than decrease) the ra...
6Balancing the Equation:Creating the Cloud         +           =
7Balancing the Equation:Managing the Cloud         + ? =
Automation Balances the            8Equation•  MeatCloud, Can’t Keep up with   Cloud Computing•  Devops & Agile IT Philoso...
DevOps – Agile Management for        9the Cloud•  Agile, fast moving•  Smaller incremental change•  Increased coordination...
10Management ToolchainsToolchain (n):                         MonitoringA set of tools where    Patching and              ...
11The Myth of the NinesAvailability %             Downtime per                 Downtime per                   Downtime per...
Open Source Tools toManage the Cloud
132011 CLOUD COMPUTING OUTLOOK SURVEYhttp://www.cloud.com/cloud-computing-outlook
144 Types of Management Tools                  Provisioning                  Installation of operating systems and other s...
15Cloudy Tools•    Network Capable•    Cloud “Aware”•    Easy-to-Integrate•    Adhere to Open Standards•    Lend Themselve...
Open Source                                                                    16Provisioning Tools                     Ye...
Open Source                                                        17Configuration Management Tools           Year Started ...
Open Source                                                                               18Monitoring Tools              ...
Open Source                                                         19Automation/Orchestration Tools              Year    ...
20Cloud APIs Aren’t Created EqualOpen Source AbstractionsAPI          Languagejclouds      javadeltacloud   pythonLib clou...
Open Source ManagementToolchains
22Toolchain Concept               Source:http://code.google.com/p/devops-toolchain/wiki/ArchitectureComponents
23Automated Toolchain                                                                       Capistrano                    ...
24Example: GeekNet•  Servers are automatically built using   con guration management software (Puppet)•  Discovery tool (Z...
Example Toolchain:     25Online Gaming Studio
Questions?
27Contact Me    Professional: mrhinkle@cloud.com    Personal: mrhinkle@gmail.com    Professional: mrhinkle@cloud.com    Pe...
Open Source Toolchains to Manage Cloud               Computinghttp://www.slideshare.net/socializedsoftwareby Mark R. Hinkl...
Upcoming SlideShare
Loading in...5
×

Open Source Toolchains to Manage Cloud Infrastructure

4,316

Published on

Open Source Toolchains to Manage Cloud Infrastructure presentation for Cloud Computing Expo East - June 6, 2011.

Added APIs (jclouds, fog, libcloud, deltacloud)

Published in: Technology, Business
0 Comments
12 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
4,316
On Slideshare
0
From Embeds
0
Number of Embeds
2
Actions
Shares
0
Downloads
285
Comments
0
Likes
12
Embeds 0
No embeds

No notes for slide
  • How many users are using
  • IN the late 1800s English economist William Stanley Jevons noticed that despite huge increases in the efficiency of coal burning, demand actually increased rather than decreased. In cloud computing despite huge cost savings the consumption of cloud computing is and will continue to grow as demand increases. The downside is that this increased consumption has the ability to increase management burden if users don’t adopt leveraged methods for managing cloud computing.
  • With a credit card and a single administrator can spin up 100s of servers in minutes.
  • How does the management personnel for the cloud scale? …Automation.
  • DevOps is a popular movement among cloud operations professionals who are advocating many of the same methodologes used in Agile software development.
  • Ideally for the cloud you create management toolchains that automate the management of your cloud.
  • Cloud computing promises highly available systems, but if you have a reactive approach you won’t achieve that goal. Build redundant, highly automated systems that can self-heal.
  • Other disciplines like back-up, log management, performance and security (virus,intrusion detection) are important but not core to the delivery of cloud computing systems
  • Tools that can automatically build your systems/install systems
  • Configuration management tools abstract infrastructure into variables that are implemented via “recipes”Interesting new development is the PXE boot Crowbar extension for Chef Server used to provision virtual machines in OpenStack.
  • API lets you develop to one API and move between cloudseasilyTypesof Tasks Accomplished by an APIProvisioning (creating, re-creating, moving, or deleting components e.g. virtual machines, vlans)Configuration (assigning or changing attributes of the architecture such as security and network settings)Cloud ProvidersJclouds – java API Abstraction Libcloud – started by CloudKick (now Rackspace) to abstract clouds, Apache incubator projectDeltacloud – started by Red Hat to abstract clouds, Apache incubator projectFog – started by EngineYard is a ruby gem that allows you to interact with number of cloud API.
  • Java applications are developed and checked into Source Control Manager (SCM)Applications built and dependencies resolved via MavenBuilt and unit-tested by HudsonChanges recorded in Change Management Database (CMDB)Deployed by ControlTier (RunDeck)
  • Transcript of "Open Source Toolchains to Manage Cloud Infrastructure"

    1. 1. Open Source Toolchainsto Manage CloudInfrastructureMark R. HinkleVP of CommunityCloud.comTwitter:@clouddotcom@mrhinkleEmail:mrhinkle@cloud.commrhinkle@gmail.com
    2. 2. 2Introduction•  Responsible for Driving Adoption of CloudStack Open Source Cloud Computing Software•  Former manager of Zenoss Open Source project 100,000 users, 1.5 million downloads•  Former Linux Desktop Advocate (Zealot?)•  Former LinuxWorld Magazine Editor-in-Chief•  Open Management Consortium Conspirator•  Open Desktop Consortium Instigator•  Author - “Windows to Linux Business Desktop Migration” - Thomson•  NetDirector Project - Open Source Con guration Management Project•  Sometimes Author and Blogger at SocializedSoftware.com/NetworkWorld•  Start-up junkie, Glutton for punishment
    3. 3. 3Agenda•  Introductions•  Cloud & Systems Management Overview•  Open Source Tools for Cloud Management•  Creating Open Source Toolchains•  Questions (and Hopefully Answers)
    4. 4. Cloud Computing &Systems Management Overview
    5. 5. 5Jevon’s Paradox and the Cloud…the efficiency with which a resource is used tendsto increase (rather than decrease) the rate ofconsumption of that resource.…in cloud computing increasedconsumption can introduces signi cantmanagement overhead despite higherhardware utilization and other bene ts 1865, the English economist William Stanley Jevons
    6. 6. 6Balancing the Equation:Creating the Cloud + =
    7. 7. 7Balancing the Equation:Managing the Cloud + ? =
    8. 8. Automation Balances the 8Equation•  MeatCloud, Can’t Keep up with Cloud Computing•  Devops & Agile IT Philosophy•  Script Repetitive Tasks•  Integrate Tools•  Automate, Automate, Automate
    9. 9. DevOps – Agile Management for 9the Cloud•  Agile, fast moving•  Smaller incremental change•  Increased coordination across groups (development/operations)•  Eye towards Automation
    10. 10. 10Management ToolchainsToolchain (n): MonitoringA set of tools where Patching and Provisioningthe output of onetool becomes the Con gurationinput of another tool
    11. 11. 11The Myth of the NinesAvailability % Downtime per Downtime per Downtime per Year Month Week99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes99.95% 4.38 hours 21.56 minutes 5.04 minutes99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes99.999% ( ve nines) 5.26 minutes 25.9 seconds 6.05 seconds99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds Average polling interval for monitoring? 5 minutes? Even superhuman operations people can’t be alerted and take action in under 5 minutes. One outage per year could drop service level to three nines or worse.
    12. 12. Open Source Tools toManage the Cloud
    13. 13. 132011 CLOUD COMPUTING OUTLOOK SURVEYhttp://www.cloud.com/cloud-computing-outlook
    14. 14. 144 Types of Management Tools Provisioning Installation of operating systems and other softwareConfiguration ManagementSets the parameters for servers, can specifyinstallation parameters Orchestration/Automation Automate tasks across systemsMonitoringRecords errors and health of IT infrastructure
    15. 15. 15Cloudy Tools•  Network Capable•  Cloud “Aware”•  Easy-to-Integrate•  Adhere to Open Standards•  Lend Themselves to Automation
    16. 16. Open Source 16Provisioning Tools Year Started Language License Installation TargetsCobbler 2007 Python GPL Red Hat, OpenSUSE Fedora, Debian, UbuntuFully Automatic 2000 Perl GPL DebianInstallation (FAI)Kickstart ? Python GPL Most .dep and RPM based Linux distrosSpacewalk 2008 Perl, Python, Java GPL Fedora, CentosCrowbar/Chef 2011 Ruby Apache ?
    17. 17. Open Source 17Configuration Management Tools Year Started Language License Client/ServerBcfg2 2003 Python BSD YesCfengine 1993 C Apache YesChef 2009 Ruby Apache Chef Solo – No Chef Server - YesPuppet 2004 Ruby GPL yes
    18. 18. Open Source 18Monitoring Tools Year Started License Language Type of Collection Monitoring MethodsCacti (RRDTool) 2001 GPL PHP Performance SNMP, syslogNagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP, IPMI, syslogOpenNMS 2000 GPL Java Availability/ SNMP, PerformanceZabbix 2001 GPL C/PHP Availability/ SNMP, TCP/ICMP, Performance and IPMI, Synthetic more TransactionsZenoss 2005 GPL Python Availability, SNMP, ICMP, SSH, Performance, Event syslog, WMI Management
    19. 19. Open Source 19Automation/Orchestration Tools Year Language License Client/ Support Started Server OrganizationCapistrano 2006 Ruby MIT Yes NoneRunDeck 2010 Java Apache Yes DTO SolutionsFunc 2007 Python GPL Yes Fedora ProjectMCollective 2009 Ruby Apache Yes PuppetLabs
    20. 20. 20Cloud APIs Aren’t Created EqualOpen Source AbstractionsAPI Languagejclouds javadeltacloud pythonLib cloud pythonFog Ruby
    21. 21. Open Source ManagementToolchains
    22. 22. 22Toolchain Concept Source:http://code.google.com/p/devops-toolchain/wiki/ArchitectureComponents
    23. 23. 23Automated Toolchain Capistrano Command and Application Service RunDeck Fabric Control Orchestration Func Provisioning Activity BCFG2 Con guration Cfengine System Con guration Chef Puppet Cloud: OS Install: Eucalyptus Kickstart Cloud Image OS OpenStack Cobbler Bootstrapping Launch Install CloudStack Spacewalk Abiquo
    24. 24. 24Example: GeekNet•  Servers are automatically built using con guration management software (Puppet)•  Discovery tool (Zenoss) nds infrastructure and populates a CMDB then spits out Hundreds of servers, information to scripts that translate serving web, databases, information to BIND con gurations for DNS and other infrastructure•  Monitoring tool adds hosts to polling tool for some of the world’s (Nagios) to check servers for availability most highly trafficked•  As infrastructure changes systems are updated websites – over 40 million automatically visitors per month.•  Servers can be spun up and managed in minutes, not hours automatically with little or no human interaction
    25. 25. Example Toolchain: 25Online Gaming Studio
    26. 26. Questions?
    27. 27. 27Contact Me Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com Professional: mrhinkle@cloud.com Personal: mrhinkle@gmail.com Professional: http://open.cloud.com Personal: http://socializedsoftware.com Mark R. Hinkle VP of Community Twitter: Open Source Enthusiast @clouddotcom @mrhinkle
    28. 28. Open Source Toolchains to Manage Cloud Computinghttp://www.slideshare.net/socializedsoftwareby Mark R. Hinkle is licensed under a Creative Commons Attribution-ShareAlike 3.0 United States License.
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×