Crash Course in Open Source Cloud Computing


Published on

Introduction on open source technologies that can be used to deploy and manage cloud computing environments. Especially geared toward Infrastructure-as-a-service environments. Updated for presentation at Indiana Linuxfest (3/26/2011).

- Open source cloud storage (CEPH, Swift, Gluster)
- Orchestration - MCollective
- Cloud Infrastructure Diagrams

Published in: Technology, Business
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Crash Course in Open Source Cloud Computing

  1. 1. Crash Course inOpen SourceCloud ComputingMark R. HinkleVP of CommunityCloud.comTwitter: @mrhinkleEmail:
  2. 2. 2%whoami•  Responsible for Driving Adoption of CloudStack Open Source Cloud Computing Software•  Former manager of Zenoss Open Source project 100,000 users, 1.5 million downloads•  Former Linux Desktop Advocate (Zealot?)•  Former LinuxWorld Magazine Editor-in-Chief•  Open Management Consortium Conspirator•  Open Desktop Consortium Instigator•  Author - “Windows to Linux Business Desktop Migration” - Thomson•  NetDirector Project - Open Source Con guration Management Project•  Sometimes Author and Blogger at•  Start-up junkie, Glutton for punishment
  3. 3. 3Agenda•  Cutting Through the Cloud Computing Hype•  Open Source Building Blocks for Cloud Computing•  Open Source Tools for Cloud Management•  Questions (and Hopefully Answers)
  4. 4. Cloud Computing:Cutting throughthe Hype
  5. 5. 5Cloud Computing Adoption•  365,000 web sites are running on Amazon EC2 - Netcraft May 2010•  £76 billion spent on Cloud Computing in the U.K. in 2010 - IT Candor•  Cloud Computing will be a $126 billion market by 2012 - IBM•  20% of Businesses won’t have IT Assets by 2012 - Gartner•  December 2010,Amazon Web Services housed 262 billion objects, more than doubling in size from 102 billion objects at the close of 2009 - Amazon
  6. 6. 6Open Source Virtual Adoption•  41% of users prefer to deploy servers virtually, 29.3% use virtualization whenever possible•  Xen is 2nd most popular virtualization technology after VMware followed by Linux Kernel-Based Virtual Machines (KVM) with 21.3% of the market•  49.2% of users planed to deploy hosted Linux servers in 2010 while 32.6% indicated that they would be deploying hosted Microsoft Windows.•  50.8% indicated they used no specific management tools for cloud computing, 33.3% indicated using tools provided by their hosting provider Source: Zenoss Cloud Computing State of the Union -
  7. 7. 7Five Characteristics of Clouds1. On-Demand Self-Service2. Broad Network Access3. Resource Pooling4. Rapid Elasticity5. Measured Service
  8. 8. 8Cloud Computing Service Models USER CLOUD a.k.a. SOFTWARE AS A SERVICE Single application, multi-tenancy, network-based, one-to-many delivery of applications, all users have same access to features. Examples:, Google Docs, Red Hat Network/RHEL DEVELOPMENT CLOUD a.k.a. PLATFORM-AS-A-SERVICE Application developer model, Application deployed to an elastic service that autoscales, low administrative overhead. No concept of virtual machines or operating system. Code it and deploy it. Examples: Google AppEngine, Windows Azure, Rackspace Site, Red Hat Makara SYSTEMS CLOUD a.k.a INFRASTRUCTURE-AS-A-SERVICE Servers and storage are made available in a scalable way over a network. Examples: EC2,Rackspace CloudFiles, OpenStack, CloudStack, Eucalyptus, Ubuntu Enterprise Cloud, OpenNebula
  9. 9. Deployment Models 9Public,Private & Hybrid Clouds
  10. 10. 10Benefits of Cloud ComputingPublic Cloud Private Cloud•  Pay for What You Need •  Full control of•  Numerous Geographical infrastructure Options •  Security (if you are good)•  Virtually unlimited •  Automation Capabilities capacity •  Flexibility, Flexibility,•  Op-Ex versus Cap-Ex Flexibility•  Automation Capabilities
  11. 11. Building Compute Cloudswith Open Source Software
  12. 12. 12Open Source HypervisorsOpen Source•  Xen Cloud Platform (XCP)•  KVM – Kernel-based Virtualization•  VirtualBox* - Oracle supported Virtualization Solutions•  OpenVZ* - Container-based, Similar to Solaris Containers or BSD Zones•  LXC – User Space chrooted installsProprietary•  VMware•  Citrix Xenserver•  Microsoft Hyper-V
  13. 13. 13Open Source Compute Clouds Year Started License Hypervisors Supported 2010 (Development GPL Xenserver, VMware, Since 2008) KVM, Hyper-V 2008 GPL Xen, KVM, VMware 2010 (Developed by Apache Xen, KVM, Hyper-V NASA by Anso Labs peviously) Ubuntu 2009 (Karmic Koala) GPL Xen,KVM Enterprise Cloud 2009 (Development LGPL v3 VMware ESX and ESXi, 2006) Microsoft Hyper-V, Xen, KVM and Virtual Box
  14. 14. Open Source 14Cloud Computing Storage•  GlusterFS – Scale Out NAS system agregating storage over Ethernet or Infiniband•  CEPH – Distributed file storage system•  OpenStack Storage (Swift) – Long-term storage object storage system•  Sheepdog – Distributed storage for KVM hypervisors•  NFS – Old standby, tried and true, not designed for cloud scale or performance
  15. 15. Cloud APIs Aren’t Created Equal 15Open Source Abstractions•  jclouds•  libcloud•  deltacloud
  16. 16. 16Private Cloud Architecture
  17. 17. Managing Clouds withOpen Source Tools
  18. 18. 18Cloudy Tools•  Network Capable•  Cloud “Aware”•  Easy-to-Integrate•  Adhere to Open Standards•  Lend Themselves to Automation
  19. 19. 19The Myth of the NinesAvailability % Downtime per Downtime per Downtime per Year Month Week99.9% (three nines) 8.76 hours 43.2 minutes 10.1 minutes99.95% 4.38 hours 21.56 minutes 5.04 minutes99.99% (four nines) 52.6 minutes 4.32 minutes 1.01 minutes99.999% ( ve nines) 5.26 minutes 25.9 minutes 6.05 minutes99.9999% (six nines) 31.5 seconds 2.59 seconds .0605 seconds Average polling interval for monitoring? 5 minutes? Even superhuman operations people can’t be alerted and take action in under 5 minutes. One outage per year could drop service level to three nines or worse.
  20. 20. Automation Unlocks 20the Potential of the Cloud•  MeatCloud, Can’t Keep up with Cloud Computing•  Devops & Agile IT Philosophy•  Script Repetitive Tasks•  Automate, Automate, Automate
  21. 21. 214 Types of Management Tools Provisioning Installation of operating systems and other softwareConfiguration ManagementSets the parameters for servers, can specifyinstallation parameters Orchestration/Automation Automate tasks across systemsMonitoringRecords errors and health of IT infrastructure
  22. 22. 22Management ToolchainsToolchain (n): MonitoringA set of tools wherethe output of one Patching and Provisioningtool becomes theinput of another tool Con guration
  23. 23. Open Source 23Provisioning Tools Year Started Language License Installation TargetsCobbler 2007 Python GPL Red Hat, OpenSUSE Fedora, Debian, UbuntuFully Automatic 2000 Perl GPL DebianInstallation (FAI)Kickstart ? Python GPL Most .dep and RPM based Linux distrosSpacewalk 2008 Perl, Python, Java GPL Fedora, CentosViper 2008 Perl GPL Debian
  24. 24. Open Source 24Configuration Management Tools Year Started Language License Client/ServerBcfg2 2003 Python BSD YesCfengine 1993 C Apache YesChef 2009 Ruby Apache Chef Solo – No Chef Server - YesPuppet 2004 Ruby GPL yes
  25. 25. Open Source 25Monitoring Tools Year Started License Language Type of Collection Monitoring MethodsCacti 2001 GPL PHP Performance SNMP, syslogNagios 1999 GPL C/PHP Availability SNMP,TCP, ICMP, IPMI, syslogOpenNMS 2000 GPL Java Availability/ SNMP, PerformanceZabbix 2001 GPL C/PHP Availability/ SNMP, TCP/ICMP, Performance and IPMI, Synthetic more TransactionsZenoss 2005 GPL Python Availability, SNMP, ICMP, SSH, Performance, Event syslog, WMI Management
  26. 26. Open Source 26Automation/Orchestration Tools Year Language License Client/ Support Started Server OrganizationAutomateIT 2009 Ruby GPL No NoneCapistrano 2006 Ruby MIT Yes NoneRunDeck 2010 Java Apache Yes DTO SolutionsFunc 2007 Python GPL Yes Fedora ProjectMCollective 2009 Ruby Apache Yes PuppetLabs
  27. 27. 27Automated Toolchain Capistrano Command and Application Service RunDeck Fabric Control Orchestration Func Provisioning Activity BCFG2 Con guration Cfengine System Con guration Chef Puppet Cloud: OS Install: Eucalyptus Kickstart Cloud Image OS OpenStack Cobbler Bootstrapping Launch Install CloudStack Spacewalk Abiquo
  28. 28. Questions?
  29. 29. 29Contact Me Professional: Personal: Professional: Personal: Professional: Mark R. Hinkle Personal: VP of Community Open Source Fanboy Twitter: @mrhinkle
  30. 30. Crash Course in Cloud Computingby Mark R. Hinkle is licensed under a Creative Commons Attribution-ShareAlike 3.0 United States License.