Cloud Expo East 2013: Essential Open Source Software for Building the Open Cloud
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

Cloud Expo East 2013: Essential Open Source Software for Building the Open Cloud

on

  • 892 views

Cloud computing is more than a buzz-phrase it’s a transformative IT paradigm shift. The emphasis in the cloud is on elasticity, scalability, agility and open. Not just open standards but open APIs ...

Cloud computing is more than a buzz-phrase it’s a transformative IT paradigm shift. The emphasis in the cloud is on elasticity, scalability, agility and open. Not just open standards but open APIs and open source. The delivery of software is also going through a paradigm shift. Open source software was often a commoditization of a market leader; Unix to Linux or Oracle to MySQL what’s changing is that the iterative nature, user context and the motto of releasing early and often are driving real innovation in open source.

This session will cover those essential open source technologies for delivering cloud computing in the enterprise.
Speaker Bio:
Mark Hinkle is the Senior Director, Open Source Solutions at Citrix Systems Inc. He joined Citrix as a result of their July 2011 acquisition of Cloud.com where he was their Vice President of Community. He is currently responsible for Citrix open source efforts around the open source cloud computing platform, Apache CloudStack and the Xen Hypervisor. Previously he was the VP of Community at Zenoss Inc., a producer of the open source application, server, and network management software, where he grew the Zenoss Core project to over 100,000 users and 20,000 organizations on all seven continents. He also is a longtime open source expert and author having served as Editor-in-Chief for both LinuxWorld Magazine and Enterprise Open Source Magazine. His blog on open source, technology, and new media can be found at http://www.socializedsoftware.com.

Statistics

Views

Total Views
892
Views on SlideShare
892
Embed Views
0

Actions

Likes
3
Downloads
37
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

CC Attribution-ShareAlike LicenseCC Attribution-ShareAlike License

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

Cloud Expo East 2013: Essential Open Source Software for Building the Open Cloud Presentation Transcript

  • 1. Essential Open Source SoftwareforBuilding the Open CloudMark R. HinkleSr. Director , OPEN SOURCE SOLUTIONSCitrix Systems INC.@mrhinklemrhinkle@gmail.comCloud Computing Expo East 2013
  • 2. Mark Hinkle, Sr. Director, Open Source Solutions•  Dedicated  to  the  success  of  the  Apache  CloudStack,  Open  Daylight  &  Xen  Project  Communi3es  on  Citrix  behalf  •  Run  BuildACloud.org  learning  ac3vi3es  all  over  the  world  •  Joined  Citrix  via  Cloud.com  acquisi3on  July  2011  •  Zenoss  Core  Open  Source  project  to  100,000  users,  1.5  million  downloads  •  Former  LinuxWorld  Magazine  Editor-­‐in-­‐Chief  •  Open  Management  ConsorGum  organizer  •  Author  -­‐  “Windows  to  Linux  Business  Desktop  MigraGon”  –  Thomson  •  NetDirector  Project  -­‐  Open  Source  Configura3on  Management    •  Some3mes  Author  and  Blogger  at  SocializedSoJware.com  •  NetworkWorld  Open  Source  Subnet  Essential OSS for Building the Open Cloud by @mrhinkle2
  • 3. Why Open Source and the Cloud Computing?•  User-­‐Driven  Context  from  Solving  Real  Problems  •  Lower  Barrier  to  Par3cipa3on  •  Larger  user  base,  users  helping  users    •  Aggressive  release  cycles  stay  current  with  the  state-­‐of-­‐the-­‐art  •  Open  Source  innova3ng  faster  than  commercial  •  Open  data,  Open  standards,  Open  APIs  Essential OSS for Building the Open Cloud by @mrhinkle3
  • 4. Quick Cloud Computing Overview orthe Obligatory “What is the CloudExplanation”
  • 5. Five Characteristics of Cloud1.  On-­‐Demand  Self-­‐Service  2.  Broad  Network  Access  3.  Resource  Pooling  4.  Rapid  Elas3city  5.  Measured  Service  Essential OSS for Building the Open Cloud by @mrhinkle5
  • 6. Cloud Computing Service ModelsUSER CLOUD a.k.a. SOFTWARE AS A SERVICESingle application, multi-tenancy, network-based, one-to-many delivery ofapplications, all users have same access to features.Examples: Salesforce.com, Google Docs, Red Hat Network/RHELDEVELOPMENT CLOUD a.k.a. PLATFORM-AS-A-SERVICEApplication developer model, Application deployed to an elastic service thatautoscales, low administrative overhead. No concept of virtual machines oroperating system. Code it and deploy it.Examples: VMware CloudFoundry, Google AppEngine, Windows Azure,Rackspace Sites, Red Hat OpenShift, Active State Stackato, AppfogSYSTEMS CLOUD a.k.a INFRASTRUCTURE-AS-A-SERVICEServers and storage are made available in a scalable way over a network.Examples: EC2,Rackspace CloudFiles, OpenStack, CloudStack,Eucalyptus, OpenNebulaEssential OSS for Building the Open Cloud by @mrhinkle6
  • 7. Deployment Models: Public, Private & HybridEssential OSS for Building the Open Cloud by @mrhinkle7
  • 8. Building Open Source Clouds
  • 9. First Question…What do you want to do?ü Self-­‐Service  enterprise  workloads  ü Customer  facing  Applica3ons    ü Dev/Test  environments  ü Provide  Something-­‐as-­‐a-­‐Service  ü Deploy  similar  workloads  at  scale  ü All  of  the  Above  9Essential OSS for Building the Open Cloud by @mrhinkle
  • 10. Generic Cloud ArchitectureEssential OSS for Building the Open Cloud by @mrhinkle10
  • 11. Building Open Source CloudsVirtualized    Compute  Virtualized    Storage  Virtualized    Networking  Infrastructure  Orchestra3on  Pla^orm-­‐as-­‐a-­‐Service    Infrastructure-­‐as-­‐a-­‐Service    Ops  Management  Tools  Enterprise 3rd Party DevelopersEssential OSS for Building the Open Cloud by @mrhinkleConfig  Mgmt  Monitoring  Provisioning   Orchestra3on  
  • 12. Choose  what  you  want,  customize  for    needs.    Essential OSS for Building the Open Cloud by @mrhinkle
  • 13. The Three Virtual Pillars of CloudComputing13Compute  Network  Storage  
  • 14. HypervisorsOpen  Source  •  Xen,  Project    Xen  Cloud  PlaMorm  (XCP)  •  KVM  –  Kernel-­‐based  VirtualizaGon  •  VirtualBox*  -­‐  Oracle  supported  Virtualiza3on  Solu3ons    •  OpenVZ*  -­‐  Container-­‐based,  Similar  to  Solaris  Containers  or  BSD  Zones  •  LXC  –  User  Space  chrooted  installs    Proprietary  •  VMware  •  Citrix  Xenserver  (based    •  Microsoc  Hyper-­‐V  •  OracleVM  (Based  on  OS  Xen)  Essential OSS for Building the Open Cloud by @mrhinkle14
  • 15. Scale-Up or Scale-OutVerGcal  Scaling  (Scale-­‐Up)    Allocate  addi3onal  resources  to  VMs,  requires  a  reboot,  no  need  for  distributed  app  logic,  single-­‐point  of  OS  failure    Horizontal  Scaling  (Scale-­‐Out)  Applica3on  needs  logic  to  work  in  distributed  fashion  (e.g.  HA-­‐Proxy  and  Apache,  Hadoop)  Essential OSS for Building the Open Cloud by @mrhinkle15
  • 16. Sourcing Cloud AppliancesTool/Project   What  you  can  do  with  them  Bitnami   BitNami  provides  free,  ready  to  run  environments  for  your  favorite  open  source  web  applica3ons  and  frameworks,  including  Drupal,  Joomla!,  Wordpress,  PHP,  Rails,  Django  and  many  more.    Boxgrinder   BoxGrinder  is  a  set  of  projects  that  help  you  grind  out  appliances  for  mul3ple  virtualiza3on  and  Cloud  providers  Oz   Command-­‐line  tool  that  has  the  ability  to  create  images  for  common  Linux  distribu3ons  to  run  on  KVM  SUSE  Studio   SUSE  Studio  supports  building  and  deploying  directly  to  cloud  services  such  as  Amazon  EC2.    Essential OSS for Building the Open Cloud by @mrhinkle16
  • 17. Cloud Computing StorageProject     DescripGon  Ceph   Distributed  file  storage  system  developed  by  DreamHost  GlusterFS   Scale  Out  NAS  system  aggrega3ng  storage  over  Ethernet  or  Infiniband  OpenStack    Storage  Long-­‐term  object  storage  system  Riak  CS     Riak  CS  is  open  source  socware  designed  to  provide  simple,  available,  distributed  cloud  storage  at  any  scale.  Riak  CS  is  S3-­‐API  compa3ble  and  supports  per-­‐tenant  repor3ng  for  billing  and  metering  use  cases.  Sheepdog   Distributed  storage  for  KVM  hypervisors  Essential OSS for Building the Open Cloud by @mrhinkle17
  • 18. Software Defined Networking (SDN)
  • 19. Cloud Promise, Reality and NetworksCloud  Promise   Cloud  Reality  Centralized  ConfiguraGon  and  AutomaGon  Without  true  virtualiza3on,  network  devices  must  s3ll  be  manually  configured.  Instant  Self-­‐Service  Provisioning  In  a  physical  network,  it  could  take  a  long  3me  for  network  engineer  to  provision  new  services.  ElasGcity  and  Scalability   By  horizontally  scaling  up  the  physical  network,  elas3city  is  lost.  Designed  for  Failure   Failover  can  be  automated  and  physical  network  limita3ons  can  be  alleviated.    Source: MidokuraEssential OSS for Building the Open Cloud by @mrhinkle19
  • 20. Open FlowOpenFlow  enables  networks  to  evolve,  by  giving  a  remote  controller  the  power  to  modify  the  behavior  of  network  devices,  through  a  well-­‐defined  "forwarding  instruc3on  set".  The  growing  OpenFlow  ecosystem  now  includes  routers,  switches,  virtual  switches,  and  access  points  from  a  range  of  vendors.  Image from http://www.openflow.org/documents/openflow-wp-latest.pdfEssential OSS for Building the Open Cloud by @mrhinkle20
  • 21. Overview of Software Defined NetworkingBusiness  Applica3ons  Network  Services  SDNControlSoftwareAPI APINetwork  Devices  Network  Devices  Network  Devices  Network  Devices  Network  Devices  Network  Devices  ApplicationLayerControlLayerInfrastructureLayerControl Data Plane Interface (e.g. OpenFlow)Essential OSS for Building the Open Cloud by @mrhinkle21
  • 22. Software Defined Networking (SDN)Project DescriptionFloodlight   The  Floodlight  controller  is  an  enterprise-­‐class,  Apache-­‐licensed,  Java-­‐based  OpenFlow  Controller.  Indigo   Indigo  is  an  open  source  project  to  support  OpenFlow  on  a  range  of  physical  switches.  By  leveraging  hardware  features  of  Ethernet  switch  ASICs,  Indigo  supports  high  rates  for  high  port  counts,  up  to  48  10-­‐gigabit  ports.  Mul3ple  gigabit  pla^orms  with  10-­‐gigabit  uplinks  are  also  supported.    Open  Daylight   Linux  Founda3on  Collabora3ve  Project  based  on  Cisco  One  Controller  and  plugins  from  numerous  vendors  in  development.    E.g  IBM  DOVE  OpenStack  “Quantum”    Networking  Pluggable,  scalable,  API-­‐driven  network  and  IP  management  Open  vSwitch   Open  vSwitch  is  a  open  source  (ASL  2.0),  mul3layer  virtual  switch  designed  to  enable  massive  network  automa3on  through  programma3c  extension,  while  s3ll  suppor3ng  standard  management  interfaces  and  protocols  (e.g.  NetFlow,  sFlow,  SPAN,  RSPAN,  CLI,  LACP,  802.1ag).  Essential OSS for Building the Open Cloud by @mrhinkle22
  • 23. Cloud Orchestration
  • 24. Compute Clouds (IaaS)Year  Started   License   VirtualizaGon  Technologies  Apache  CloudStack  2008   Apache   Xenserver,  Xen  Cloud  Pla^orm,  KVM,  VMware  (Hyper-­‐V  developing)  Eucalyptus   2006   GPL     Xen,  KVM,  VMware  (commercial  version)  OpenNebula   2005   Apache   Xen,  KVM,  VMware  OpenStack   2010  (Developed  by    NASA  by  Anso  Labs    previously)    Apache   VMware  ESX  and  ESXi,  ,  Xen,  Xen  Cloud  Pla^orm  KVM,  LXC,  QEMU  and  Virtual  Box  Numerous companies are building cloud software on OpenStack including Nebula, Piston Inc., CloudScalingEssential OSS for Building the Open Cloud by @mrhinkle24
  • 25. OpenStack – Ecosystem of ProjectsEnterprise  Message  Queue  based  on  Rabbit  MQ  (ESB)  Object  Storage  “Swic”  Image  Service  “Glance”    Compute  “Nova”  Dashboard  “Horizon”  KVM,  VMware,  Xen  Cloud  Pla^orm  Ceph,  Gluster  Advanced  Cloud  and  Networking  services  accessing  the  Quantum  API  Firewall  Service    Gateway  Service  Quantum  Networking  Fabric  REST  API  Plugins  OpenvSwitch  Quantum  Plugin-­‐ins  Iden3ty  Services  “Keystone”  API  20+ Collective projects hosted at: https://launchpad.net/openstackEssential OSS for Building the Open Cloud by @mrhinkle25
  • 26. Platform-as-a-Service (PaaS)Project   Year  Started   Sponsors   Languages/Frameworks  CloudFoundry   2011   VMware   Spring  for  Java,  Ruby  for  Rails  and  Sinatra,  node.js,  Grails,  Scala  on  Lic  and  more  via  partners  (e.g.  Python,  PHP)  Cloudify   2012   Gigaspaces   [Groovy  for  deployment  recipes]  OpenShic   2011   Red  Hat   Java,  Ruby,  PHP,  Perl  and  Python    Stackato   2012   Ac3veState   Java,  Python,  PHP,  Ruby,  Perl,  Node.js,  others  WSO2  Stratus   2010   WSO2   Jboss,  Java  EE6  Essential OSS for Building the Open Cloud by @mrhinkle26
  • 27. Cloud APIs•  jclouds  •  libcloud  •  deltacloud  •  fog  Essential OSS for Building the Open Cloud by @mrhinkle27
  • 28. Cloud Management Tools
  • 29. Automation in the CloudMeat Cloud Cloud OperationsEssential OSS for Building the Open Cloud by @mrhinkle29
  • 30. 4 Types of Management ToolsProvisioningInstallation of operating systems and other softwareConfiguration ManagementSets the parameters for servers, can specify installation parametersOrchestration/AutomationAutomate tasks across systemsMonitoringRecords errors and health of IT infrastructureEssential OSS for Building the Open Cloud by @mrhinkle30
  • 31. Management ToolchainsConfigura3on  Patching  and  Provisioning  Monitoring  Toolchain (n):A set of tools wherethe output of onetool becomes theinput of another toolEssential OSS for Building the Open Cloud by @mrhinkle31
  • 32. ProvisioningProject Installation TargetsApache  Provisionr(incuba3ng)  Can  provision  10s  to  1000s  of  machines  on  various  clouds.    Cobbler   Distributed  virtual  infrastructure  using  koan  (kickstart  of  a  network  to  PXE  boot  VMs)  for  Red  Hat,  OpenSUSE  Fedora,  Debian,  Ubuntu  VMs  Crowbar    (Bare  metal  provisioning)  JuJu   Public  Clouds  -­‐    Amazon  Web  Services  HP  Cloud,    Private  OpenStack  clouds,  Bare  Metal  via  MAAS.    Salt  Cloud     Tool  to  provision  “salted”  VMs  that  can  then  be  updated  by  a  central  server  via  ZeroMQ  Essential OSS for Building the Open Cloud by @mrhinkle32
  • 33. Configuration Management ToolsProject   Year  Started   Language   License   Client/Server  Cfengine   1993   C   Apache   Yes  Chef   2009   Ruby   Apache   Chef  Solo  –  No    Chef  Server  -­‐  Yes  Puppet   2004   Ruby   GPL     Yes  &  standalone  Salt   2011   Python   Apache   yes  Essential OSS for Building the Open Cloud by @mrhinkle33
  • 34. Automation/Orchestration ToolsProject   DescripGon  Ansible   Ansibles  SSH-­‐key  based  access  allows  contributors  to  the  Fedora  Project  to  assist  in  automa3ng  infrastructure  while  having  access  limited  appropriately.    Capistrano   U3lity  and  framework  for  execu3ng  commands  in  parallel  on  mul3ple  remote  machines,  via  SSH.  It  uses  a  simple  DSL  that  allows  you  to  define  tasks,  which  may  be  applied  to  machines  in  certain  roles  RunDeck   Rundeck  is  an  open-­‐source  process  automa3on  and  command  orchestra3on  tool  with  a  web  console.  Func   Func  provides  a  two-­‐way  authen3cated  system  for  generically  execu3ng  tasks,  integra3ons  with  puppet  and  cobbler.  MCollec3ve   The  Marionese  Collec3ve  AKA  MCollec3ve  is  a  framework  to  build  server  orchestra3on  or  parallel  job  execu3on  systems.  Salt   Execute  arbitrary  shell  commands  or  choose  from  dozens  of  pre-­‐built  modules  of  common  (or  complex)  commands.  Scalr   Provide  scaling  across  mul3ple  cloud  compu3ng  pla^orms,  integrates  with  Chef.    Essential OSS for Building the Open Cloud by @mrhinkle34
  • 35. Configuration Management ToolsProject   Year  Started   Language   License   Client/Server  Cfengine   1993   C   Apache   Yes  Chef   2009   Ruby   Apache   Chef  Solo  –  No    Chef  Server  -­‐  Yes  Puppet   2004   Ruby   GPL     Yes  &  standalone  Salt   2011   Python   Apache   yes  Essential OSS for Building the Open Cloud by @mrhinkle35
  • 36. Automation/Orchestration ToolsProject   DescripGon  Ansible   Ansibles  SSH-­‐key  based  access  allows  contributors  to  the  Fedora  Project  to  assist  in  automa3ng  infrastructure  while  having  access  limited  appropriately.    Capistrano   U3lity  and  framework  for  execu3ng  commands  in  parallel  on  mul3ple  remote  machines,  via  SSH.  It  uses  a  simple  DSL  that  allows  you  to  define  tasks,  which  may  be  applied  to  machines  in  certain  roles  RunDeck   Rundeck  is  an  open-­‐source  process  automa3on  and  command  orchestra3on  tool  with  a  web  console.  Func   Func  provides  a  two-­‐way  authen3cated  system  for  generically  execu3ng  tasks,  integra3ons  with  puppet  and  cobbler.  MCollec3ve   The  Marionese  Collec3ve  AKA  MCollec3ve  is  a  framework  to  build  server  orchestra3on  or  parallel  job  execu3on  systems.  Salt   Execute  arbitrary  shell  commands  or  choose  from  dozens  of  pre-­‐built  modules  of  common  (or  complex)  commands.  Scalr   Provide  scaling  across  mul3ple  cloud  compu3ng  pla^orms,  integrates  with  Chef.    Essential OSS for Building the Open Cloud by @mrhinkle36
  • 37. Monitoring ToolsLicense   Type  of  Monitoring   CollecGon  Methods  Cac3    /  RRDTool   GPL     Performance   SNMP,  syslog  Graphite   Apache  2.0   Performance   Agent  Nagios   GPL   Availability   SNMP,TCP,  ICMP,  IPMI,  syslog  Zabbix   GPL     Availability/  Performance  and  more  SNMP,  TCP/ICMP,  IPMI,  Synthe3c  Transac3ons  Zenoss   GPL   Availability,  Performance,  Event  Management  SNMP,  ICMP,  SSH,  syslog,  WMI  Essential OSS for Building the Open Cloud by @mrhinkle37
  • 38. Conceptual Automated ToolchainBootStrapped  Image  CloudStack  OpenStack  ConfiguraGon  Puppet  Chef  Start/Stop  Services  RunDeck  Capistrano  MCollec3ve  Provision  Cobbler  SUSE  Stuido  Monitoring  Nagios  Zenoss    Cac3    Generate  Images  SUSE  Studio  BoxGrinder  Essential OSS for Building the Open Cloud by @mrhinkle38
  • 39. Questions?Essential OSS for Building the Open Cloud by @mrhinkle39Slides  Can  be  Viewed  and  Downloaded  at:  hsp://www.slideshare.net/socializedsocware/  Copyright Mark R. Hinkle, available under the CCbySA license some rights reserved. 2012 -2013Professional: mark.hinkle@citrix.comPersonal: mrhinkle@gmail.comPhone: 919.228.8049Personal: http://www.socializedsoftware.comTwitter: @mrhinkleMark R. HinkleSenior Director,Open Source SolutionsCitrix Systems Inc.Open Source Enthusiast