Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud Computing


Published on

Presented on April 27th, 2013 at LinuxFest NW

Imagine it’s eight o’clock on a Thursday morning and you awake to see a bulldozer out your window ready to plow over your data center. Normally you may wish to consult the Encyclopedia Galáctica to discern the best course of action but your copy is likely out of date. And while the Hitchhiker’s Guide to the Galaxy (HHGTTG) is a wholly remarkable book it doesn’t cover the nuances of cloud computing. That’s why you need the Hitchhiker’s Guide to Cloud Computing (HHGTCC) or at least to attend this talk understand the state of open source cloud computing. Specifically this talk will cover infrastructure-as-a-service, platform-as-a-service and developments in big data and how to more effectively take advantage of these technologies using open source software. Technologies that will be covered in this talk include Apache CloudStack, Chef, CloudFoundry, NoSQL, OpenStack, Puppet and many more.

Specific topics for discussion will include:

Infrastructure-as-a-Service - The Systems Cloud - Get a comparision of the open source cloud platforms including OpenStack, Apache CloudStack, Eucalyptus, OpenNebula
Platform-as-a-Service - The Developers Cloud - Find out what tools are availble to build portable auto-scaling applications including CloudFoundry, OpenShift, Stackato and more.
Data-as-a-Service - The Analytics Cloud - Want to figure out the who, what , where , when and why of big data ? You get an overview of open source NoSQL databases and technologies like MapReduce to help crunch massive data sets in the cloud.
Finally you'll get a overview of the tools that can help you really take advantage of the cloud? Want to auto-scale virtual machiens to serve millions of web pages or want to automate the configuration of cloud computing environments. You'll learn how to combine these tools to provide continous deployment systems that will help you earn DevOps cred in any data center.

[Finally, for those of you that are Douglas Adams fans please accept the deepest apologies for bad analogies to the HHGTTG.]

Published in: Technology
  • Be the first to comment

LinuxFest NW 2013: Hitchhiker's Guide to Open Source Cloud Computing

  1. 1. Hitchhiker’s Guideto The Open CloudLinuxFest NorthWest 2013Mark R. HinkleSr. Director , OPEN SOURCE SOLUTIONSCitrix Systems INC.@mrhinklemrhinkle@gmail.comRev 1.1
  2. 2. Mark HinkleSr.Director,OpenSourceSolutions,Citrix•  Dedicated  to  the  success  of  the  Apache  CloudStack,  Linux  Founda3on’s  Open  Daylight  &  Xen  Project  communi3es  on  Citrix  behalf  •  Run  learning  ac3vi3es  all  over  the  world  •  Joined  Citrix  via  acquisi3on  July  2011  •  VP  of  Community  at  Zenoss  drove  Zenoss  Core  Open  Source  project  to  100,000  users,  1.5  million  downloads  •  Former  LinuxWorld  Magazine  Editor-­‐in-­‐Chief  •  Open  Management  ConsorHum  organizer  •  Author  -­‐  “Windows  to  Linux  Business  Desktop  MigraHon”  –  Thomson(2006)  •  NetDirector  Project  -­‐  Open  Source  Configura3on  Management    •  Some3mes  Author  and  Blogger  at  •  Blogger  NetworkWorld  Open  Source  Subnet  theLawsofAbstraction
  3. 3. Mark Hinkle, Sr. Director, Open Source Solutions•  Dedicated  to  the  success  of  the  Apache  CloudStack,  Open  Daylight  &  Xen  Project  Communi3es  on  Citrix  behalf  •  Run  learning  ac3vi3es  all  over  the  world  •  Joined  Citrix  via  acquisi3on  July  2011  •  Zenoss  Core  Open  Source  project  to  100,000  users,  1.5  million  downloads  •  Former  LinuxWorld  Magazine  Editor-­‐in-­‐Chief  •  Open  Management  ConsorHum  organizer  •  Author  -­‐  “Windows  to  Linux  Business  Desktop  MigraHon”  –  Thomson  •  NetDirector  Project  -­‐  Open  Source  Configura3on  Management    •  Some3mes  Author  and  Blogger  at  •  NetworkWorld  Open  Source  Subnet  Hitchhiker’s Guide to the Open Cloud by @mrhinkle3
  4. 4. Why Open Source and the Cloud Computing?•  User-­‐Driven  Context  from  Solving  Real  Problems  •  Lower  Barrier  to  Par3cipa3on  •  Larger  user  base,  users  helping  users    •  Aggressive  release  cycles  stay  current  with  the  state-­‐of-­‐the-­‐art  •  Open  Source  innova3ng  faster  than  commercial  •  Open  data,  Open  standards,  Open  APIs  Hitchhiker’s Guide to the Open Cloud by @mrhinkle4
  5. 5. Quick Cloud Computing Overview orthe Obligatory “What is the CloudExplanation”Infinite Improbability Drive
  6. 6. Five Characteristics of Cloud1. On-­‐Demand  Self-­‐Service  2. Broad  Network  Access  3. Resource  Pooling  4. Rapid  Elas3city  5. Measured  Service  Hitchhiker’s Guide to the Open Cloud by @mrhinkle6
  7. 7. Cloud Computing Service ModelsUSER CLOUD a.k.a. SOFTWARE AS A SERVICESingle application, multi-tenancy, network-based, one-to-many delivery ofapplications, all users have same access to features.Examples:, Google Docs, Red Hat Network/RHELDEVELOPMENT CLOUD a.k.a. PLATFORM-AS-A-SERVICEApplication developer model, Application deployed to an elastic service thatautoscales, low administrative overhead. No concept of virtual machines oroperating system. Code it and deploy it.Examples: VMware CloudFoundry, Google AppEngine, Windows Azure,Rackspace Sites, Red Hat OpenShift, Active State Stackato, AppfogSYSTEMS CLOUD a.k.a INFRASTRUCTURE-AS-A-SERVICEServers and storage are made available in a scalable way over a network.Examples: EC2,Rackspace CloudFiles, OpenStack, CloudStack,Eucalyptus, OpenNebulaHitchhiker’s Guide to the Open Cloud by @mrhinkle7
  8. 8. Deployment Models: Public, Private & HybridHitchhiker’s Guide to the Open Cloud by @mrhinkle8
  9. 9. Building Open Source Clouds
  10. 10. Cloud ArchitectureHitchhiker’s Guide to the Open Cloud by @mrhinkle10
  11. 11. HypervisorsOpen  Source  •  Xen,  Project    Xen  Cloud  PlaNorm  (XCP)  •  KVM  –  Kernel-­‐based  VirtualizaHon  •  VirtualBox*  -­‐  Oracle  supported  Virtualiza3on  Solu3ons    •  OpenVZ*  -­‐  Container-­‐based,  Similar  to  Solaris  Containers  or  BSD  Zones  •  LXC  –  User  Space  chrooted  installs    Proprietary  •  VMware  •  Citrix  Xenserver  (based    •  Microsod  Hyper-­‐V  •  OracleVM  (Based  on  OS  Xen)  Hitchhiker’s Guide to the Open Cloud by @mrhinkle11
  12. 12. Open Virtual Machine FormatsOpen  VirtualizaHon  Format  (OVF)  is  an  open  standard  for  packaging  and  distribu3ng  virtual  appliances  or  more  generally  sodware  to  be  run  in  virtual  machines.  Formats  for  hypervisors/cloud  technologies:      •  Amazon  -­‐  AMI  •  KVM  –  QCOW2  •  VMware  –  VMDK  •  Xen  Project–  IMG  •  VHD  –  Virtual  Hard  Disk    -­‐  Hyper-­‐V  Hitchhiker’s Guide to the Open Cloud by @mrhinkle12
  13. 13. Sourcing Cloud AppliancesTool/Project   What  you  can  do  with  them  Bitnami   BitNami  provides  free,  ready  to  run  environments  for  your  favorite  open  source  web  applica3ons  and  frameworks,  including  Drupal,  Joomla!,  Wordpress,  PHP,  Rails,  Django  and  many  more.    Boxgrinder   BoxGrinder  is  a  set  of  projects  that  help  you  grind  out  appliances  for  mul3ple  virtualiza3on  and  Cloud  providers  Oz   Command-­‐line  tool  that  has  the  ability  to  create  images  for  common  Linux  distribu3ons  to  run  on  KVM  SUSE  Studio   SUSE  Studio  supports  building  and  deploying  directly  to  cloud  services  such  as  Amazon  EC2.    Hitchhiker’s Guide to the Open Cloud by @mrhinkle13
  14. 14. Scale-Up or Scale-OutVerHcal  Scaling  (Scale-­‐Up)    Allocate  addi3onal  resources  to  VMs,  requires  a  reboot,  no  need  for  distributed  app  logic,  single-­‐point  of  OS  failure    Horizontal  Scaling  (Scale-­‐Out)  Applica3on  needs  logic  to  work  in  distributed  fashion  (e.g.  HA-­‐Proxy  and  Apache,  Hadoop)  Hitchhiker’s Guide to the Open Cloud by @mrhinkle14
  15. 15. Compute Clouds (IaaS)Year  Started   License   VirtualizaHon  Technologies  Apache  CloudStack  2008   Apache   Xenserver,  Xen  Cloud  Planorm,  KVM,  VMware  (Hyper-­‐V  developing)  Eucalyptus   2006   GPL     Xen,  KVM,  VMware  (commercial  version)  OpenNebula   2005   Apache   Xen,  KVM,  VMware  OpenStack   2010  (Developed  by    NASA  by  Anso  Labs    previously)    Apache   VMware  ESX  and  ESXi,  ,  Xen,  Xen  Cloud  Planorm  KVM,  LXC,  QEMU  and  Virtual  Box  Numerous companies are building cloud software on OpenStack including Nebula, Piston Inc., CloudScalingHitchhiker’s Guide to the Open Cloud by @mrhinkle15
  16. 16. OpenStack – Ecosystem of ProjectsEnterprise  Message  Queue  based  on  Rabbit  MQ  (ESB)  Object  Storage  “Swid”  Image  Service  “Glance”    Compute  “Nova”  Dashboard  “Horizon”  KVM,  VMware,  Xen  Cloud  Planorm  Ceph,  Gluster  Advanced  Cloud  and  Networking  services  accessing  the  Quantum  API  Firewall  Service    Gateway  Service  Quantum  Networking  Fabric  REST  API  Plugins  OpenvSwitch  Quantum  Plugin-­‐ins  Iden3ty  Services  “Keystone”  API  20+ Collective projects hosted at:’s Guide to the Open Cloud by @mrhinkle16
  17. 17. Cloud APIs•  jclouds  •  libcloud  •  deltacloud  •  fog  Hitchhiker’s Guide to the Open Cloud by @mrhinkle17
  18. 18. Cloud Computing StorageProject     DescripHon  Ceph   Distributed  file  storage  system  developed  by  DreamHost  GlusterFS   Scale  Out  NAS  system  aggrega3ng  storage  over  Ethernet  or  Infiniband  OpenStack    Storage  Long-­‐term  object  storage  system  Riak  CS     Riak  CS  is  open  source  sodware  designed  to  provide  simple,  available,  distributed  cloud  storage  at  any  scale.  Riak  CS  is  S3-­‐API  compa3ble  and  supports  per-­‐tenant  repor3ng  for  billing  and  metering  use  cases.  Sheepdog   Distributed  storage  for  KVM  hypervisors  Hitchhiker’s Guide to the Open Cloud by @mrhinkle18
  19. 19. Platform-as-a-Service (PaaS)Project   Year  Started   Sponsors   Languages/Frameworks  CloudFoundry   2011   VMware   Spring  for  Java,  Ruby  for  Rails  and  Sinatra,  node.js,  Grails,  Scala  on  Lid  and  more  via  partners  (e.g.  Python,  PHP)  Cloudify   2012   Gigaspaces   [Groovy  for  deployment  recipes]  OpenShid   2011   Red  Hat   Java,  Ruby,  PHP,  Perl  and  Python    Stackato   2012   Ac3veState   Java,  Python,  PHP,  Ruby,  Perl,  Node.js,  others  WSO2  Stratus   2010   WSO2   Jboss,  Java  EE6  Hitchhiker’s Guide to the Open Cloud by @mrhinkle19
  20. 20. Software Defined Networking (SDN)
  21. 21. Overview of Software Defined NetworkingBusiness  Applica3ons  Network  Services  SDNControlSoftwareAPI APINetwork  Devices  Network  Devices  Network  Devices  Network  Devices  Network  Devices  Network  Devices  ApplicationLayerControlLayerInfrastructureLayerControl Data Plane Interface (e.g. OpenFlow)Hitchhiker’s Guide to the Open Cloud by @mrhinkle21
  22. 22. Cloud Promise, Reality and NetworksCloud  Promise   Cloud  Reality  Centralized  ConfiguraHon  and  AutomaHon  Without  true  virtualiza3on,  network  devices  must  s3ll  be  manually  configured.  Instant  Self-­‐Service  Provisioning  In  a  physical  network,  it  could  take  a  long  3me  for  network  engineer  to  provision  new  services.  ElasHcity  and  Scalability   By  horizontally  scaling  up  the  physical  network,  elas3city  is  lost.  Designed  for  Failure   Failover  can  be  automated  and  physical  network  limita3ons  can  be  alleviated.    Source: MidokuraHitchhiker’s Guide to the Open Cloud by @mrhinkle22
  23. 23. Open FlowOpenFlow  enables  networks  to  evolve,  by  giving  a  remote  controller  the  power  to  modify  the  behavior  of  network  devices,  through  a  well-­‐defined  "forwarding  instruc3on  set".  The  growing  OpenFlow  ecosystem  now  includes  routers,  switches,  virtual  switches,  and  access  points  from  a  range  of  vendors.  Image from’s Guide to the Open Cloud by @mrhinkle23
  24. 24. Software Defined Networking (SDN)Project DescriptionFloodlight   The  Floodlight  controller  is  an  enterprise-­‐class,  Apache-­‐licensed,  Java-­‐based  OpenFlow  Controller.  Indigo   Indigo  is  an  open  source  project  to  support  OpenFlow  on  a  range  of  physical  switches.  By  leveraging  hardware  features  of  Ethernet  switch  ASICs,  Indigo  supports  high  rates  for  high  port  counts,  up  to  48  10-­‐gigabit  ports.  Mul3ple  gigabit  planorms  with  10-­‐gigabit  uplinks  are  also  supported.    Open  Daylight   Linux  Founda3on  Collabora3ve  Project  based  on  Cisco  One  Controller  and  plugins  from  numerous  vendors  in  development.    E.g  IBM  DOVE  OpenStack  “Quantum”    Networking  Pluggable,  scalable,  API-­‐driven  network  and  IP  management  Open  vSwitch   Open  vSwitch  is  a  open  source  (ASL  2.0),  mul3layer  virtual  switch  designed  to  enable  massive  network  automa3on  through  programma3c  extension,  while  s3ll  suppor3ng  standard  management  interfaces  and  protocols  (e.g.  NetFlow,  sFlow,  SPAN,  RSPAN,  CLI,  LACP,  802.1ag).  Hitchhiker’s Guide to the Open Cloud by @mrhinkle24
  25. 25. Big DataDeep Thought
  26. 26. 1 Billion Facebook Users - October 20120  200  400  600  800  1000  1200  Dec-­‐04  Mar-­‐05  Jun-­‐05  Sep-­‐05  Dec-­‐05  Mar-­‐06  Jun-­‐06  Sep-­‐06  Dec-­‐06  Mar-­‐07  Jun-­‐07  Sep-­‐07  Dec-­‐07  Mar-­‐08  Jun-­‐08  Sep-­‐08  Dec-­‐08  Mar-­‐09  Jun-­‐09  Sep-­‐09  Dec-­‐09  Mar-­‐10  Jun-­‐10  Sep-­‐10  Dec-­‐10  Mar-­‐11  Jun-­‐11  Sep-­‐11  Dec-­‐11  Mar-­‐12  Jun-­‐12  Sep-­‐12  FacebookUsersinMillionsSource: Benphoster.comHitchhiker’s Guide to the Open Cloud by @mrhinkle26
  27. 27. Data  is  growing  faster  than  storage  capacity  and  compu3ng  power.  Legacy  systems  hold  organiza3ons  back;  storage  sodware  must  include  mul3-­‐petabyte  capacity,  support  poten3ally  billions  of  objects,  and  provide  applica3on  performance  awareness  and  agile  provisioning.      -­‐Gartner,  Big  Data  Challenges  for  the  IT  Infrastructure  Team  Big Data and Storage InfrastructureHitchhiker’s Guide to the Open Cloud by @mrhinkle27
  28. 28. Open Source NoSQL DatabasesName Type DescriptionApache  Cassandra  Wide  Column  Store/Families  API:  many  »  Query  Method:  MapReduce,  Replicaton:  ,  Wriven  in:  Java,  Concurrency:  eventually  consistent  ,  Misc:  like  "Big-­‐Table  on  Amazon  Dynamo  alike",  ini3ated  by  Facebook  CouchDB   Document  Store   API:  Memcached  API+protocol  (binary  and  ASCII)  ,  most  languages,  Protocol:  Memcached  REST  interface  for  cluster  conf  +  management,  Wriven  in:  C/C++  +  Erlang  (clustering),  Replica3on:  Peer  to  Peer,  fully  consistent,  Misc:  Transparent  topology  changes  during  opera3on,  provides  memcached-­‐compa3ble  caching  buckets  HBase   Wide  Column  Store/Families    API:  Java  /  any  writer,  Protocol:  any  write  call,  Query  Method:  MapReduce  Java  /  any  exec,  Replica3on:  HDFS  Replica3on,  Wriven  in:  Java  Hypertable   Wide  Column  Store/Families    PI:  Thrid  (Java,  PHP,  Perl,  Python,  Ruby,  etc.),  Protocol:  Thrid,  Query  Method:  HQL,  na3ve  Thrid  API,  Replica3on:  HDFS  Replica3on,  Concurrency:  MVCC,  Consistency  Model:  Fully  consistent  Misc:  High  performance  C++  implementa3on  of  Googles  Bigtable.  MongoDB   Document  Store   API:  BSON,  Protocol:  C,  Query  Method:  dynamic  object-­‐based  language  &  MapReduce,  Replica3on:  Master  Slave  &  Auto-­‐Sharding,  Wriven  in:  C++,Concurrency  Redis   Key  Value/  Tuple  Store  API:  Tons  of  languages,  Wriven  in:  C,  Concurrency:  in  memory  and  saves  asynchronous  disk  ader  a  defined  3me.  Append  only  mode  available.  Different  kinds  of  fsync  policies.  Replica3on:  Master  /  Slave,  Misc:  also  lists,  sets,  sorted  sets,  hashes,  queues.    Riak     Key  Value  /  Tuple  Store  API:  JSON,  Protocol:  REST,  Query  Method:  MapReduce  term  matching  ,  Scaling:  Mul3ple  Masters;  Wriven  in:  Erlang,  Concurrency:  eventually  consistent  (stronger  then  MVCC  via  Vector  Clocks)  Hitchhiker’s Guide to the Open Cloud by @mrhinkle28
  29. 29. MapReduceProblem  Data  Master    Node  Worker  Node  1  Worker  Node  2  Worker  Node  3  Solu3on  Data  MapReduceHitchhiker’s Guide to the Open Cloud by @mrhinkle29
  30. 30. Apache HadoopOverview  •  Handles  large  amounts  of  data  •  Stores  data  in  na3ve  format  •  Delivers  linear  scalability  at  low  cost  •  Resilient  in  case  of  infrastructure  failures  •  Transparent  applica3on  scalability  Facts    •  Apache  top-­‐level  open  source  project  •  One  framework  for  storage  and  compute  –  HDFS  –  Scalable  storage  in  Hadoop  Distributed  File  System  (HDFS)  –  Compute  via  the  MapReduce  distributed  processing  planorm  •  Domain  Specific  Language  (DSL)  -­‐  Java  Hitchhiker’s Guide to the Open Cloud by @mrhinkle30
  31. 31. Hadoop ArchitectureHadoop  Common      HDFS  Distributes  &  replicates  data  across  machines  MapReduce  Distributes  &  monitors  tasks    Hive    Data  warehouse  that  provides  SQL  interface.  Ad  hoc  projec3on  of  data  structure  to  unstructured  MapReduce•  Parallel programming•  Handles large data blocksNon-Relational DBHBase    Column-­‐oriented  schema-­‐less  distributed  DB  modeled  ader  Google’s  BigTable  Random  real  3me  read/write.      ScriptingPig  Planorm  for  manipula3ng  and  analyzing  large  data  sets.  Scrip3ng  language  for  analysts.    Mahout    Machine  learning  libraries  for  recommenda3ons  ,  clustering,  classifica3ons  and  item  sets.    Machine LearningChuckwa  Zookeeper  ManagementHitchhiker’s Guide to the Open Cloud by @mrhinkle31
  32. 32. Big Data Summary•  Quan3ty  of  Machine  Created  Data  Increasing  Dras3cally  (examples:  networked  sensor  data  from  mobile  phones  and  GPS  devices)  •  Data  manipula3on  moving  from  batched  to  real-­‐3me  •  Cloud  services  giving  everyone  Big  Data  tools    •  Consumer  company  speed  and  scale  requirements  driving  efficiencies  in  Big  Data  storage  and  analy3cs  •  New  and  broader  number  of  data  sources  being  meshed  together  •  Big  Data  Apps  means  using  Big  Data  is  faster  and  easier    Hitchhiker’s Guide to the Open Cloud by @mrhinkle32
  33. 33. Cloud Management Tools
  34. 34. Automation in the CloudMeat Cloud Cloud OperationsHitchhiker’s Guide to the Open Cloud by @mrhinkle34
  35. 35. 4 Types of Management ToolsProvisioningInstallation of operating systems and other softwareConfiguration ManagementSets the parameters for servers, can specify installation parametersOrchestration/AutomationAutomate tasks across systemsMonitoringRecords errors and health of IT infrastructureHitchhiker’s Guide to the Open Cloud by @mrhinkle35
  36. 36. Management ToolchainsConfigura3on  Patching  and  Provisioning  Monitoring  Toolchain (n):A set of tools wherethe output of onetool becomes theinput of another toolHitchhiker’s Guide to the Open Cloud by @mrhinkle36
  37. 37. Conceptual Automated ToolchainBootStrapped  Image  CloudStack  OpenStack  ConfiguraHon  Puppet  Chef  Start/Stop  Services  RunDeck  Capistrano  MCollec3ve  Provision  Cobbler  SUSE  Stuido  Monitoring  Nagios  Zenoss    Cac3    Generate  Images  SUSE  Studio  BoxGrinder  Hitchhiker’s Guide to the Open Cloud by @mrhinkle37
  38. 38. NetFlix Open Source ToolBag for AWSASGARD ASTYANAX EDDAEUREKA PRIAM SIMIAN ARMY38Hitchhiker’s Guide to the Open Cloud by @mrhinkle
  39. 39. Hitchhiker’s Guide to the Open Cloud by @mrhinkle39
  40. 40. Goodbye and Thanks for All the Fish!Hitchhiker’s Guide to the Open Cloud by @mrhinkle40Slides  Can  be  Viewed  and  Downloaded  at:  hvp://  Copyright Mark R. Hinkle, availableunder the CCbySA license somerights reserved. 2012 -2013
  41. 41. Contact MeProfessional: mark.hinkle@citrix.comPersonal: mrhinkle@gmail.comPhone: 919.228.8049Personal: http://www.socializedsoftware.comTwitter: @mrhinkleMark R. HinkleSenior Director,Open Source SolutionsCitrix Systems Inc.Open Source EnthusiastHitchhiker’s Guide to the Open Cloud by @mrhinkle41
  42. 42. Version 1.1 - ChangelogThis  presenta3on  is  constantly  updated  to  reflect  the  changes  in  the  open  source  cloud  compu3ng  landscape.  Hence  the  informa3on,  links  and  speakers  notes  are  always  being  updated.    •  Moved  Big  Data  Landscape  to  Appendix    •  Dropped  Twiver  growth  chart  •  Moved  Provisioning  and  Orchestra3on  tools  to  Appendix    •  Removed  qualifiers  from  PaaS,  Added  Gigaspaces  Cloudify    42Hitchhiker’s Guide to the Open Cloud by @mrhinkle
  43. 43. Appendix(A.K.A Great Stuff I didn’t say)
  44. 44. Additional Resources•  Devops  Toolchains  Group  •  Sodware  Defined  Networking:    The  New  Norm  for  Networks  (Whitepaper)  •  DevOps  Wikipedia  Page  •  NoSQL-­‐  –  Ul3mate  Guide  to  the  Non-­‐Rela3onal  Universe  •  Open  Cloud  Ini3a3ve  •  NIST  Cloud  Compu3ng  Planorm  •  Open  Virtualiza3on  Format  Specs  •  Cloudera3  Twiver  Account  •  Planet  DevOps  •  Nicira  Whitepaper  –  It’s  Time  to  Virtualize  the  Network  •  Why  Open  vSwitch  FAQ  Hitchhiker’s Guide to the Open Cloud by @mrhinkle44
  45. 45. Big Data LandscapeSource:BigDataGroup.comHitchhiker’s Guide to the Open Cloud by @mrhinkle45
  46. 46. Monitoring ToolsLicense   Type  of  Monitoring   CollecHon  Methods  Cac3    /  RRDTool   GPL     Performance   SNMP,  syslog  Graphite   Apache  2.0   Performance   Agent  Nagios   GPL   Availability   SNMP,TCP,  ICMP,  IPMI,  syslog  Zabbix   GPL     Availability/  Performance  and  more  SNMP,  TCP/ICMP,  IPMI,  Synthe3c  Transac3ons  Zenoss   GPL   Availability,  Performance,  Event  Management  SNMP,  ICMP,  SSH,  syslog,  WMI  Hitchhiker’s Guide to the Open Cloud by @mrhinkle46
  47. 47. ProvisioningProject Installation TargetsApache  Provisionr(incuba3ng)  Can  provision  10s  to  1000s  of  machines  on  various  clouds.    Cobbler   Distributed  virtual  infrastructure  using  koan  (kickstart  of  a  network  to  PXE  boot  VMs)  for  Red  Hat,  OpenSUSE  Fedora,  Debian,  Ubuntu  VMs  Crowbar    (Bare  metal  provisioning)  JuJu   Public  Clouds  -­‐    Amazon  Web  Services  HP  Cloud,    Private  OpenStack  clouds,  Bare  Metal  via  MAAS.    Salt  Cloud     Tool  to  provision  “salted”  VMs  that  can  then  be  updated  by  a  central  server  via  ZeroMQ  Hitchhiker’s Guide to the Open Cloud by @mrhinkle47
  48. 48. Configuration Management ToolsProject   Year  Started   Language   License   Client/Server  Cfengine   1993   C   Apache   Yes  Chef   2009   Ruby   Apache   Chef  Solo  –  No    Chef  Server  -­‐  Yes  Puppet   2004   Ruby   GPL     Yes  &  standalone  Salt   2011   Python   Apache   yes  Hitchhiker’s Guide to the Open Cloud by @mrhinkle48
  49. 49. Automation/Orchestration ToolsProject   DescripHon  Ansible   Ansibles  SSH-­‐key  based  access  allows  contributors  to  the  Fedora  Project  to  assist  in  automa3ng  infrastructure  while  having  access  limited  appropriately.    Capistrano   U3lity  and  framework  for  execu3ng  commands  in  parallel  on  mul3ple  remote  machines,  via  SSH.  It  uses  a  simple  DSL  that  allows  you  to  define  tasks,  which  may  be  applied  to  machines  in  certain  roles  RunDeck   Rundeck  is  an  open-­‐source  process  automa3on  and  command  orchestra3on  tool  with  a  web  console.  Func   Func  provides  a  two-­‐way  authen3cated  system  for  generically  execu3ng  tasks,  integra3ons  with  puppet  and  cobbler.  MCollec3ve   The  Marioneve  Collec3ve  AKA  MCollec3ve  is  a  framework  to  build  server  orchestra3on  or  parallel  job  execu3on  systems.  Salt   Execute  arbitrary  shell  commands  or  choose  from  dozens  of  pre-­‐built  modules  of  common  (or  complex)  commands.  Scalr   Provide  scaling  across  mul3ple  cloud  compu3ng  planorms,  integrates  with  Chef.    Hitchhiker’s Guide to the Open Cloud by @mrhinkle49