Published on

"Puppet and Apache CloudStack" by David Nalley of Apache Cloudstack at Puppet Camp Boston 2013.

Published in: Technology
  • Be the first to comment

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide


  1. 1. Infrastructure as code withPuppet and Apache CloudStackDavid Nalleyke4qqq@apache.org@ke4qqq
  2. 2. #whoami• Apache Software Foundation Member• Apache CloudStack PMC Member• Recovering Sysadmin• Fedora Project Contributor• Zenoss contributor• Employed by Citrix in the Open Source Business Office
  3. 3. Setting the stageApache CloudStack is...● an open source IaaS platform● proven in production at massive scale● awesome
  4. 4. Gorgeous UI
  5. 5. API● Native: http://cloudstack.apache.org/docs/api● EC2
  6. 6. IaaS removes one constraintNo longer waiting days or weeks to get a VM provisioned
  7. 7. but introduces another...Now you have to get a machine configured in a timelymanner.
  8. 8. Self service● UI● API● Some external tool
  9. 9. People provision stuff...Not ops folksOften not familiar with environmental intricaciesDont care
  10. 10. Baseline can be important....
  11. 11. ClassificationProblem: We spin up, dynamically, 1-500 VMs at any given time - how dowe decide what configurations apply.
  12. 12. ClassificationThe wrong way - dedicated images for each purpose
  13. 13. Classificationediting nodes.ppnode foo-356.cloud.com {include httpd}
  14. 14. Classificationglobbingnode mysql* {include mysqld}
  15. 15. ClassificationEverything is defaultnode default {include httpd}
  16. 16. ClassificationExternal Node Classifier
  17. 17. ClassificationExternal Node Classifier
  18. 18. ClassificationFactsclass base {case $::fact {httpd: {include httpd}otherrole: {include nginx}}}
  19. 19. Classification - One Solution● During instance provisioning define metadata.● Custom fact for that metadata● Case statement based on that fact
  20. 20. Example Metadatarole=webserverlocation=datacenter1environment=production
  21. 21. Corresponding manifestclass base {case $::fact {webserver: {include httpd}database: {include postgresql}}}
  22. 22. Corresponding manifestclass base {case $::fact {webserver: {include httpd}database: {include postgresql}}}
  23. 23. Links, et al.● Fact:http://s.apache.org/acs_userdata● Blog with details:http://s.apache.org/acs_userdata2
  24. 24. Video - go watch it● I only have 45 minutes - so cant delveinto everything, you should watch thevideo- it’s great.● http://youtu.be/c8YWctfOpwo
  25. 25. Video - go watch it● I only have 45 minutes - so cant delveinto everything, you should watch thevideo- it’s great.● http://youtu.be/c8YWctfOpwo
  26. 26. And then there was a knife-pluginThe folks at Edmunds.com wrote a knife plugin forCloudStackThe knife plugin had the ability to define an applicationstack, potentially hundreds of nodes, that are interrelated,and provision them with a single knife command.https://github.com/cloudstack-extras/knife-cloudstack
  27. 27. Deploying a machine with knife~ knife cs server create
  28. 28. {"name": "hadoop_cluster_a","description": "A small hadoop cluster with hbase","version": "1.0","environment": "production","servers": [{"name": "zookeeper-a, zookeeper-b, zookeeper-c","description": "Zookeeper nodes","template": "rhel-5.6-base","service": "small","port_rules": "2181","run_list": "role[cluster_a], role[zookeeper_server]","actions": [{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }]},{"name": "hadoop-master","description": "Hadoop master node","template": "rhel-5.6-base","service": "large","networks": "app-net, storage-net","port_rules": "50070, 50030, 60010","run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"},{"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c","description": "Hadoop worker nodes","template": "rhel-5.6-base","service": "medium","port_rules": "50075, 50060, 60030","run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]","actions": [{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },{ "http_request": "http://${hadoop-master}:50070/index.jsp" }]}}
  29. 29. Deploy that Hadoop cluster withknife cs stack create hadoop_cluster_a
  30. 30. I was jealous....
  31. 31. Then at FOSDEM 2012● CloudStack user shows me Puppet types and resourcesfor OpenNebula.● https://puppetlabs.com/blog/puppetizing-opennebula/● They indicated they wanted this awesomeness forCloudStack....
  32. 32. Why?● They wanted to define each of their applicationstacks in puppet, so that not only the configurationof software on the machine, but the machinesthemselves would be configured by Puppet.● Automated deployment of test environments thatare exactly the same● Really gets outside of machine configuration toentire infrastructure configuration
  33. 33. What we are used to● Puppet _defines_ the configurationwithin the machine
  34. 34. What we want● Puppet _defines_ the machine.● Puppet _defines_ collection ofmachines● Puppet _defines_ the machines,networks, and rest of infrastructure
  35. 35. Then at Puppetconf● There was Google ComputeEngine types and resources forPuppet.● Dan Bode gave a presentation showing off the work hehad done... that presentation is worth seeing...● http://www.slideshare.net/bodepd/google-compute-presentati
  36. 36. And then for Christmas● puppet types and providers arrived - courtesy of DanBode● https://github.com/bodepd/cloudstack_resources
  37. 37. How does this work?cloudstack_instance { foo1:ensure => present,flavor => Small Instance,zone => FMT-ACS-001,image => CentOS 5.6(64-bit) no GUI(XenServer),network => puppetlabs-network,# domain# account# hostname}●
  38. 38. Setting defaultsCloudstack_instance {image => CentOS 6.3,flavor => M1.medium,zone => San Jose,network => davids_net,keypair => david_keys,}cloudstack_instance {ensure => $::ensure,group => role=db,}
  39. 39. A simple stackclass my_web_stack {cloudstack_instance { foo4:ensure => present,group => role=apache,}cloudstack_instance { foo5:ensure => present,group => role=db,}}
  40. 40. Questions
  41. 41. Contact● Project– http://cloudstack.apache.org– #cloudstack on irc.freenode.net● Me– ke4qqq on irc.freenode.net– ke4qqq@apache.org