Infrastructure as code with Puppet and Apache CloudStack

  • 270 views
Uploaded on

 

More in: Technology
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Be the first to comment
No Downloads

Views

Total Views
270
On Slideshare
0
From Embeds
0
Number of Embeds
0

Actions

Shares
Downloads
7
Comments
0
Likes
1

Embeds 0

No embeds

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
    No notes for slide

Transcript

  • 1. Infrastructure as code with Puppet and Apache CloudStackDavid Nalley <ke4qqq@apache.org> @ke4qqq
  • 2. #whoami• Recovering sysadmin• Apache CloudStack Committer• Designer of ugly slides
  • 3. To set the stage...• Apache CloudStack is... • an open source IaaS platform • proven in production at massive scale • awesome
  • 4. Gorgeous UI
  • 5. Decent API• EC2 or native• http://cloudstack.apache.org/docs/api
  • 6. So IaaS removes one constraint....No longer waiting days/weeks to provision a machine.
  • 7. ...but introduces another constraint..Now have to get a machine configured in a timely manner.
  • 8. People provision stuffOften not ops folksOften not intimately familiar with intricacies
  • 9. Baseline is important; but...
  • 10. ClassificationProblem: We spin up, dynamically, 1-500 VMs at any given time - how do we decide what configurations apply.
  • 11. ClassificationThe wrong way - dedicated images for each purpose
  • 12. Classificationediting nodes.ppnode foo-356.cloud.com { include httpd}
  • 13. Classificationglobbingnode foo* { include httpd}
  • 14. ClassificationEverything is defaultnode default { include httpd}
  • 15. ClassificationExternal node classifier
  • 16. ClassificationFactsclass base { case $::fact { httpd: { include httpd } otherrole: { include nginx } } }
  • 17. One solutionDuring instance provisioning define metadata.Custom fact for that metadataCase statement based on that fact
  • 18. Example metadatarole=webserverlocation=datacenter1environment=production
  • 19. Correspondingclass base { manifestcase $::fact { webserver: { include httpd } database: { include postgresql } }}
  • 20. Links, et alFact: http://s.apache.org/acs_userdataBlog with details: http://s.apache.org/acs_userdata2
  • 21. Video is here, go watch it• I only have 45 minutes - so cant delve into everything, you should watch the video- it’s great.• http://youtu.be/c8YWctfOpwo
  • 22. And then there was a knife.....plugin• So the folks at Edmunds.com wrote a knife plugin for CloudStack.• The knife plugin had the ability to define an application stack, potentially hundreds of nodes, that are interrelated, and provision them with a single knife command.• https://github.com/cloudstack-extras/knife-cloudstack
  • 23. Deploying a machine with knifeknife cs server create
  • 24. "name": "hadoop_cluster_a","description": "A small hadoop cluster with hbase","version": "1.0","environment": "production","servers": [ { "name": "zookeeper-a, zookeeper-b, zookeeper-c", "description": "Zookeeper nodes", "template": "rhel-5.6-base", "service": "small", "port_rules": "2181", "run_list": "role[cluster_a], role[zookeeper_server]", "actions": [ { "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] } ] }, { "name": "hadoop-master", "description": "Hadoop master node", "template": "rhel-5.6-base", "service": "large", "networks": "app-net, storage-net", "port_rules": "50070, 50030, 60010", "run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]" }, { "name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c", "description": "Hadoop worker nodes", "template": "rhel-5.6-base", "service": "medium", "port_rules": "50075, 50060, 60030", "run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]", "actions": [ { "knife_ssh": ["role:hadoop_master", "sudo chef-client"] }, { "http_request": "http://${hadoop-master}:50070/index.jsp" } ] }
  • 25. Deploy that with...knife cs stack create hadoop_cluster_a
  • 26. I was jealous...
  • 27. Then at FOSDEM 2012• CloudStack user shows me Puppet types and resources for OpenNebula.• https://puppetlabs.com/blog/puppetizing-opennebula/• They indicated they wanted this awesomeness for CloudStack....
  • 28. Why???• They wanted to define each of their application stacks in puppet, so that not only the configuration of software on the machine, but the machines themselves would be configured by Puppet.• Automated deployment of test environments that are exactly the same• Really gets outside of machine configuration to entire infrastructure configuration
  • 29. What we have grown used toPuppet _defines_ the configuration within the machine
  • 30. What we want...
  • 31. What we want...Puppet _defines_ the machine
  • 32. What we want...Puppet _defines_ a collection of machines
  • 33. What we want...Puppet _defines_ ALL the machines
  • 34. ...and then at PuppetConf• There was Google Compute Engine types and resources for Puppet.• Dan Bode gave a presentation showing off the work he had done... that presentation is worth seeing...• http://www.slideshare.net/bodepd/google-compute-presentation-puppet-conf
  • 35. So then for Christmas...• puppet types and providers arrived - courtesy of Dan Bode• https://github.com/bodepd/cloudstack_resources
  • 36. So how does this workcloudstack_instance { foo1: ensure => present, flavor => Small Instance, zone => FMT-ACS-001, image => CentOS 5.6(64-bit) no GUI (XenServer), network => puppetlabs-network, # domain # account # hostname }
  • 37. Setting defaultsCloudstack_instance { image => CentOS 6.3, flavor => M1.medium, zone => San Jose, network => davids_net, keypair => david_keys, } cloudstack_instance { baz: ensure => $::ensure, group => role=db, }
  • 38. A simple stackclass my_web_stack { cloudstack_instance { foo4: ensure => present, group => role=apache, } cloudstack_instance { foo5: ensure => present, group => role=db, } }
  • 39. Define all yourinfrastructure
  • 40. Resources/contactMe: David Nalley <ke4qqq@apache.org> @ke4qqqCloudStack: http://cloudstack.apache.org users@cloudstack.apache.orgcloudstack_resources: https://github.com/bodepd/cloudstack_resourcesJason Hancock Vids http://youtu.be/c8YWctfOpwo http://youtu.be/8W0BqCmNZQQ