This document discusses using Puppet and infrastructure as code to manage Apache CloudStack infrastructure. It introduces the cloudstack_resources Puppet module which allows defining CloudStack instances and entire application stacks in Puppet manifests. This enables treating infrastructure like code where Puppet can deploy and configure entire environments on CloudStack. Examples are given of classifying servers and deploying a Hadoop cluster with a single Puppet resource definition. Links are provided to resources for using Puppet with CloudStack and videos that further explain the concepts.

1. Infrastructure as code
with Puppet and
Apache CloudStack
David Nalley <david@gnsa.us> @ke4qqq

2. #whoami
• Recovering sysadmin
• Apache CloudStack Committer
• Designer of ugly slides

3. To set the stage...
• Apache CloudStack is...
• an open source IaaS platform
• proven in production at massive scale
• awesome

4. Gorgeous UI

5. Decent API
• EC2 or native
• http://incubator.apache.org/cloudstack/docs/api

6. So IaaS removes one
constraint....
No longer waiting days/weeks to provision a
machine.

7. ...but introduces another
constraint..
Now have to get a machine configured in a
timely manner.

8. People provision stuff
Often not ops folks
Often not intimately familiar with intricacies

9. Baseline is important; but...

10. Classification
Problem: We spin up, dynamically, 1-500 VMs at
any given time - how do we decide what
configurations apply.

11. Classification
The wrong way - dedicated images for each
purpose

12. Classification
editing nodes.pp
node 'foo-356.cloud.com' {
include httpd
}

13. Classification
globbing
node 'foo*' {
include httpd
}

14. Classification
Everything is default
node 'default' {
include httpd
}

15. Classification
External node classifier

16. Classification
Facts
class base {
case $::fact {
'httpd': {
include httpd
}
'otherrole': {
include nginx
}
}
}

17. One solution
During instance provisioning define metadata.
Custom fact for that metadata
Case statement based on that fact

18. Example metadata
role=webserver
location=datacenter1
environment=production

19. Corresponding
class base {
manifest
case $::fact {
'webserver': {
include httpd
}
'database': {
include postgresql
}
}
}

20. Links, et al
Fact:
http://s.apache.org/acs_userdata
Blog with details:
http://s.apache.org/acs_userdata2

21. Video is here, go watch it
• I only have 45 minutes - so can't delve into
everything, you should watch the video- it’s
great.
• http://youtu.be/c8YWctfOpwo

22. And then there was a
knife.....plugin
• So the folks at Edmunds.com wrote a knife
plugin for CloudStack.
• The knife plugin had the ability to define an
application stack, potentially hundreds of
nodes, that are interrelated, and provision
them with a single knife command.
• https://github.com/cloudstack-extras/knife-cloudstack

23. Deploying a machine with knife
knife cs server create

24. "name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
{
"name": "zookeeper-a, zookeeper-b, zookeeper-c",
"description": "Zookeeper nodes",
"template": "rhel-5.6-base",
"service": "small",
"port_rules": "2181",
"run_list": "role[cluster_a], role[zookeeper_server]",
"actions": [
{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
]
},
{
"name": "hadoop-master",
"description": "Hadoop master node",
"template": "rhel-5.6-base",
"service": "large",
"networks": "app-net, storage-net",
"port_rules": "50070, 50030, 60010",
"run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
},
{
"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
"description": "Hadoop worker nodes",
"template": "rhel-5.6-base",
"service": "medium",
"port_rules": "50075, 50060, 60030",
"run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
"actions": [
{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
{ "http_request": "http://${hadoop-master}:50070/index.jsp" }
]
}

25. Deploy that with...
knife cs stack create hadoop_cluster_a

26. I was jealous...

27. Then at FOSDEM 2012
• CloudStack user shows me Puppet types
and resources for OpenNebula.
• https://puppetlabs.com/blog/puppetizing-opennebula/
• They indicated they wanted this
awesomeness for CloudStack....

28. Why???
• They wanted to define each of their application
stacks in puppet, so that not only the
configuration of software on the machine, but
the machines themselves would be configured
by Puppet.
• Automated deployment of test environments
that are exactly the same
• Really gets outside of machine configuration to
entire infrastructure configuration

29. What we have grown used to
Puppet _defines_ the configuration within the
machine

30. What we want...

31. What we want...
Puppet _defines_ the machine

32. What we want...
Puppet _defines_ a collection of machines

33. What we want...
Puppet _defines_ ALL the machines

34. This even has its own buzzword
Software defined datacenter

35. ...and then at PuppetConf
• There was Google Compute Engine types
and resources for Puppet.
• Dan Bode gave a presentation showing off
the work he had done... that presentation is
worth seeing...
• http://www.slideshare.net/bodepd/google-compute-presentation-puppet-conf

37. So then for Christmas...
• puppet types and providers arrived -
courtesy of Dan Bode
• https://github.com/bodepd/cloudstack_resources

38. So how does this work
cloudstack_instance { 'foo1':
ensure => present,
flavor => 'Small Instance',
zone => 'FMT-ACS-001',
image => 'CentOS 5.6(64-bit) no GUI (XenServer)',
network => 'puppetlabs-network',
# domain
# account
# hostname
}

39. Setting defaults
Cloudstack_instance {
image => 'CentOS 6.3',
flavor => 'M1.medium',
zone => 'San Jose',
network => 'davids_net',
keypair => 'david_keys',
}
cloudstack_instance {
ensure => $::ensure,
group => 'role=db',
}

40. A simple stack
class my_web_stack {
cloudstack_instance { 'foo4':
ensure => present,
group => 'role=apache',
}
cloudstack_instance { 'foo5':
ensure => present,
group => 'role=db',
}
}

41. Define all your
infrastructure

42. Resources/contact
Me: David Nalley <david@gnsa.us> @ke4qqq
CloudStack: http://cloudstack.org
cloudstack-users@incubator.apache.org
cloudstack_resources:
https://github.com/bodepd/cloudstack_resources
Jason Hancock Vids
http://youtu.be/c8YWctfOpwo
http://youtu.be/8W0BqCmNZQQ