This document discusses using Puppet to manage infrastructure as code with Apache CloudStack. It describes how Puppet types and providers were developed to allow defining CloudStack instances and entire application stacks in Puppet manifests. This enables automated deployment and configuration of infrastructure along with software configuration. Examples are given of using Puppet to define CloudStack instances, groups of instances that make up an application stack, and setting defaults for attributes. Resources mentioned include the CloudStack and Puppet GitHub pages.

1. Infrastructure as code
with Puppet and
Apache CloudStack
David Nalley <david@gnsa.us> @ke4qqq

2. #whoami
• Recovering sysadmin
• Apache CloudStack Committer
• Designer of ugly slides

3. To set the stage...
• Apache CloudStack is...
• an open source IaaS platform
• proven in production at massive scale
• awesome

4. Gorgeous UI

5. Decent API
• EC2 or native
• http://incubator.apache.org/cloudstack/docs/api

6. So IaaS has several problems
for puppet

7. People provision stuff
Often not ops folks
Often not intimately familiar with intricacies

8. Baseline is important; but...

9. Classification
Problem: We spin up, dynamically, 1-500 VMs at
any given time - how do we decide what
configurations apply.

10. Classification
The wrong way - dedicated images for each
purpose

11. Classification
editing nodes.pp
node 'foo-356.cloud.com' {
include httpd
}

12. Classification
globbing
node 'foo*' {
include httpd
}

13. Classification
Everything is default
node 'default' {
include httpd
}

14. Classification
External node classifier

15. Classification
Facts
class base {
case $::fact {
'httpd': {
include httpd
}
'otherrole': {
include nginx
}
}
}

16. One solution
During instance provisioning define metadata.
Custom fact for that metadata
Case statement based on that fact

17. Example metadata
role=webserver
location=datacenter1
environment=production

18. Corresponding
class base {
manifest
case $::fact {
'webserver': {
include httpd
}
'database': {
include postgresql
}
}
}

19. Links, et al
Fact:
http://s.apache.org/acs_userdata
Blog with details:
http://s.apache.org/acs_userdata2

20. Video is here, go watch it
• I only have 45 minutes - so can't delve into
everything, you should watch the video- it’s
great.
• http://youtu.be/c8YWctfOpwo

21. And then there was a
knife.....plugin
• So the folks at Edmunds.com wrote a knife
plugin for CloudStack.
• The knife plugin had the ability to define an
application stack, potentially hundreds of
nodes, that are interrelated, and provision
them with a single knife command.
• https://github.com/cloudstack-extras/knife-cloudstack

22. Deploying a machine with knife
knife cs server create

23. "name": "hadoop_cluster_a",
"description": "A small hadoop cluster with hbase",
"version": "1.0",
"environment": "production",
"servers": [
{
"name": "zookeeper-a, zookeeper-b, zookeeper-c",
"description": "Zookeeper nodes",
"template": "rhel-5.6-base",
"service": "small",
"port_rules": "2181",
"run_list": "role[cluster_a], role[zookeeper_server]",
"actions": [
{ "knife_ssh": ["role:zookeeper_server", "sudo chef-client"] }
]
},
{
"name": "hadoop-master",
"description": "Hadoop master node",
"template": "rhel-5.6-base",
"service": "large",
"networks": "app-net, storage-net",
"port_rules": "50070, 50030, 60010",
"run_list": "role[cluster_a], role[hadoop_master], role[hbase_master]"
},
{
"name": "hadoop-worker-a hadoop-worker-b hadoop-worker-c",
"description": "Hadoop worker nodes",
"template": "rhel-5.6-base",
"service": "medium",
"port_rules": "50075, 50060, 60030",
"run_list": "role[cluster_a], role[hadoop_worker], role[hbase_regionserver]",
"actions": [
{ "knife_ssh": ["role:hadoop_master", "sudo chef-client"] },
{ "http_request": "http://${hadoop-master}:50070/index.jsp" }
]
}

24. Deploy that with...
knife cs stack create hadoop_cluster_a

25. I was jealous...

26. Then at FOSDEM 2012
• CloudStack user shows me Puppet types
and resources for OpenNebula.
• https://puppetlabs.com/blog/puppetizing-opennebula/
• They indicated they wanted this
awesomeness for CloudStack....

27. Why???
• They wanted to define each of their application
stacks in puppet, so that not only the
configuration of software on the machine, but
the machines themselves would be configured
by Puppet.
• Automated deployment of test environments
that are exactly the same
• Really gets outside of machine configuration to
entire infrastructure configuration

28. ...and then at
PuppetConf
• There was Google Compute Engine types
and resources for Puppet.
• Dan Bode gave a presentation showing off
the work he had done... that presentation is
worth seeing...
• http://www.slideshare.net/bodepd/google-compute-presentation-puppet-conf

30. So then for Christmas...
• puppet types and providers arrived -
courtesy of Dan Bode
• https://github.com/bodepd/cloudstack_resources

31. So how does this work
cloudstack_instance { 'foo1':
ensure => present,
flavor => 'Small Instance',
zone => 'FMT-ACS-001',
image => 'CentOS 5.6(64-bit) no GUI (XenServer)',
network => 'puppetlabs-network',
# domain
# account
# hostname
}

32. A simple stack
class my_web_stack {
cloudstack_instance { 'foo4':
ensure => present,
group => 'role=apache',
}
cloudstack_instance { 'foo5':
ensure => present,
group => 'role=db',
}
}

33. Setting defaults
Cloudstack_instance {
image => 'CentOS 6.3',
flavor => 'M1.medium',
zone => 'San Jose',
network => 'davids_net',
keypair => 'david_keys',
}
cloudstack_instance {
ensure => $::ensure,
group => 'role=db',
}

34. Define all your
infrastructure

35. Resources/contact
Me: David Nalley <david@gnsa.us> @ke4qqq
CloudStack: http://cloudstack.org
cloudstack-users@incubator.apache.org
cloudstack_resources:
https://github.com/bodepd/cloudstack_resources
Jason Hancock Vids
http://youtu.be/c8YWctfOpwo
http://youtu.be/8W0BqCmNZQQ