Puppet and AWS                           Getting The Best of Both Worlds                       Mike Ryan - Epitech.nl - mi...
• What is AWS?                       • How does Puppet work in AWS?                       • CloudFormation                ...
Hello, I’m Mike                       • Londoner in Amsterdam                       • Sysadmin with a passion for automati...
Amazon Web Services                       • EC2                       • Amazon Machine Images                       • User...
EC2 - Auto ScalingSunday, April 14, 13
CloudFormation{           "AWSTemplateFormatVersion" : "2010-09-09",           "Description" : "EC2 instance",           "...
"Enabled" : "true",                   "Logging" : {                     "Bucket" : "webapplication.s3.amazonaws.com",     ...
Bootstrapping an                           Infrastructure                       www.infrastructures.orgSunday, April 14, 13
CloudFormation or PuppetSunday, April 14, 13
Files, Services and Packages with cfn-init"Metadata": { "AWS::CloudFormation::Init": {  "config": {   "sources" : {    "/e...
Run a script at launch time with User Data{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "EC2 instance", "Re...
cfn-create-stack               $ cfn-create-stack puppetmaster                --template-file=puppetmaster.jsonhttp://aws....
CloudFormation +                           Puppet +                        Deploy scripts =                             Pa...
Autoscale = autosignSunday, April 14, 13
The Hostname Issuemike@ip-10-32-34-116:~$ hostname -fip-10-32-34-116.eu-west-1.compute.internalSunday, April 14, 13
User Data - Web ConsoleSunday, April 14, 13
User Data in CloudFormation          },          "UserData": {             "Fn::Base64": {               "Fn::Join": [    ...
include stdlib         node default {           $userdata = parsejson($ec2_userdata)           $role = userdata[‘role’]   ...
Vagrant -> EC2Sunday, April 14, 13
Vagrant -> EC2 WorkflowSunday, April 14, 13
User Data and Vagrant Vagrant::Config.run do |config|   config.vm.provision :puppet do |puppet|       puppet.manifests_pat...
DecouplingSunday, April 14, 13
Broken Puppet master                          = no autoscalingSunday, April 14, 13
/etc/rc.local  #!/bin/bash  /usr/local/bin/update.sh  /usr/bin/puppet apply site.ppSunday, April 14, 13
But you lose...                       •   Puppet Dashboard                       •   Exported resources                   ...
Role-based Puppet runsSunday, April 14, 13
EC2 TagsSunday, April 14, 13
env.roledefs = {      web: [www1, www2, www3],      db: [db1, db2]  }  @roles(db)  def restart_db():      sudo(/etc/init.d...
env.roledefs = {      web: [www1, www2, www3],      db: [db1, db1]  }  @roles(db, ‘web’)  def run_puppet():      sudo(pupp...
env.roledefs = configure_roles()  @roles(db, ‘web’)  def run_puppet():      sudo(puppet apply site.pp)  $fab run_puppet #r...
def configure_roles():  tags = EC2TagManager(AWS_KEY,    AWS_SECRET,    regions=[eu-west-1])  roles = {}    for role in [‘...
Questions?                       Mike Ryan - Epitech.nl - mike@epitech.nlSunday, April 14, 13
Upcoming SlideShare
Loading in …5
×

Puppet and AWS: Getting the best of both worlds

10,285 views

Published on

"Puppet and AWS: Getting the best of both worlds" by Mike Ryan of Epitech.nl at Puppet Camp Amsterdam 2013.

Published in: Technology, Business

Puppet and AWS: Getting the best of both worlds

  1. 1. Puppet and AWS Getting The Best of Both Worlds Mike Ryan - Epitech.nl - mike@epitech.nlSunday, April 14, 13
  2. 2. • What is AWS? • How does Puppet work in AWS? • CloudFormation • PaaS replacement? • Vagrant -> EC2 • Using Puppet to build AMIs • Some workarounds to common problems • QuestionsSunday, April 14, 13
  3. 3. Hello, I’m Mike • Londoner in Amsterdam • Sysadmin with a passion for automation • Epitech.nl - sysadmin as a serviceSunday, April 14, 13
  4. 4. Amazon Web Services • EC2 • Amazon Machine Images • User DataSunday, April 14, 13
  5. 5. EC2 - Auto ScalingSunday, April 14, 13
  6. 6. CloudFormation{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "EC2 instance", "Resources" : { "MyEC2Instance" : { "Type" : "AWS::EC2::Instance", "Properties" : { "ImageId" : "ami-79fd7eee", "KeyName" : "my-ssh-key", } } }}Sunday, April 14, 13
  7. 7. "Enabled" : "true", "Logging" : { "Bucket" : "webapplication.s3.amazonaws.com", "Prefix" : "webapp-logging/" } } } }, "asgMyAutoScalingGroup": { "Type": "AWS::AutoScaling::AutoScalingGroup", "Properties": { "AvailabilityZones": [ "us-east-1b", "us-east-1c" ], "Cooldown": "300", "DesiredCapacity": "1", "MaxSize": "1", "MinSize": "1", "LaunchConfigurationName": { "Ref": "lcMyLC" }, "LoadBalancerNames": [ { "Ref": "elbMyLB" } ] } }, "s3webapplication": { "Type": "AWS::S3::Bucket" }, "sgwebappsecuritygroup": { "Type": "AWS::EC2::SecurityGroup", "Properties": { "GroupDescription": "for web app", "SecurityGroupIngress": [ {Sunday, April 14, 13
  8. 8. Bootstrapping an Infrastructure www.infrastructures.orgSunday, April 14, 13
  9. 9. CloudFormation or PuppetSunday, April 14, 13
  10. 10. Files, Services and Packages with cfn-init"Metadata": { "AWS::CloudFormation::Init": { "config": { "sources" : { "/etc/puppet" : "http://example.com/puppet.tar.gz" }, "packages": { "yum": { "puppet-server": [], }, }, "services": { "sysvinit": { "puppetmaster": { "ensureRunning": "true", "enabled": "true" } }Sunday, April 14, 13
  11. 11. Run a script at launch time with User Data{ "AWSTemplateFormatVersion" : "2010-09-09", "Description" : "EC2 instance", "Resources" : { "PuppetMasterInstance": { "Type": "AWS::EC2::Instance", "Metadata": { }, "Properties": { "UserData": { "Fn::Base64": { "Fn::Join": [ "", [ "#!/bin/bashn", "/opt/aws/bin/cfn-init --region ", "n", "/usr/bin/puppet apply site.pp, "n" ] ] } } }Sunday, April 14, 13
  12. 12. cfn-create-stack $ cfn-create-stack puppetmaster --template-file=puppetmaster.jsonhttp://aws.amazon.com/developertools/2555753788650372 http://aws.amazon.com/cli/Sunday, April 14, 13
  13. 13. CloudFormation + Puppet + Deploy scripts = PaaSSunday, April 14, 13
  14. 14. Autoscale = autosignSunday, April 14, 13
  15. 15. The Hostname Issuemike@ip-10-32-34-116:~$ hostname -fip-10-32-34-116.eu-west-1.compute.internalSunday, April 14, 13
  16. 16. User Data - Web ConsoleSunday, April 14, 13
  17. 17. User Data in CloudFormation }, "UserData": { "Fn::Base64": { "Fn::Join": [ "", [ “{”role”: ”web”, ”, “”env”: ”staging”} ” ] ] } } {“role”: “web”, “env”: “staging”}Sunday, April 14, 13
  18. 18. include stdlib node default {   $userdata = parsejson($ec2_userdata) $role = userdata[‘role’] $environment = userdata[‘env’] case $role { ‘web’: { include nginx } ‘db’: { include postgresql } } }Sunday, April 14, 13
  19. 19. Vagrant -> EC2Sunday, April 14, 13
  20. 20. Vagrant -> EC2 WorkflowSunday, April 14, 13
  21. 21. User Data and Vagrant Vagrant::Config.run do |config| config.vm.provision :puppet do |puppet| puppet.manifests_path = "../puppet/manifests" puppet.module_path = "../puppet/modules" puppet.manifest_file = "site.pp" puppet.options = "--verbose --debug" puppet.facter = { :ec2_userdata => { :role => "database", :env => "local_dev", }.to_json, :vagrant => "true" } end endSunday, April 14, 13
  22. 22. DecouplingSunday, April 14, 13
  23. 23. Broken Puppet master = no autoscalingSunday, April 14, 13
  24. 24. /etc/rc.local #!/bin/bash /usr/local/bin/update.sh /usr/bin/puppet apply site.ppSunday, April 14, 13
  25. 25. But you lose... • Puppet Dashboard • Exported resources • Stored configs • Shared modulesSunday, April 14, 13
  26. 26. Role-based Puppet runsSunday, April 14, 13
  27. 27. EC2 TagsSunday, April 14, 13
  28. 28. env.roledefs = { web: [www1, www2, www3], db: [db1, db2] } @roles(db) def restart_db(): sudo(/etc/init.d/postgresql restart) @roles(‘web’) def restart_web(): sudo(/etc/init.d/nginx restart) $fab restart_web restart_dbSunday, April 14, 13
  29. 29. env.roledefs = { web: [www1, www2, www3], db: [db1, db1] } @roles(db, ‘web’) def run_puppet(): sudo(puppet apply site.pp) $fab run_puppet #run puppet everywhere $fab run_puppet --roles db # role-specificSunday, April 14, 13
  30. 30. env.roledefs = configure_roles() @roles(db, ‘web’) def run_puppet(): sudo(puppet apply site.pp) $fab run_puppet #run puppet everywhere $fab run_puppet --roles db # role-specificSunday, April 14, 13
  31. 31. def configure_roles():  tags = EC2TagManager(AWS_KEY, AWS_SECRET,    regions=[eu-west-1])  roles = {} for role in [‘db’, ‘web’]: roles[role] = tags.get_instances(role=role) return roles https://github.com/mikery/fabric-ec2Sunday, April 14, 13
  32. 32. Questions? Mike Ryan - Epitech.nl - mike@epitech.nlSunday, April 14, 13

×