Adventures in infrastructure as code

2,689 views

Published on

We're all developers now. Infrastructure as code is a new art and science, and we'd all better learn to get past hello world.

Published in: Technology, Self Improvement
0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
2,689
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
23
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide
  • Explain the 50% habit - rather have walkouts than reds Aussie developers seem smart Organisations make people smart Code has been reviewed by opscode and puppet labs
  • Ask them to make a mental note of it Agenda: demo, code and then theory You can ask questions: stop me if I’m going too fast/too slow
  • Story of the change management freak - no more than 5 minutes Introduction: sysadmin, build monkey, devops believer CM is being redefined in some people’s eyes/BCS ‘honest broker’
  • Define Puppet and Chef: both DSL’s for systems administration Why would we bother?
  • Enables seamless testing *Drive home that this *IS* what prod looks like Especially helpful if you use windows
  • All examples assume a single node with no server Point out the use of server is common and neccessary with any real number of nodes Chef 5 nodes, Puppetmaster easy to install
  • Here’s our corporate application
  • Shebangs - very simple command line tools to run Puppetmaster vs local
  • Chef server vs chef-solo the first tool is always a framework
  • PUPPET resources: package, file, service providers
  • CHEF Chef has cookbooks/recipes/roles
  • PUPPET - with Jetty pp file is getting big
  • CHEF line 1 is as about as graph-like
  • PUPPET show file distribution
  • CHEF: show file distribution
  • puppet modules are the unit of reuse
  • puppet is a directed graph
  • Role class
  • Some of the tooling goes back to the early 90’s It was easy desktop virtualisation, cloud and decent dynamic langs that made it possible
  • This ignores a lot, like LCFG, BCFG2, all the commercial tools, all the johnny-come lateleys
  • Puppet has a Ruby DSL now Providers and Resources are useful like Apache Ant’s tasks Chef has databags, puppet needs to implement other things
  • Congruent: how many Java or .NET VM’s did you kill yesterday? Congruent school of thought insists that order matters Convergent school is that order sometimes matters, and machines can drift closer
  • Puppet’s ordering is declarative. Chef’s isn’t. Chef is more declarative in some ways than Puppet: see the brevity of code
  • querying and command
  • Adventures in infrastructure as code

    1. 1. We’re all developers now <ul><li>Adventures in Infrastructure as Code </li></ul>Julian Simpson, The Build Doctor Ltd
    2. 2. Before we begin <ul><li>For gory detail, talk to me at the drinks </li></ul><ul><li>Examples: http://github.com/builddoctor/infrastructure-as-code </li></ul><ul><li>You can ask questions </li></ul>
    3. 3. Poll <ul><li>Are you using Puppet or Chef? </li></ul><ul><li>What about Vagrant? </li></ul><ul><li>Do you hate Ruby on Rails? </li></ul><ul><li>Do you hate Apache Ant/NAnt? </li></ul>
    4. 4. Demo
    5. 5. What just happened? <ul><li>We spawned a fresh Ubuntu machine </li></ul><ul><li>and mounted Puppet code </li></ul><ul><li>and executed it </li></ul><ul><li>and tested the desired state of the machine </li></ul>
    6. 6. Vagrant
    7. 7. Examples
    8. 9. Hello Puppet 1 #!/usr/bin/env puppet apply2 3 file {4 '/tmp/PuppetHelloWorld':5 content => 'Hello Yow!n'6 }
    9. 10. Hello Puppet knox:puppet jsimpson$ ./manifests/classes/01_hello_world.pp warning: Could not retrieve fact fqdn notice: /Stage[main]//File[/tmp/PuppetHelloWorld]/ensure: defined content as '{md5}54393566ca75844a50baf0c6bccd84b5' notice: Finished catalog run in 0.16 seconds knox:puppet jsimpson$ cat /tmp/PuppetHelloWorld Hello Yow! knox:puppet jsimpson$
    10. 11. Hello Chef cat -n chef-repo/cookbooks/hello/recipes/default.rb 1 file &quot;/tmp/Chef_Hello_World&quot; do2 content &quot;Hello, world!&quot;3 end
    11. 12. Hello Chef knox:chef-repo jsimpson$ ./go [Thu, 01 Dec 2011 14:26:46 +1100] INFO: *** Chef 0.10.4 ***[Thu, 01 Dec 2011 14:26:46 +1100] INFO: Setting the run_list to [&quot;recipe[hello]&quot;] from JSON[Thu, 01 Dec 2011 14:26:46 +1100] INFO: Run List is [recipe[hello]][Thu, 01 Dec 2011 14:26:46 +1100] INFO: Run List expands to [hello][Thu, 01 Dec 2011 14:26:46 +1100] INFO: Starting Chef Run for knox[Thu, 01 Dec 2011 14:26:46 +1100] INFO: Processing file[/tmp/Chef_Hello_World] action create (hello::default line 1)[Thu, 01 Dec 2011 14:26:47 +1100] INFO: file[/tmp/Chef_Hello_World] created file /tmp/Chef_Hello_World[Thu, 01 Dec 2011 14:26:47 +1100] INFO: Chef Run complete in 0.084655 seconds[Thu, 01 Dec 2011 14:26:47 +1100] INFO: Running report handlers[Thu, 01 Dec 2011 14:26:47 +1100] INFO: Report handlers complete
    12. 13. 1 class nginx::install {2 3 package {4 'nginx':5 ensure => present;6 }7 8 file {9 'default web page':10 path => '/var/www/nginx-default/index.html',11 content => 'This page, courtesy of puppet',12 require => Package['nginx'];13 }14 15 service {16 'nginx':17 ensure => running,18 enable => true,19 hasstatus => true,20 require => [Package['nginx'], File['default web page']];21 }22 23 }24 25 node default {26 include nginx::install27 }
    13. 14. cat -n recipes/default.rb 1 require_recipe &quot;apt&quot;2 3 package &quot;nginx&quot; do4 action :install5 end6 7 service &quot;nginx&quot; do8 action [:enable, :start]9 end cat -n config/node.json 1 {2 &quot;run_list&quot;: [ 3 &quot;recipe[nginx]&quot;4 ]5 }6
    14. 15. 30 class jetty::install {31 include ubuntu::common32 33 package {34 'jetty':35 ensure => present,36 require => Class['ubuntu::common'];37 }38 39 service {40 'jetty':41 ensure => running,42 enable => true,43 hasstatus => true,44 require => [Package['jetty'], File['/etc/default/jetty']];45 }46 47 file {48 '/etc/default/jetty':49 ensure => present,50 content => &quot;51 NO_START=052 VERBOSE=yes53 &quot;54 }55 }
    15. 16. 1 require_recipe &quot;apt&quot;2 3 package &quot;jetty&quot; do4 action :install5 end6 7 service &quot;jetty&quot; do8 action [:enable]9 end10 11 file &quot;/etc/default/jetty&quot; do12 mode &quot;0644&quot;13 content &quot;NO_START=0nVERBOSE=yesn&quot;14 notifies :restart, resources(:service => &quot;jetty&quot;)15 end
    16. 17. <ul><li>1 class corporateapp::install {2 include jetty::install3 include nginx::install4 file {5 'corporate app':6 ensure => file,7 path => '/usr/share/jetty/webapps/app.war',8 source => 'puppet:///modules/corporateapp/app.war',9 require => [Class['jetty::install'], Class['nginx::install']], </li></ul><ul><ul><ul><ul><ul><li>notify => Exec['restart jetty'];10 }11 } </li></ul></ul></ul></ul></ul><ul><li>notify => Exec['restart jetty'];10 }11 } </li></ul>
    17. 18. 1 include_recipe &quot;jetty&quot;2 3 # We do it like this for the example...4 cookbook_file &quot;/usr/share/jetty/webapps/app.war&quot; do5 source &quot;app.war&quot;6 mode &quot;0644&quot;7 end8 9 # In real life I'd do something like...10 #remote_file &quot;/usr/share/jetty/webapps/app.war&quot; do11 # source node[:jetty][:deploy][:source]12 # checksum node[:jetty][:deploy][:checksum] if node[:jetty][:deploy][:checksum]13 # notifies :restart, &quot;service[jetty]&quot;14 #end
    18. 19. ├── Gemfile├── Gemfile.lock├── Rakefile├── TODO├── Vagrantfile├── manifests│   ├── classes│   │   ├── 01_hello_world.pp│   │   ├── 02_installed_app_on_nginx.pp│   │   └── 03_installed_app_on_nginx_and_jetty.pp│   └── site.pp└── modules ├── corporateapp │   ├── files │   │   └── app.war │   └── manifests │   └── install.pp ├── jetty │   └── manifests │   └── install.pp ├── nginx │   ├── files │   │   └── nginx_ www.corporateapp.com │   └── manifests │   └── install.pp ├── puppet │   └── manifests │   └── fudge.pp └── ubuntu └── manifests └── common.pp15 directories, 16 files
    19. 21. ├── config│   ├── node.json│   └── solo.rb├── cookbooks│   ├── apt│   │   └── recipes│   │   └── default.rb│   ├── hello│   │   └── recipes│   │   └── default.rb│   ├── jetty│   │   ├── files│   │   │   └── default│   │   │   └── app.war│   │   └── recipes│   │   ├── default.rb│   │   └── deploy.rb│   └── nginx│   ├── attributes│   │   └── default.rb│   ├── recipes│   │   ├── default.rb│   │   └── loadbalancer.rb│   └── templates│   └── default│   └── lb.erb├── go└── roles └── corporateapp.rb
    20. 22. 1 name &quot;corporateapp&quot;2 description &quot;Corporate App Server&quot;3 run_list(4 &quot;recipe[jetty]&quot;, 5 &quot;recipe[jetty::deploy]&quot;, 6 &quot;recipe[nginx::loadbalancer]&quot;7 )8 9 override_attributes(10 &quot;nginx&quot; => {11 &quot;loadbalancer&quot; => {12 &quot;name&quot; => &quot;corporateapp&quot;,13 &quot;source&quot; => &quot; http://localhost:8080 &quot;14 }15 }, 16 &quot;jetty&quot; => {17 &quot;deploy&quot; => {18 &quot;source&quot; => &quot; http://build-reposito ry/build-number/app.war &quot;19 }20 } 21 )22
    21. 23. History
    22. 24. This isn’t new
    23. 25. domain = ( sequenceapp.com )actionsequence = ( tidy disable resolve files directories copy shellcommands links editfiles processes )################################################################################tidy:################################################################################$(sequenceetc) pattern=*.cfsaved age=0!cfmaster::# stops cfservd running on everything except master server/etc/rc2.d/ pattern=S97cfservd age=0# get rid of any hosts.equiv/etc pattern=hosts.equiv age=0 recurse=0homedirs::# no .rhosts files!Hr00::/export/home pattern=.rhosts age=0 recurse=inf
    24. 26. Overview http://verticalsysadmin.com/blog/uncategorized/relative-origins-of-cfengine-chef-and-puppet
    25. 27. Differences
    26. 28. *IMHO Puppet Chef Ruby DSL Ruby DSL Parsed Internal Declarative* Imperative* Configuration Convention Convergent Congruent Sysadmins* Developers*
    27. 29. Convergent vs Congruent
    28. 30. “ if you want a tool to be congruent, you really have to never re-write history. You have to constantly apply every byte-for-byte change in the same order, on every system you build. Skip a step in history, and everything goes off-kilter” Adam Jacob, 4 Dec
    29. 31. Luke Kanies is here to present on “Essential Incompleteness in Program Modeling”, and starts by getting right into Godel’s Incompleteness Theorem. [it] says that for any system that attempts to model reality (“any sufficiently complex system”), it can never be both consistent and complete.
    30. 32. Declarative vs Imperative vs Imperative <ul><li>Declarative programming is a programming paradigm that expresses the logic of a computation without describing its control flow. </li></ul><ul><li>Imperative programming is a programming paradigm that describes computation in terms of statements that change a program state ... imperative programs define sequences of commands for the computer to perform. </li></ul>imperative programming is a programming paradigm that describes computation in terms of statements that chan ge a progr am state . In much the same way th at im perative mood in natural language s expresses com mands to take actio n, imperative pro grams define sequences of commands for the computer to perform.
    31. 33. Cluster-wide facts <ul><li>Chef recipes can access attributes of other nodes </li></ul><ul><li>Puppet can collect node info with stored config </li></ul><ul><li>but ... </li></ul>
    32. 34. Nothing beats realtime
    33. 35. What about ...
    34. 36. Windows? Resource Puppet Chef File ✔ ✔ User ✔ ✔ Group ✔ ✔ Scheduled Task ✔ ✔ Service ✔ ✔ Exec ✔ ✔ Host ✔ ✔ Package(MSI) ✔ ✘ Powershell ✘ ✔
    35. 37. Testing? <ul><li>https://github.com/nistude/cucumber-puppet </li></ul><ul><li>https://github.com/Atalanta/cucumber-chef </li></ul><ul><li>https://github.com/gregretkowski/vmth/ </li></ul><ul><li>https://github.com/rodjek/puppet-lint </li></ul><ul><li>http://cloudsmith.github.com/geppetto/ </li></ul>
    36. 38. Validating <ul><li>task :parse_pp do file = File.open(&quot;parseonly.pp&quot;,&quot;w+&quot;) Dir.glob('etc/puppet/**/*.pp').each do |manifest| </li></ul><ul><li>file << &quot;import '#{manifest}'n&quot; </li></ul><ul><ul><li>end sh &quot;bundle exec puppet parser validate parseonly.pp&quot; end </li></ul></ul><ul><li>end sh &quot;bundle exec puppet parser validate parseonly.pp&quot; end </li></ul>
    37. 39. Poll Results <ul><li>You might prefer Chef if you get annoyed by Ant’s ‘depends’ model </li></ul><ul><li>You might prefer Puppet if you despise the magic of Ruby on Rails </li></ul>
    38. 40. Disclaimer <ul><li>I don’t endorse either tool - you’ll need to make your own decision </li></ul><ul><li>You can really, really screw things up with these tools. Test twice, run once. </li></ul>
    39. 41. Thank you - questions? <ul><li>http://github.com/builddoctor/infrastructure-as-code </li></ul><ul><li>@builddoctor </li></ul><ul><li>Much thanks to @cread, @tastapod, @adamgibbins, @puppetmasterd, @adamhjk </li></ul>

    ×