0
11 Ways to hackPuppet for Fun and            Luke Kanies            Founder and CEO, Puppet            Labs            Fou...
Puppet                 Deploy 1,800 machines in 2 hoursMobile Phone     vs. 25 machines per day with HP              Inves...
Built for you
Great Design http://www.encorbio.com/Album/pages/ChkNFH-neuron1.htm
Configuration  Platform
FearEmbarrassment    SSH
20%
More great sysadmins           Programmers   Sysadmins 5000       Assembly      Scripts           Ruby, Java,Millions     ...
Incremental Investment                                    Fully Automated Infrastructure                              Whol...
Why use Puppet?
Scale at speed
Availability
Security
Auditability
Golden image?Image from http://www.flickr.com/photos/fungep/2516767121/sizes/
Puppet: A brief introduction
A language for configuration       specification
Resource Abstractioncomputer      packagecron          portfile          resourcesgroup         routerhost          servic...
Cross PlatformRed Hat   SolarisFedora     OS X     WindowsDebian      AIX                     CiscoUbuntu     HP-UXCentOS ...
Workflow                                                                                            1   Define: With Puppets...
Change Propagation                                              Node                    1   Facts                        T...
domain => local                   Automatic Inventoryfacterversion => 1.5.8fqdn => sliver.localhardwaremodel => i386hostna...
Centralized, Serverless, or     Puppet Dashboard         Puppet Module Forge      3rd Party Systems                       ...
Scales like HTTPS2 known 50k node sitesMultiple 30k node sites Tens of 3k node sites
Built as a platform
Model-based hackability• Hosts• Inventory data  • IP, hostname, platform, etc.• Resource lists• Resource dependencies• Cha...
Puppet Facesca                            keycatalog                       mancertificate                   nodecertificat...
MCollective:Infrastructure message bus
Puppet Forge404 free, reusable modules
Hacking Puppet forFun and Productivity
QuestionAuthority
Model-based Hackability
Exampleshttps://github.com/lak/velocity_2012-Hacking_Puppet https://github.com/puppetlabs/puppetlabs-stdlib/
Hack #1: Puppet as you       ~/.puppet/puppet.conf
Editing files as youCode:Result:
Simpler versionCode:Result:
Client/serverCode:Server:              (and lots more text)Client:
Hack #2: Curl speaksPuppet:Curl:
Hack #3: Data, and lots of it
Data                                              Node                    1   Facts                        The node sends ...
#3.1: Catalog
Catalog as JSON{    "data": {      "edges": [         {            "source": "Stage[main]",            "target": "Class[Se...
So?Control separately:Resource list:Code:
#3.2: Facts
Every host, automatically
#3.3: Reports
EveryChange
Node   Fact ResourceDependency  Change
Hack #4: Static Compiler
Checksum vs. URLBefore:After:
Normal data flows
Static compiler, run 1
Static compiler, run 2
Hack #5: config_versionDefault:After:
Extensions
Hack #6: Report       http://puppetlabs.com/blog/when-puppet-reports-part-1/https://github.com/jamtur01/puppet-growl/blob/...
Hack #6: Report       http://puppetlabs.com/blog/when-puppet-reports-part-1/https://github.com/jamtur01/puppet-growl/blob/...
Hack #7: Functionhttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/                           ...
Hack #7: Functionhttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/                           ...
Hack #8: Facthttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/facter/                            root_home.rb
Hack #8: Facthttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/facter/                            root_home.rb
#9 and #10:    Types and ProvidersExamples from http://www.kartar.net/2010/02/puppet-types-and-providers-are-easy/
Exec in Puppet
Native resource
Native resources areExec:Native:
Resource Type
Resource Type
Resource Type
Resource Type
Resource Provider
Resource Provider
Resource Provider
Resource Provider
Hack #11: Face https://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
Setuphttps://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
actualwork.com https://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
Summary
Configurable
Controllable                                              Node                    1   Facts                        The nod...
Node   Fact ResourceDependency  Change
Hackable
Questions?
Upcoming SlideShare
Loading in...5
×

11 Ways to Hack Puppet for Fun and Productivity - Luke Kanies - Velocity 2012

24,862

Published on

Luke Kanies, founder and CEO of Puppet Labs, talks at Velocity '12 on 11 ways to hack Puppet for fun and productivity. http://www.puppetlabs.com

1 Comment
18 Likes
Statistics
Notes
  • good job
       Reply 
    Are you sure you want to  Yes  No
    Your message goes here
No Downloads
Views
Total Views
24,862
On Slideshare
0
From Embeds
0
Number of Embeds
18
Actions
Shares
0
Downloads
324
Comments
1
Likes
18
Embeds 0
No embeds

No notes for slide
  • \n
  • \n
  • Commercial tools built for the execs\nOSS built for the toolbuilders, or maybe advanced users\nWe needed a tool that everyone could use\n
  • We make decisions for you\n1000 knobs\nBrain, complexity blah blah blah\nComplexity through building blocks, rather than big things\n
  • Fundamental technology that everyone could build on\nThis presentation is largely about the platform aspects\n
  • Fear that the world would still look the same in 10 years\n- after all, it hadn’t changed much in the previous 10\n- rsh to ssh, but...\nEmbarrassment at how bad the state of IT was\nHatred of thinking SSH was a management tool\n
  • \n
  • We don’t want no-ops, we want pervasive ops, accessible ops\n
  • \n
  • \n
  • Remember when 1000 machines was a lot?\n1000 machines by Friday\n1000 machines every Friday\n
  • Remember maintenance windows?\n96% of outages are caused by human error\n
  • Air gap\nLeast privilege\nUntrusted clients\n
  • \n
  • Platonic ideal of a machine\n
  • \n
  • Full ruby DSL\n
  • Do you really care how RPM works?\nFull simulation mode\nDiscovery, diff, and change\nEasily extensible\nLots of custom types\n
  • \n
  • \n
  • Every half an hour\nHow change progresses through your infrastructure\nExplain:\nFacts\nCatalog\nReport\n
  • \n
  • Agent on all of your nodes\nOptional master for compilation, reporting, etc.\n- with no central master, no reporting\n- compilation can be distributed for load reasons\nDashboard is reporting\nForge for code sharing\nLots and lots of data\nAll modes share the same code paths\n
  • \n
  • \n
  • What we’re talking about today\n
  • \n
  • \n
  • \n
  • Focused on content, not form\nWant to be clear I haven’t sold out\n
  • \n
  • \n
  • These slides and examples prove I haven’t sold out\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • 750 GB of data per day\n
  • Every half an hour\nHow change progresses through your infrastructure\nExplain:\nFacts\nCatalog\nReport\n
  • Graph\nResources and dependencies`\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Unfortunately seems to be broken in 2.7.17\nWill be default in 3.0, hopefully\n
  • Remote files can change w/o affecting configuration\nEasily compare contents\nMuch less network traffic\n
  • Note the lots of files, throughout the conversation\n
  • Note that all conversation is up front, rather than on demand\n
  • Few file content changes, thus much more efficient\nHack because it’s complex\nBut will be default soon - seamless upgrade\nAll about performance and change control\n
  • \n
  • \n
  • 29 lines of code\n8 line method does all the work\n
  • 29 lines of code\n8 line method does all the work\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Simple, but...\nNote very reusable\nLots of content built into the shell script\n
  • Existing vcsrepo type that you should use for this\n
  • Better logs\nCreate and delete\nAuditing\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Every half an hour\nHow change progresses through your infrastructure\nExplain:\nFacts\nCatalog\nReport\n
  • \n
  • \n
  • \n
  • Transcript of "11 Ways to Hack Puppet for Fun and Productivity - Luke Kanies - Velocity 2012"

    1. 1. 11 Ways to hackPuppet for Fun and Luke Kanies Founder and CEO, Puppet Labs Founder of Puppet @puppetmasterd luke@puppetlabs.com
    2. 2. Puppet Deploy 1,800 machines in 2 hoursMobile Phone vs. 25 machines per day with HP Investors Company Opsware Scaled from 0 to over 10,000 servers in 2 months without training 287 servers per SysAdmin vs. 19 for BMC BladeLogic Over 50,000 systems managed by PuppetFinancial Entertainment Technology Defense Web
    3. 3. Built for you
    4. 4. Great Design http://www.encorbio.com/Album/pages/ChkNFH-neuron1.htm
    5. 5. Configuration Platform
    6. 6. FearEmbarrassment SSH
    7. 7. 20%
    8. 8. More great sysadmins Programmers Sysadmins 5000 Assembly Scripts Ruby, Java,Millions Puppet PHP, C
    9. 9. Incremental Investment Fully Automated Infrastructure Whole machines Multiple services AwesomenessOne file Investment
    10. 10. Why use Puppet?
    11. 11. Scale at speed
    12. 12. Availability
    13. 13. Security
    14. 14. Auditability
    15. 15. Golden image?Image from http://www.flickr.com/photos/fungep/2516767121/sizes/
    16. 16. Puppet: A brief introduction
    17. 17. A language for configuration specification
    18. 18. Resource Abstractioncomputer packagecron portfile resourcesgroup routerhost serviceinterface sshkeyk5login stagemailalias usermaillist vcsrepomcx vlanmount yumrepo
    19. 19. Cross PlatformRed Hat SolarisFedora OS X WindowsDebian AIX CiscoUbuntu HP-UXCentOS OpenBSD F5 SuSE FreeBSD
    20. 20. Workflow 1 Define: With Puppets declarative language you design a graph of relationships between resources within reusable modules. These modules define your infrastructure in its desired state. E R AG VE CO SE EA CR IN ND E A I T E R AT4 Report: Puppet Dashboard reports 2 Simulate: With this resource track relationships between graph, Puppet is unique in itscomponents and all changes, allowing ability to simulate deployments, enablingyou to keep up with security and you to test changes without disruptioncompliance mandates. And with the to your infrastructure.open API you can integrate Puppet withthird party monitoring tools. CURRENT 3 Enforce: Puppet compares your STATE system to the desired state as you define it, and automatically enforces it DESIRED STATE to the desired state ensuring your system is in compliance.
    21. 21. Change Propagation Node 1 Facts The node sends normalized data about itself to the Puppet Master. SSL secure 2 Catalog Puppet uses the Facts to encryption compile a Catalog that on all data specifies how the node transport should be configured.Report 3The nodereports backto Puppetindicating theconfiguration iscomplete, whichis visible in the PuppetPuppet Dashboard. Master 4 Report Collector (Puppet or 3rd party tool) Report can also send data to third party tools.
    22. 22. domain => local Automatic Inventoryfacterversion => 1.5.8fqdn => sliver.localhardwaremodel => i386hostname => sliverinterfaces => lo0,gif0,stf0,en0,en1,fw0,vmnet1,vboxnet0ipaddress => 192.168.174.1ipaddress_lo0 => 127.0.0.1ipaddress_vmnet1 => 192.168.174.1kernel => Darwinkernelmajversion => 10.6kernelrelease => 10.6.0macosx_productname => Mac OS Xmacosx_productversion => 10.6.6netmask => 255.255.255.0netmask_lo0 => 255.0.0.0netmask_vmnet1 => 255.255.255.0network_lo0 => 127.0.0.0network_vmnet1 => 192.168.174.0operatingsystem => Darwinoperatingsystemrelease => 10.6.0path => /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/binps => ps auxwwwpuppetversion => 2.6.4rubysitedir => /opt/local/lib/ruby/site_ruby/1.8timezone => PSTuptime => 1 dayrubyversion => 1.8.7sp_bus_speed => 1.07 GHz
    23. 23. Centralized, Serverless, or Puppet Dashboard Puppet Module Forge 3rd Party Systems Data Modules MCollective handles Puppet Master orchestration Facter Facter Facter Puppet Agent Puppet Agent Puppet Agent NODE NODE NODE
    24. 24. Scales like HTTPS2 known 50k node sitesMultiple 30k node sites Tens of 3k node sites
    25. 25. Built as a platform
    26. 26. Model-based hackability• Hosts• Inventory data • IP, hostname, platform, etc.• Resource lists• Resource dependencies• Change events
    27. 27. Puppet Facesca keycatalog mancertificate nodecertificate_request parsercertificate_revocation_list pluginconfig reportfacts resourcefile resource_typehelp status
    28. 28. MCollective:Infrastructure message bus
    29. 29. Puppet Forge404 free, reusable modules
    30. 30. Hacking Puppet forFun and Productivity
    31. 31. QuestionAuthority
    32. 32. Model-based Hackability
    33. 33. Exampleshttps://github.com/lak/velocity_2012-Hacking_Puppet https://github.com/puppetlabs/puppetlabs-stdlib/
    34. 34. Hack #1: Puppet as you ~/.puppet/puppet.conf
    35. 35. Editing files as youCode:Result:
    36. 36. Simpler versionCode:Result:
    37. 37. Client/serverCode:Server: (and lots more text)Client:
    38. 38. Hack #2: Curl speaksPuppet:Curl:
    39. 39. Hack #3: Data, and lots of it
    40. 40. Data Node 1 Facts The node sends normalized data about itself to the Puppet Master. SSL secure 2 Catalog Puppet uses the Facts to encryption compile a Catalog that on all data specifies how the node transport should be configured.Report 3The nodereports backto Puppetindicating theconfiguration iscomplete, whichis visible in the PuppetPuppet Dashboard. Master 4 Report Collector (Puppet or 3rd party tool) Report can also send data to third party tools.
    41. 41. #3.1: Catalog
    42. 42. Catalog as JSON{ "data": { "edges": [ { "source": "Stage[main]", "target": "Class[Settings]" }, { "source": "Stage[main]", "target": "Class[main]" }, { "source": "Class[main]", "target": "File[/tmp/local_file]" } ], "resources": [ { "type": "Class", "exported": false, "title": "Settings", "tags": [ "class", "settings" ] },
    43. 43. So?Control separately:Resource list:Code:
    44. 44. #3.2: Facts
    45. 45. Every host, automatically
    46. 46. #3.3: Reports
    47. 47. EveryChange
    48. 48. Node Fact ResourceDependency Change
    49. 49. Hack #4: Static Compiler
    50. 50. Checksum vs. URLBefore:After:
    51. 51. Normal data flows
    52. 52. Static compiler, run 1
    53. 53. Static compiler, run 2
    54. 54. Hack #5: config_versionDefault:After:
    55. 55. Extensions
    56. 56. Hack #6: Report http://puppetlabs.com/blog/when-puppet-reports-part-1/https://github.com/jamtur01/puppet-growl/blob/master/lib/puppet/reports/growl.rb
    57. 57. Hack #6: Report http://puppetlabs.com/blog/when-puppet-reports-part-1/https://github.com/jamtur01/puppet-growl/blob/master/lib/puppet/reports/growl.rb
    58. 58. Hack #7: Functionhttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/ functions/flatten.rb
    59. 59. Hack #7: Functionhttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/ functions/flatten.rb
    60. 60. Hack #8: Facthttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/facter/ root_home.rb
    61. 61. Hack #8: Facthttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/facter/ root_home.rb
    62. 62. #9 and #10: Types and ProvidersExamples from http://www.kartar.net/2010/02/puppet-types-and-providers-are-easy/
    63. 63. Exec in Puppet
    64. 64. Native resource
    65. 65. Native resources areExec:Native:
    66. 66. Resource Type
    67. 67. Resource Type
    68. 68. Resource Type
    69. 69. Resource Type
    70. 70. Resource Provider
    71. 71. Resource Provider
    72. 72. Resource Provider
    73. 73. Resource Provider
    74. 74. Hack #11: Face https://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
    75. 75. Setuphttps://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
    76. 76. actualwork.com https://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
    77. 77. Summary
    78. 78. Configurable
    79. 79. Controllable Node 1 Facts The node sends normalized data about itself to the Puppet Master. SSL secure 2 Catalog Puppet uses the Facts to encryption compile a Catalog that on all data specifies how the node transport should be configured.Report 3The nodereports backto Puppetindicating theconfiguration iscomplete, whichis visible in the PuppetPuppet Dashboard. Master 4 Report Collector (Puppet or 3rd party tool) Report can also send data to third party tools.
    80. 80. Node Fact ResourceDependency Change
    81. 81. Hackable
    82. 82. Questions?
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×