11 Ways to Hack Puppet for Fun and Productivity - Luke Kanies - Velocity 2012
Upcoming SlideShare
Loading in...5
×
 

11 Ways to Hack Puppet for Fun and Productivity - Luke Kanies - Velocity 2012

on

  • 21,710 views

Luke Kanies, founder and CEO of Puppet Labs, talks at Velocity '12 on 11 ways to hack Puppet for fun and productivity. http://www.puppetlabs.com

Luke Kanies, founder and CEO of Puppet Labs, talks at Velocity '12 on 11 ways to hack Puppet for fun and productivity. http://www.puppetlabs.com

Statistics

Views

Total Views
21,710
Views on SlideShare
18,996
Embed Views
2,714

Actions

Likes
16
Downloads
282
Comments
1

13 Embeds 2,714

http://puppetlabs.com 2483
http://lanyrd.com 76
http://drupal-dev.puppetlabs.com 49
https://twitter.com 35
http://scott.local 32
http://eventifier.co 13
http://richards-mbp.corp.puppetlabs.net 8
http://scott.wifi.puppetlabs.net 6
http://translate.googleusercontent.com 4
http://www.google.com 3
http://www.puppetlabs.com 3
https://puppetlabs.com 1
http://www.google.com.pe 1
More...

Accessibility

Upload Details

Uploaded via as Apple Keynote

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
  • good job
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment
  • \n
  • \n
  • Commercial tools built for the execs\nOSS built for the toolbuilders, or maybe advanced users\nWe needed a tool that everyone could use\n
  • We make decisions for you\n1000 knobs\nBrain, complexity blah blah blah\nComplexity through building blocks, rather than big things\n
  • Fundamental technology that everyone could build on\nThis presentation is largely about the platform aspects\n
  • Fear that the world would still look the same in 10 years\n- after all, it hadn’t changed much in the previous 10\n- rsh to ssh, but...\nEmbarrassment at how bad the state of IT was\nHatred of thinking SSH was a management tool\n
  • \n
  • We don’t want no-ops, we want pervasive ops, accessible ops\n
  • \n
  • \n
  • Remember when 1000 machines was a lot?\n1000 machines by Friday\n1000 machines every Friday\n
  • Remember maintenance windows?\n96% of outages are caused by human error\n
  • Air gap\nLeast privilege\nUntrusted clients\n
  • \n
  • Platonic ideal of a machine\n
  • \n
  • Full ruby DSL\n
  • Do you really care how RPM works?\nFull simulation mode\nDiscovery, diff, and change\nEasily extensible\nLots of custom types\n
  • \n
  • \n
  • Every half an hour\nHow change progresses through your infrastructure\nExplain:\nFacts\nCatalog\nReport\n
  • \n
  • Agent on all of your nodes\nOptional master for compilation, reporting, etc.\n- with no central master, no reporting\n- compilation can be distributed for load reasons\nDashboard is reporting\nForge for code sharing\nLots and lots of data\nAll modes share the same code paths\n
  • \n
  • \n
  • What we’re talking about today\n
  • \n
  • \n
  • \n
  • Focused on content, not form\nWant to be clear I haven’t sold out\n
  • \n
  • \n
  • These slides and examples prove I haven’t sold out\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • 750 GB of data per day\n
  • Every half an hour\nHow change progresses through your infrastructure\nExplain:\nFacts\nCatalog\nReport\n
  • Graph\nResources and dependencies`\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Unfortunately seems to be broken in 2.7.17\nWill be default in 3.0, hopefully\n
  • Remote files can change w/o affecting configuration\nEasily compare contents\nMuch less network traffic\n
  • Note the lots of files, throughout the conversation\n
  • Note that all conversation is up front, rather than on demand\n
  • Few file content changes, thus much more efficient\nHack because it’s complex\nBut will be default soon - seamless upgrade\nAll about performance and change control\n
  • \n
  • \n
  • 29 lines of code\n8 line method does all the work\n
  • 29 lines of code\n8 line method does all the work\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Simple, but...\nNote very reusable\nLots of content built into the shell script\n
  • Existing vcsrepo type that you should use for this\n
  • Better logs\nCreate and delete\nAuditing\n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • \n
  • Every half an hour\nHow change progresses through your infrastructure\nExplain:\nFacts\nCatalog\nReport\n
  • \n
  • \n
  • \n

11 Ways to Hack Puppet for Fun and Productivity - Luke Kanies - Velocity 2012 11 Ways to Hack Puppet for Fun and Productivity - Luke Kanies - Velocity 2012 Presentation Transcript

  • 11 Ways to hackPuppet for Fun and Luke Kanies Founder and CEO, Puppet Labs Founder of Puppet @puppetmasterd luke@puppetlabs.com
  • Puppet Deploy 1,800 machines in 2 hoursMobile Phone vs. 25 machines per day with HP Investors Company Opsware Scaled from 0 to over 10,000 servers in 2 months without training 287 servers per SysAdmin vs. 19 for BMC BladeLogic Over 50,000 systems managed by PuppetFinancial Entertainment Technology Defense Web
  • Built for you
  • Great Design http://www.encorbio.com/Album/pages/ChkNFH-neuron1.htm
  • Configuration Platform
  • FearEmbarrassment SSH
  • 20%
  • More great sysadmins Programmers Sysadmins 5000 Assembly Scripts Ruby, Java,Millions Puppet PHP, C
  • Incremental Investment Fully Automated Infrastructure Whole machines Multiple services AwesomenessOne file Investment
  • Why use Puppet?
  • Scale at speed
  • Availability
  • Security
  • Auditability
  • Golden image?Image from http://www.flickr.com/photos/fungep/2516767121/sizes/
  • Puppet: A brief introduction
  • A language for configuration specification
  • Resource Abstractioncomputer packagecron portfile resourcesgroup routerhost serviceinterface sshkeyk5login stagemailalias usermaillist vcsrepomcx vlanmount yumrepo
  • Cross PlatformRed Hat SolarisFedora OS X WindowsDebian AIX CiscoUbuntu HP-UXCentOS OpenBSD F5 SuSE FreeBSD
  • Workflow 1 Define: With Puppets declarative language you design a graph of relationships between resources within reusable modules. These modules define your infrastructure in its desired state. E R AG VE CO SE EA CR IN ND E A I T E R AT4 Report: Puppet Dashboard reports 2 Simulate: With this resource track relationships between graph, Puppet is unique in itscomponents and all changes, allowing ability to simulate deployments, enablingyou to keep up with security and you to test changes without disruptioncompliance mandates. And with the to your infrastructure.open API you can integrate Puppet withthird party monitoring tools. CURRENT 3 Enforce: Puppet compares your STATE system to the desired state as you define it, and automatically enforces it DESIRED STATE to the desired state ensuring your system is in compliance.
  • Change Propagation Node 1 Facts The node sends normalized data about itself to the Puppet Master. SSL secure 2 Catalog Puppet uses the Facts to encryption compile a Catalog that on all data specifies how the node transport should be configured.Report 3The nodereports backto Puppetindicating theconfiguration iscomplete, whichis visible in the PuppetPuppet Dashboard. Master 4 Report Collector (Puppet or 3rd party tool) Report can also send data to third party tools.
  • domain => local Automatic Inventoryfacterversion => 1.5.8fqdn => sliver.localhardwaremodel => i386hostname => sliverinterfaces => lo0,gif0,stf0,en0,en1,fw0,vmnet1,vboxnet0ipaddress => 192.168.174.1ipaddress_lo0 => 127.0.0.1ipaddress_vmnet1 => 192.168.174.1kernel => Darwinkernelmajversion => 10.6kernelrelease => 10.6.0macosx_productname => Mac OS Xmacosx_productversion => 10.6.6netmask => 255.255.255.0netmask_lo0 => 255.0.0.0netmask_vmnet1 => 255.255.255.0network_lo0 => 127.0.0.0network_vmnet1 => 192.168.174.0operatingsystem => Darwinoperatingsystemrelease => 10.6.0path => /opt/local/bin:/opt/local/sbin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/binps => ps auxwwwpuppetversion => 2.6.4rubysitedir => /opt/local/lib/ruby/site_ruby/1.8timezone => PSTuptime => 1 dayrubyversion => 1.8.7sp_bus_speed => 1.07 GHz
  • Centralized, Serverless, or Puppet Dashboard Puppet Module Forge 3rd Party Systems Data Modules MCollective handles Puppet Master orchestration Facter Facter Facter Puppet Agent Puppet Agent Puppet Agent NODE NODE NODE
  • Scales like HTTPS2 known 50k node sitesMultiple 30k node sites Tens of 3k node sites
  • Built as a platform
  • Model-based hackability• Hosts• Inventory data • IP, hostname, platform, etc.• Resource lists• Resource dependencies• Change events
  • Puppet Facesca keycatalog mancertificate nodecertificate_request parsercertificate_revocation_list pluginconfig reportfacts resourcefile resource_typehelp status
  • MCollective:Infrastructure message bus
  • Puppet Forge404 free, reusable modules
  • Hacking Puppet forFun and Productivity
  • QuestionAuthority
  • Model-based Hackability
  • Exampleshttps://github.com/lak/velocity_2012-Hacking_Puppet https://github.com/puppetlabs/puppetlabs-stdlib/
  • Hack #1: Puppet as you ~/.puppet/puppet.conf
  • Editing files as youCode:Result:
  • Simpler versionCode:Result:
  • Client/serverCode:Server: (and lots more text)Client:
  • Hack #2: Curl speaksPuppet:Curl:
  • Hack #3: Data, and lots of it
  • Data Node 1 Facts The node sends normalized data about itself to the Puppet Master. SSL secure 2 Catalog Puppet uses the Facts to encryption compile a Catalog that on all data specifies how the node transport should be configured.Report 3The nodereports backto Puppetindicating theconfiguration iscomplete, whichis visible in the PuppetPuppet Dashboard. Master 4 Report Collector (Puppet or 3rd party tool) Report can also send data to third party tools.
  • #3.1: Catalog
  • Catalog as JSON{ "data": { "edges": [ { "source": "Stage[main]", "target": "Class[Settings]" }, { "source": "Stage[main]", "target": "Class[main]" }, { "source": "Class[main]", "target": "File[/tmp/local_file]" } ], "resources": [ { "type": "Class", "exported": false, "title": "Settings", "tags": [ "class", "settings" ] },
  • So?Control separately:Resource list:Code:
  • #3.2: Facts
  • Every host, automatically
  • #3.3: Reports
  • EveryChange
  • Node Fact ResourceDependency Change
  • Hack #4: Static Compiler
  • Checksum vs. URLBefore:After:
  • Normal data flows
  • Static compiler, run 1
  • Static compiler, run 2
  • Hack #5: config_versionDefault:After:
  • Extensions
  • Hack #6: Report http://puppetlabs.com/blog/when-puppet-reports-part-1/https://github.com/jamtur01/puppet-growl/blob/master/lib/puppet/reports/growl.rb
  • Hack #6: Report http://puppetlabs.com/blog/when-puppet-reports-part-1/https://github.com/jamtur01/puppet-growl/blob/master/lib/puppet/reports/growl.rb
  • Hack #7: Functionhttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/ functions/flatten.rb
  • Hack #7: Functionhttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/puppet/parser/ functions/flatten.rb
  • Hack #8: Facthttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/facter/ root_home.rb
  • Hack #8: Facthttps://github.com/puppetlabs/puppetlabs-stdlib/blob/master/lib/facter/ root_home.rb
  • #9 and #10: Types and ProvidersExamples from http://www.kartar.net/2010/02/puppet-types-and-providers-are-easy/
  • Exec in Puppet
  • Native resource
  • Native resources areExec:Native:
  • Resource Type
  • Resource Type
  • Resource Type
  • Resource Type
  • Resource Provider
  • Resource Provider
  • Resource Provider
  • Resource Provider
  • Hack #11: Face https://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
  • Setuphttps://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
  • actualwork.com https://github.com/puppetlabs/puppet/blob/2.7rc/lib/puppet/face/config.rb
  • Summary
  • Configurable
  • Controllable Node 1 Facts The node sends normalized data about itself to the Puppet Master. SSL secure 2 Catalog Puppet uses the Facts to encryption compile a Catalog that on all data specifies how the node transport should be configured.Report 3The nodereports backto Puppetindicating theconfiguration iscomplete, whichis visible in the PuppetPuppet Dashboard. Master 4 Report Collector (Puppet or 3rd party tool) Report can also send data to third party tools.
  • Node Fact ResourceDependency Change
  • Hackable
  • Questions?