• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
Puppet at GitHub - PuppetConf 2013
 

Puppet at GitHub - PuppetConf 2013

on

  • 2,362 views

"Puppet at GitHub" by Will Farrington Junior Executive Assistant to the Assistant VP of Operations, GitHub, Inc. ...

"Puppet at GitHub" by Will Farrington Junior Executive Assistant to the Assistant VP of Operations, GitHub, Inc.

Presentation Overview: At GitHub, we've got a pretty large Puppet code base to manage all of the GitHub.com infrastructure. It's also pretty old; a little over four and a half years of history lives in our Puppet repo. We've also hired a lot more people over the past few years, going from around 70 people a year ago to more than double that now. This talk focuses on how we continue to embrace the GitHub principles of constantly shipping, iterative improvement, constant experimenting, and no managers to manage our rather large Puppet infrastructure with success.

Speaker Bio: Will is a developer and operations engineer. Will works on system operations at GitHub, where he spends most of his time slinging Puppet to manage GitHub's extensive production environment where he spends a great deal of time on automating, homogenizing, and improving GitHub's development environments. Prior to GitHub, he spent the previous 3 years using Ruby and Puppet to build and scale massive web services with Highgroove Studios and Rails Machine.

Statistics

Views

Total Views
2,362
Views on SlideShare
1,718
Embed Views
644

Actions

Likes
6
Downloads
41
Comments
0

3 Embeds 644

http://puppetlabs.com 642
https://puppetlabs.com 1
http://www.linkedin.com 1

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    Puppet at GitHub - PuppetConf 2013 Puppet at GitHub - PuppetConf 2013 Presentation Transcript

    • Puppet at GitHub
    • @wfarr GitHub Operations Known Aliases: King of Kebabs The Chairman Mr. Caremad !
    • The State of Puppet at GitHub "
    • " The State of Puppet at GitHub github/boxen
    • " The State of Puppet at GitHub ~1.5 years old
    • " The State of Puppet at GitHub open-sourced ~7 months ago
    • " The State of Puppet at GitHub ~240 open-source puppet modules
    • " The State of Puppet at GitHub puppet 3.latest
    • " The State of Puppet at GitHub supports ruby 1.8.7, 1.9.3, 2.0.0
    • " The State of Puppet at GitHub hiera
    • " The State of Puppet at GitHub OS X
    • " The State of Puppet at GitHub Linux support in-progress
    • " The State of Puppet at GitHub # tomorrow @ 2:20pm
    • " The State of Puppet at GitHub github/puppet
    • " The State of Puppet at GitHub $
    • " The State of Puppet at GitHub ~5 years old
    • " The State of Puppet at GitHub 0.24.x — 2.7.x
    • " The State of Puppet at GitHub 121 modules
    • " The State of Puppet at GitHub ~280k lines of code
    • " The State of Puppet at GitHub every single employee has commit access
    • " The State of Puppet at GitHub 15.5k commits to master past 12 months
    • " The State of Puppet at GitHub by 86 contributors
    • " The State of Puppet at GitHub % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % %
    • " The State of Puppet at GitHub % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % 42.5%
    • " The State of Puppet at GitHub % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % % now with ops taken out
    • " The State of Puppet at GitHub 0 100 200 300 400 2012-08-25 2012-10-20 2012-12-15 2013-02-09 2013-04-06 2013-06-01 2013-07-27 commits on master per week, last 12 months
    • " The State of Puppet at GitHub 0 1000 2000 3000 4000 total commits by author past year
    • " The State of Puppet at GitHub 0 175 350 525 700 total commits by author past year, except ops
    • " The State of Puppet at GitHub 0 175 350 525 700 total commits by authors with >10 commits, past year, except ops
    • " The State of Puppet at GitHub single puppetmaster
    • " The State of Puppet at GitHub rubygems
    • " The State of Puppet at GitHub ruby 1.8.7
    • " The State of Puppet at GitHub unicorn
    • " The State of Puppet at GitHub puppet 2.7.latest
    • " The State of Puppet at GitHub ~600 nodes
    • " The State of Puppet at GitHub run hourly via crond
    • " The State of Puppet at GitHub puppetdb
    • " The State of Puppet at GitHub nagiosdb
    • " The State of Puppet at GitHub "yo puppetdb, gimme all your nagios::object::* resources so I can update this nagios config"
    • " The State of Puppet at GitHub filtergendb
    • " The State of Puppet at GitHub "yo puppetdb, gimme all the filtergen::rule resources I would realize so I can update this filtergen config"
    • " The State of Puppet at GitHub ⚡ puppetdb ⚡
    • " The State of Puppet at GitHub as it turns out, an api call is faster than running puppet on a host
    • " The State of Puppet at GitHub gpanel
    • " The State of Puppet at GitHub "Imagine Puppet Dashboard meets Razor and went on a weekend trip to the beach with the Heroku API and drank epic amounts of blue drink"
    • " The State of Puppet at GitHub aka we reinvented our own, smaller version of Foreman
    • " The State of Puppet at GitHub inventory
    • " The State of Puppet at GitHub
    • " The State of Puppet at GitHub
    • " The State of Puppet at GitHub app configuration
    • " The State of Puppet at GitHub
    • " The State of Puppet at GitHub versioning of configuration values
    • " The State of Puppet at GitHub Create nil => 1 Update 1 => 2 Delete 2 => nil
    • " The State of Puppet at GitHub let's make credential rolling less awful
    • " The State of Puppet at GitHub provisioning
    • " The State of Puppet at GitHub provisioning is typically awful
    • " The State of Puppet at GitHub we sprinkled in some ChatOps
    • " The State of Puppet at GitHub
    • " The State of Puppet at GitHub a little bit later...
    • " The State of Puppet at GitHub <MACHINE>
    • " The State of Puppet at GitHub enc
    • " The State of Puppet at GitHub we have the dumbest ENC out there
    • " The State of Puppet at GitHub ⋆
    • " The State of Puppet at GitHub # /usr/local/sbin/fetch_gpanel_enc fe1.rs.github.com --- parameters: gpanel_cabinet: D20-13 gpanel_enabled: true gpanel_monitored: false
    • " The State of Puppet at GitHub we never specify classes via the ENC
    • " The State of Puppet at GitHub any variables we pass through are prefixed with gpanel_
    • How GitHub writes Puppet (
    • ( How GitHub Writes Puppet rodjek/puppet-lint
    • ( How GitHub Writes Puppet
    • ( How GitHub Writes Puppet if you aren't using puppet-lint to audit your puppet codebase, you are doing it wrong
    • ( How GitHub Writes Puppet puppet-lint enforces the Puppet Labs style guide
    • ( How GitHub Writes Puppet puppet-lint can even fix a ton of linter errors for you
    • ( How GitHub Writes Puppet put it in a pre-commit hook
    • ( How GitHub Writes Puppet $ git commit -am "can't lint this" modules/github/manifests/role/redis.pp: syntax ok modules/github/manifests/role/redis.pp - WARNING: => is not properly aligned on line 118 1 errors found, aborting commit.
    • ( How GitHub Writes Puppet and then buy rodjek a beer
    • ( How GitHub Writes Puppet rodjek/rspec-puppet
    • ( How GitHub Writes Puppet
    • ( How GitHub Writes Puppet if you aren't writing tests for your puppet code before running it on a server, you are doing it wrong
    • ( How GitHub Writes Puppet use whatever framework/library
    • ( How GitHub Writes Puppet rspec-puppet just happens to be a pretty good one
    • ( How GitHub Writes Puppet put it in a pre-commit hook
    • ( How GitHub Writes Puppet $ git commit -am "tests dont pass but whatever lol" 1) Expected redis::server would include class "more_than_a_single_c_thread" 1 failures encountered, aborting commit.
    • ( How GitHub Writes Puppet and then buy rodjek another beer
    • ( How GitHub Writes Puppet node definitions
    • ( How GitHub Writes Puppet we don't use an ENC to describe node classes
    • ( How GitHub Writes Puppet node /^github-redisd+/ { class { 'github::role::polling_redis': enabled => $::gpanel_enabled, environment => $::gpanel_environment, private_ipv4 => $::ipaddress, } }
    • ( How GitHub Writes Puppet abstractions all the way down
    • ( How GitHub Writes Puppet treat your site classes as cascades down to your dist classes
    • ( How GitHub Writes Puppet class redis::server( # params ) { class { 'redis::config': ... } -> class { 'redis::package': ... } ~> class { 'redis::service': ... } }
    • ( How GitHub Writes Puppet class github::redis( # params ) inherits github::defaults { $memory = $environment ? { 'stg' => '2G', default => $half_memory_gb } class { 'redis::server': ... } }
    • ( How GitHub Writes Puppet class github::role::polling_redis( # params ) { class { 'github::redis': # overrides based on specific node } }
    • ( How GitHub Writes Puppet augeas
    • ( How GitHub Writes Puppet you know what's not awesome?
    • ( How GitHub Writes Puppet an erb template that requires your class to take 52876423 parameters so you can configure every possible value in my.cnf
    • ( How GitHub Writes Puppet augeas { 'my.cnf/performance': context => '/files/etc/mysql/my.cnf/mysqld', changes => [ # automatic dump/restore 'set innodb_auto_lru_dump 18000', # innodb "set innodb_buffer_pool_size ${innodb_buffer_pool_size}", 'set innodb_log_file_size 256M', 'set innodb_log_buffer_size 8M', 'set innodb_lazy_drop_table 1', ], require => Percona::Server[$::fqdn] }
    • ( How GitHub Writes Puppet it can seem complex and scary
    • ( How GitHub Writes Puppet it is an amazing tool
    • How GitHub ships Puppet )
    • ) How GitHub Ships Puppet continuous integration
    • ) How GitHub Ships Puppet commit gets pushed
    • ) How GitHub Ships Puppet jenkins runs the test suite
    • ) How GitHub Ships Puppet status gets posted back to GitHub.com
    • ) How GitHub Ships Puppet tmm1/test-queue
    • ) How GitHub Ships Puppet
    • ) How GitHub Ships Puppet remembers how to better parallelize tests
    • ) How GitHub Ships Puppet continuous deployment
    • ) How GitHub Ships Puppet tests passed for commit, Hubot auto-deploys
    • ) How GitHub Ships Puppet branch deploy everything
    • ) How GitHub Ships Puppet a lot of shops have a few environments
    • ) How GitHub Ships Puppet testing staging production
    • ) How GitHub Ships Puppet currently we have 181 environments
    • ) How GitHub Ships Puppet that's not the same as 181 nodes not running production
    • ) How GitHub Ships Puppet we only have 5 of those
    • ) How GitHub Ships Puppet Hubot automatically merges the master branch before allowing any branch to deploy
    • ) How GitHub Ships Puppet ChatOps
    • ) How GitHub Ships Puppet 202 employees 100% access to Puppet 100% trust
    • ) How GitHub Ships Puppet the list of roles you'd never think would touch production puppet webkit developer core git developers core ruby developers svn developers search developer frontend designers windows developers mac developers support enterprise sales
    • ) How GitHub Ships Puppet every puppet run happens in chat
    • ) How GitHub Ships Puppet all puppet output goes to chat
    • ) How GitHub Ships Puppet everyone can see everything
    • ) How GitHub Ships Puppet everyone can do anything
    • ) How GitHub Ships Puppet hands-on learning, by accident
    • ) How GitHub Ships Puppet eliminate disruptive questions
    • The Future of Puppet at GitHub *
    • * The Future of Puppet at GitHub puppet 3.x
    • * The Future of Puppet at GitHub upgrading a large, old codebase from 2.7 to 3 is really painful
    • * The Future of Puppet at GitHub we've been talking about it for a year
    • * The Future of Puppet at GitHub we're still trying to push forward, but...
    • * The Future of Puppet at GitHub helping maintain security fixes for 2.7
    • * The Future of Puppet at GitHub goal is to be on 3.2 by end of year
    • * The Future of Puppet at GitHub ruby 2.0
    • * The Future of Puppet at GitHub moar faster rubby
    • * The Future of Puppet at GitHub we get this for free when we move to 3
    • * The Future of Puppet at GitHub mcollective
    • * The Future of Puppet at GitHub aka the story of github/shell and the wonders of ssh in a for-loop
    • * The Future of Puppet at GitHub there are a lot of cool things about mcollective
    • * The Future of Puppet at GitHub I think it will be a part of our stack in the future
    • * The Future of Puppet at GitHub but you can still go pretty damn far with bash and ssh
    • * The Future of Puppet at GitHub even more puppetdb tooling
    • * The Future of Puppet at GitHub waiting for multiple runs to converge exported resources is painful
    • * The Future of Puppet at GitHub use the puppetdb API to skip all that
    • * The Future of Puppet at GitHub even more gpanel integration
    • * The Future of Puppet at GitHub "databags"
    • * The Future of Puppet at GitHub "node search"
    • * The Future of Puppet at GitHub steal good ideas from other tools
    • closing thoughts
    • "all software is terrible" — anyone who's worked with software long enough
    • "no software is better than no software" — rtomayko
    • the plumbing doesn't matter when all you care about is the porcelain
    • write some damned good porcelain
    • THIS IS RODJEK
    • HE'S NORMALLY NOT THIS BLURRY
    • HE IS ALSO NOT MY BROTHER, BUT YOU REALLY SHOULD BUY HIM A BEER
    • YOU SHOULD BUY THIS MAN A LOT OF BEER
    • THANKS https://speakerdeck.com/wfarr/ puppet-at-github-puppetconf-2013 !