Puppet @ Citygrid


        PuppetCamp LA 2012
Who am I ?



My name is Julien Rottenberg
Recovering System Administrator
Puppet user since 2009 (at GN)
I like to write code and answer questions
Puppet promises

                          No need to ssh to a
                          server to understand
                          how it works



Look at the puppet code
"Those who cannot remember the
past are condemned to repeat it."
                                     George Santayana




literally... go to a server, run `history` as root.
Puppet promises



Guarantee that a server is setup exactly the
same way on all environments
Puppet promises
Puppet promises



All the building steps
are auditable - SDLC
Drift management

Open knowledge, no
blackbox
Puppet promises

    and finally...
Toolset


    Open source tools :

    ●   No need to reinvent the wheel
    ●   You are not alone with 'that problem',
        get help
Toolset
Toolset


 puppet doc
 ● validate code (puppet parser validate)
 ● produce html doc

 Running on commit through Jenkins ~ 10
 sec
Toolset
Toolset



  puppet-lint == coding standard




          Takes ~ 2 min
          Part of the 'Ci-Game'
Toolset
trunk = work in progress - could break but fix
ASAP
production = tested code - do not break
Toolset    Trunk --> Production (CherryPicking)
                          We use svnmerge.py



   Code in progress = technical debt
   It needs to be merged and deployed or
   reverted

   Fail the build if pending Cherry Picks > 30

   Can't CherryPick yourself ~ code review
Toolset
          Svn even with svnmerge has limitations




                                  Git ?
Toolset
  Vagrant




Sits on top of Virtualbox
Allows to 'go crazy'
Test, experiment, share configured box
$ vagrant provision == puppet apply
Toolset
nVentory




As an External Node Classifier
Classes:
    - default_install
    - gcd
Toolset
  nVentory




Helper to build configuration files, etc.
ie replacement for Stored Config
Toolset
Foreman

Api for puppet status, visibility of puppetrun




 (Failed attempt on ENC - probably pebkac)
Toolset

 Man ! that's a lot of tools !1!!

And it's fine, design by contract.

Each application does one thing
and was designed to do that
thing only (so does it well)
Pipe them
                                     Dennis Ritchie
Toolset
Boto for

 ●   ec2
 ●   route53
 ●   elb
 ●   sqs
 ●   sns



 River dolphin native to the amazon WP
Culture cloud
EC2 glitches, expect them

It used to `just work` :
 ●  gmond (split view dns side effect, using
    gmetric)
 ●  jmx (need to specify public name at runtime)
 ●  ELB pseudo load balancer
 ●  Bad neighbors
Culture cloud
Culture cloud

# Build 10 nodes
$ create_ec2_instance.py -e dev -s m1.medium -c thd -n 10

# assign them to the hadoop::datanode class (specific install on top of our vanilla)
$ nv --name aws1devthd --addtonodegroup hadoop::datanode

# send traffic & enable monitoring (configs to be updated on the next puppet run)
$ nv --name aws1devthd --set status=inservice



  One AMI to rule them all (map set per region)
Culture Collaborate
Culture Collaborate



 "If the company's doing well and people
 don't hate each other, you're probably
 doing ok."
             John Allspaw, speaking at Devopsdays 2011 MV.
You guys !

Make it better :

- Integrated function for 'secrets' (no get_var or
hiera + gpg)
Something like an encrypted data luggage

 - puppet agent stability, puppetd vs cron ?
Guys !
Yesterday...
Two last things...


 On call rotation side effect


 Don't be a human keyboard
Shameless plug


We aggregate advertising & content from local
businesses and distribute across our network of
web & mobile sites



  ●   Ingest : Hadoop & Cassandra
  ●   Serve : Solr & MongoDb + Tomcat
  ●   Code : Java mostly
Q/A                                              This page : goo.gl/soxEs


puppet doc : http://docs.puppetlabs.com/guides/style_guide.html#puppet-doc
puppet-lint : http://puppet-lint.com/
Jenkins : http://jenkins-ci.org/
svn strategy : http://weblogs.java.net/blog/johnsmart/archive/2009/05/ci_branching_an.html
svnmerge.py : http://www.orcaware.com/svn/wiki/Svnmerge.py
vagrant : http://vagrantup.com/
nventory : http://sourceforge.net/apps/trac/nventory/wiki
foreman : http://theforeman.org/
boto : https://github.com/boto/boto

Don't be an Ahole : http://www.hollenback.net/DevOpsMeansDontBeAnAhole
Developers on call - Mitchell Hashimoto : http://vimeo.com/31367609
Don't be a human keyboard : http://sysadvent.blogspot.com/2010/12/day-13-dont-be-human-keyboard.html

Citygrid is hiring : http://careers.citygrid.com/

Puppet Master LA User Group : http://www.meetup.com/LAPMUG/

Puppet@Citygrid - Julien Rottenberg - PuppetCamp LA '12

  • 1.
    Puppet @ Citygrid PuppetCamp LA 2012
  • 2.
    Who am I? My name is Julien Rottenberg Recovering System Administrator Puppet user since 2009 (at GN) I like to write code and answer questions
  • 3.
    Puppet promises No need to ssh to a server to understand how it works Look at the puppet code
  • 4.
    "Those who cannotremember the past are condemned to repeat it." George Santayana literally... go to a server, run `history` as root.
  • 5.
    Puppet promises Guarantee thata server is setup exactly the same way on all environments
  • 6.
  • 7.
    Puppet promises All thebuilding steps are auditable - SDLC Drift management Open knowledge, no blackbox
  • 8.
    Puppet promises and finally...
  • 9.
    Toolset Open source tools : ● No need to reinvent the wheel ● You are not alone with 'that problem', get help
  • 10.
  • 11.
    Toolset puppet doc ● validate code (puppet parser validate) ● produce html doc Running on commit through Jenkins ~ 10 sec
  • 12.
  • 13.
    Toolset puppet-lint== coding standard Takes ~ 2 min Part of the 'Ci-Game'
  • 14.
    Toolset trunk = workin progress - could break but fix ASAP production = tested code - do not break
  • 15.
    Toolset Trunk --> Production (CherryPicking) We use svnmerge.py Code in progress = technical debt It needs to be merged and deployed or reverted Fail the build if pending Cherry Picks > 30 Can't CherryPick yourself ~ code review
  • 16.
    Toolset Svn even with svnmerge has limitations Git ?
  • 17.
    Toolset Vagrant Sitson top of Virtualbox Allows to 'go crazy' Test, experiment, share configured box $ vagrant provision == puppet apply
  • 18.
    Toolset nVentory As an ExternalNode Classifier Classes: - default_install - gcd
  • 19.
    Toolset nVentory Helperto build configuration files, etc. ie replacement for Stored Config
  • 20.
    Toolset Foreman Api for puppetstatus, visibility of puppetrun (Failed attempt on ENC - probably pebkac)
  • 21.
    Toolset Man !that's a lot of tools !1!! And it's fine, design by contract. Each application does one thing and was designed to do that thing only (so does it well) Pipe them Dennis Ritchie
  • 22.
    Toolset Boto for ● ec2 ● route53 ● elb ● sqs ● sns River dolphin native to the amazon WP
  • 23.
    Culture cloud EC2 glitches,expect them It used to `just work` : ● gmond (split view dns side effect, using gmetric) ● jmx (need to specify public name at runtime) ● ELB pseudo load balancer ● Bad neighbors
  • 24.
  • 25.
    Culture cloud # Build10 nodes $ create_ec2_instance.py -e dev -s m1.medium -c thd -n 10 # assign them to the hadoop::datanode class (specific install on top of our vanilla) $ nv --name aws1devthd --addtonodegroup hadoop::datanode # send traffic & enable monitoring (configs to be updated on the next puppet run) $ nv --name aws1devthd --set status=inservice One AMI to rule them all (map set per region)
  • 26.
  • 27.
    Culture Collaborate "Ifthe company's doing well and people don't hate each other, you're probably doing ok." John Allspaw, speaking at Devopsdays 2011 MV.
  • 28.
    You guys ! Makeit better : - Integrated function for 'secrets' (no get_var or hiera + gpg) Something like an encrypted data luggage - puppet agent stability, puppetd vs cron ?
  • 29.
  • 30.
  • 31.
    Two last things... On call rotation side effect Don't be a human keyboard
  • 32.
    Shameless plug We aggregateadvertising & content from local businesses and distribute across our network of web & mobile sites ● Ingest : Hadoop & Cassandra ● Serve : Solr & MongoDb + Tomcat ● Code : Java mostly
  • 33.
    Q/A This page : goo.gl/soxEs puppet doc : http://docs.puppetlabs.com/guides/style_guide.html#puppet-doc puppet-lint : http://puppet-lint.com/ Jenkins : http://jenkins-ci.org/ svn strategy : http://weblogs.java.net/blog/johnsmart/archive/2009/05/ci_branching_an.html svnmerge.py : http://www.orcaware.com/svn/wiki/Svnmerge.py vagrant : http://vagrantup.com/ nventory : http://sourceforge.net/apps/trac/nventory/wiki foreman : http://theforeman.org/ boto : https://github.com/boto/boto Don't be an Ahole : http://www.hollenback.net/DevOpsMeansDontBeAnAhole Developers on call - Mitchell Hashimoto : http://vimeo.com/31367609 Don't be a human keyboard : http://sysadvent.blogspot.com/2010/12/day-13-dont-be-human-keyboard.html Citygrid is hiring : http://careers.citygrid.com/ Puppet Master LA User Group : http://www.meetup.com/LAPMUG/