Puppet, Jenkins & CI

                            SNAP Interactive, Inc.

                                  Presented by:


                               Matthew Barr
                             Technical Architect




Thursday, February 21, 13
Company Overview
                     •      SNAP Interactive, Inc.

                     •      AreYouInterested.com

                     •      Believes it is one of the largest social discovery platforms on the web
                            (based on monthly active users)

                     •      More than 5 million monthly active users

                     •      Over 1 billion total pieces of structured data from its users

                     •      Synced to millions of Facebook profiles

                     •      Receives over 1,000 real-time updates per minute on like actions from
                            Facebook

                     •      Subscription-based business model

                     •      SNAP is publicly traded - Ticker: STVI
                                                                                              www.snap-interactive.com
Thursday, February 21, 13
About me
                     • Team lead for the operations group
                     • Sysadmin for over 15 years
                     • Dev/ops, WebOps, etc.



                                                            www.snap-interactive.com
Thursday, February 21, 13
Continuous Integration
                     •       The practice of frequently integrating one's
                             new or changed code with the existing code
                             repository




                 •          Maintain a     •   Make the       •   Everyone
                            code               build self-        can see the
                            repository         testing            results of the
                                                                  latest build
                 •          Automate       •   Keep the
                            the build          build fast



                                                                      www.snap-interactive.com
Thursday, February 21, 13
Puppet
                     • Puppet Labs
                     • https://puppetlabs.com/
                     • You’re at PuppetNYC :)



                                                 www.snap-interactive.com
Thursday, February 21, 13
Jenkins
                     •      An extendable open source continuous integration
                            server

                     •      http://jenkins-ci.org/




                                                                    www.snap-interactive.com
Thursday, February 21, 13
So, let’s do this!
                     • Puppet Dynamic Environments
                     • Install Jenkins (master)
                     • Install Jenkins slaves (swarm clients)
                     • Mergeatron
                     • Puppet Parsing & Linting
                     • Puppet Testing
                                                                www.snap-interactive.com
Thursday, February 21, 13
Puppet Dynamic Env’s
                     • google: puppet dynamic environments
                     • Git based
                     • Each branch is it’s own environment
                     • Commit hook / web
                     • https://puppetlabs.com/blog/git-workflow-
                            and-puppet-environments/


                                                          www.snap-interactive.com
Thursday, February 21, 13
Jenkins install
                     • puppet module install rtyler-jenkins
                     • Puppet forge module: rtyler/puppet-jenkins
                      • But: this code hasn’t been merged :(
                     • It’s in my github repo:
                      • https://github.com/matthewbarr/puppet-
                            jenkins


                                                          www.snap-interactive.com
Thursday, February 21, 13
Jenkins Master
                 include jenkins
                 jenkins::plugin {'swarm':}
                                        OR
                class {'jenkins':
                   lts => 1,     # Long term Stable version
                   repo => 0, # Disable Jenkins repo config
                 }
               jenkins::plugin {'swarm':}

                                                       www.snap-interactive.com
Thursday, February 21, 13
Jenkins Slave: Easy!


                            include jenkins::slave




                                                     www.snap-interactive.com
Thursday, February 21, 13
Jenkins Slave: Features!
     class {'jenkins::slave':
       masterurl => 'http://jenkins01.ayisnap.com:8080',
       ui_user => 'UI_USER',
       ui_pass => 'UI_PASS',
    # version => '1.8',
       executors => 4,
    # manage_slave_user => 1,
       slave_user => $slave_user,
       slave_uid => '600',
       slave_home => $slave_home,
     }
                                                           www.snap-interactive.com
Thursday, February 21, 13
Github
                     • Private repos for organizations
                     • Pull Requests!




                                                         www.snap-interactive.com
Thursday, February 21, 13
Demo!


                                    www.snap-interactive.com
Thursday, February 21, 13
PR: Swarm plugin code




                            www.snap-interactive.com
Thursday, February 21, 13
Mergeatron
                     • Developed in house
                     • Open Source
                     • Nodejs
                     • Not yet in NPM :(
                     • http://snapinteractive.github.com/
                            mergeatron/


                                                            www.snap-interactive.com
Thursday, February 21, 13
Mergeatron ^2
                     • Monitors github (or other) for a pull
                            request
                     • Triggers a Jenkins “build”
                     • Build can do anything: unit tests, etc



                                                                www.snap-interactive.com
Thursday, February 21, 13
Mergeatron ^3
                     • We use it to monitor 2 repo’s
                      • Web site code
                        • phpunit
                        • phpcs
                        • jshint
                      • Ops repo (Puppet) - puppet parser
                                                        www.snap-interactive.com
Thursday, February 21, 13
Puppet Testing
                     • Puppet Lint
                      • http://puppet-lint.com/
                     • Rspec-Puppet
                      • http://rspec-puppet.com/
                     • ERB Parsing:erb	
  -­‐P	
  -­‐x	
  -­‐T	
  '-­‐'	
  mytempl.erb	
  |	
  ruby	
  -­‐c


                     • Cucumber-puppet
                      • (Discontinued.)
                                                                                              www.snap-interactive.com
Thursday, February 21, 13
Puppet Testing - Future
                     • Parsing is a nice start
                     • Catalog checks can help, especially for
                            supporting systems & edge cases
                     • Actually running it, though, is the only real
                            test that matters.
                     • VM’s, functional / unit tests agains real
                            systems that deliver your app.


                                                               www.snap-interactive.com
Thursday, February 21, 13
Updates
                     • Catalog checker w/ other facts- Ohad Levy:
                            •   https://github.com/ohadlevy/manitest


                     • Github Pull Request Checker for Jenkins
                            •   https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+plugin




                                                                                                                 www.snap-interactive.com
Thursday, February 21, 13
Contact Us
                     •      SNAP Interactive, Inc.
                            www.snap-interactive.com

                     •      Matthew Barr
                            Technical Architect
                            mbarr@snap-interactive.com
                            Twitter & Github:
                            @MatthewBarr




                                                         www.snap-interactive.com
Thursday, February 21, 13

Puppet & Jenkins

  • 1.
    Puppet, Jenkins &CI SNAP Interactive, Inc. Presented by: Matthew Barr Technical Architect Thursday, February 21, 13
  • 2.
    Company Overview • SNAP Interactive, Inc. • AreYouInterested.com • Believes it is one of the largest social discovery platforms on the web (based on monthly active users) • More than 5 million monthly active users • Over 1 billion total pieces of structured data from its users • Synced to millions of Facebook profiles • Receives over 1,000 real-time updates per minute on like actions from Facebook • Subscription-based business model • SNAP is publicly traded - Ticker: STVI www.snap-interactive.com Thursday, February 21, 13
  • 3.
    About me • Team lead for the operations group • Sysadmin for over 15 years • Dev/ops, WebOps, etc. www.snap-interactive.com Thursday, February 21, 13
  • 4.
    Continuous Integration • The practice of frequently integrating one's new or changed code with the existing code repository • Maintain a • Make the • Everyone code build self- can see the repository testing results of the latest build • Automate • Keep the the build build fast www.snap-interactive.com Thursday, February 21, 13
  • 5.
    Puppet • Puppet Labs • https://puppetlabs.com/ • You’re at PuppetNYC :) www.snap-interactive.com Thursday, February 21, 13
  • 6.
    Jenkins • An extendable open source continuous integration server • http://jenkins-ci.org/ www.snap-interactive.com Thursday, February 21, 13
  • 7.
    So, let’s dothis! • Puppet Dynamic Environments • Install Jenkins (master) • Install Jenkins slaves (swarm clients) • Mergeatron • Puppet Parsing & Linting • Puppet Testing www.snap-interactive.com Thursday, February 21, 13
  • 8.
    Puppet Dynamic Env’s • google: puppet dynamic environments • Git based • Each branch is it’s own environment • Commit hook / web • https://puppetlabs.com/blog/git-workflow- and-puppet-environments/ www.snap-interactive.com Thursday, February 21, 13
  • 9.
    Jenkins install • puppet module install rtyler-jenkins • Puppet forge module: rtyler/puppet-jenkins • But: this code hasn’t been merged :( • It’s in my github repo: • https://github.com/matthewbarr/puppet- jenkins www.snap-interactive.com Thursday, February 21, 13
  • 10.
    Jenkins Master include jenkins jenkins::plugin {'swarm':} OR class {'jenkins': lts => 1, # Long term Stable version repo => 0, # Disable Jenkins repo config } jenkins::plugin {'swarm':} www.snap-interactive.com Thursday, February 21, 13
  • 11.
    Jenkins Slave: Easy! include jenkins::slave www.snap-interactive.com Thursday, February 21, 13
  • 12.
    Jenkins Slave: Features! class {'jenkins::slave': masterurl => 'http://jenkins01.ayisnap.com:8080', ui_user => 'UI_USER', ui_pass => 'UI_PASS', # version => '1.8', executors => 4, # manage_slave_user => 1, slave_user => $slave_user, slave_uid => '600', slave_home => $slave_home, } www.snap-interactive.com Thursday, February 21, 13
  • 13.
    Github • Private repos for organizations • Pull Requests! www.snap-interactive.com Thursday, February 21, 13
  • 14.
    Demo! www.snap-interactive.com Thursday, February 21, 13
  • 15.
    PR: Swarm plugincode www.snap-interactive.com Thursday, February 21, 13
  • 16.
    Mergeatron • Developed in house • Open Source • Nodejs • Not yet in NPM :( • http://snapinteractive.github.com/ mergeatron/ www.snap-interactive.com Thursday, February 21, 13
  • 17.
    Mergeatron ^2 • Monitors github (or other) for a pull request • Triggers a Jenkins “build” • Build can do anything: unit tests, etc www.snap-interactive.com Thursday, February 21, 13
  • 18.
    Mergeatron ^3 • We use it to monitor 2 repo’s • Web site code • phpunit • phpcs • jshint • Ops repo (Puppet) - puppet parser www.snap-interactive.com Thursday, February 21, 13
  • 19.
    Puppet Testing • Puppet Lint • http://puppet-lint.com/ • Rspec-Puppet • http://rspec-puppet.com/ • ERB Parsing:erb  -­‐P  -­‐x  -­‐T  '-­‐'  mytempl.erb  |  ruby  -­‐c • Cucumber-puppet • (Discontinued.) www.snap-interactive.com Thursday, February 21, 13
  • 20.
    Puppet Testing -Future • Parsing is a nice start • Catalog checks can help, especially for supporting systems & edge cases • Actually running it, though, is the only real test that matters. • VM’s, functional / unit tests agains real systems that deliver your app. www.snap-interactive.com Thursday, February 21, 13
  • 21.
    Updates • Catalog checker w/ other facts- Ohad Levy: • https://github.com/ohadlevy/manitest • Github Pull Request Checker for Jenkins • https://wiki.jenkins-ci.org/display/JENKINS/GitHub+pull+request+builder+plugin www.snap-interactive.com Thursday, February 21, 13
  • 22.
    Contact Us • SNAP Interactive, Inc. www.snap-interactive.com • Matthew Barr Technical Architect mbarr@snap-interactive.com Twitter & Github: @MatthewBarr www.snap-interactive.com Thursday, February 21, 13